00000100:00100000:3.0:1780298639.018103:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf348318a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777604628224:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298639.018227:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.018857:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348318a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777604628224:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298639.018872:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298639.018945:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777608994048:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1082us (1417us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298639.018951:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14635 00000100:00100000:3.0:1780298639.019104:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777608994304 00000100:00100000:3.0:1780298639.019109:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777608994304 00002000:00010000:3.0:1780298639.019176:0:8364:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (5903857->18446744073709551615) 00002000:00100000:3.0:1780298639.019184:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24310ce00 with x1866777608994304 ext(5903857->18446744073709551615) 00002000:00010000:3.0:1780298639.019197:0:8364:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24310ce00. 00000100:00100000:3.0:1780298639.019222:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777608994432 00000100:00100000:3.0:1780298639.019226:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777608994432 00002000:00010000:3.0:1780298639.019239:0:8364:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4194304->18446744073709551615) 00002000:00100000:3.0:1780298639.019244:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24310c380 with x1866777608994432 ext(4194304->18446744073709551615) 00002000:00010000:3.0:1780298639.019251:0:8364:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24310c380. 00000100:00100000:3.0:1780298639.021533:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.021606:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.023726:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777608994944 00000100:00100000:3.0:1780298639.023736:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777608994944 00000100:00100000:3.0:1780298639.023821:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7115 00000100:00100000:3.0:1780298639.023835:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777608994944:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.023850:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.023866:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007dc48e1c/0xaf35132e6302fa31 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf198 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.023934:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34831a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604628736:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.023995:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.024754:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34831a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604628736:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.024785:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007dc48e1c/0xaf35132e6302fa31 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf198 expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.024834:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007dc48e1c/0xaf35132e6302fa31 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf198 expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.024872:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830c800, rc 0) 00000100:00100000:3.0:1780298639.024907:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777608994944:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1078us (1262us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.024916:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7115 00000100:00100000:3.0:1780298639.026007:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777608995200 00000100:00100000:3.0:1780298639.026011:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777608995200 00000100:00100000:3.0:1780298639.026083:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777608995328 00000100:00100000:3.0:1780298639.026088:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777608995328 00000100:00100000:3.0:1780298639.028058:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777608995456 00000100:00100000:3.0:1780298639.028063:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777608995456 00000100:00100000:3.0:1780298639.028092:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7119 00000100:00100000:3.0:1780298639.028098:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244813100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777608995456:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.028108:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.028117:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e4811bb2/0xaf35132e6302fa4d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf1b4 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.028153:0:19992:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e8a7e8eb/0xaf35132e6302f935 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf0c6 expref: 7 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.028174:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298638 v=(94 16) 00010000:00010000:3.0:1780298639.028177:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298639 v=(95 16) 00010000:00010000:3.0:1780298639.028182:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e4811bb2/0xaf35132e6302fa4d lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2400256->2412543] (req 2400256->2412543) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf1b4 expref: 11 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.030096:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.034103:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777608996096 00000100:00100000:3.0:1780298639.034107:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777608996096 00000100:00100000:3.0:1780298639.034149:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7121 00000100:00100000:3.0:1780298639.034155:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244811880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777608996096:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.034162:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.034169:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007dc48e1c/0xaf35132e6302fa69 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf1c9 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.034192:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf348319180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604629504:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.034224:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.034851:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348319180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604629504:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.034863:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007dc48e1c/0xaf35132e6302fa69 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf1c9 expref: 8 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.034872:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007dc48e1c/0xaf35132e6302fa69 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf1c9 expref: 8 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.034878:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830c800, rc 0) 00000100:00100000:3.0:1780298639.034905:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244811880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777608996096:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 751us (834us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.034910:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7121 00000100:00100000:3.0:1780298639.042063:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777608996864 00000100:00100000:3.0:1780298639.042067:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777608996864 00000100:00100000:3.0:1780298639.042099:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7125 00000100:00100000:3.0:1780298639.042107:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343964700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777608996864:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.042115:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.042140:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a3e0d5df/0xaf35132e6302fa7e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf1de expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.042170:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf348319180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604629760:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.042219:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.042857:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348319180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604629760:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.042875:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a3e0d5df/0xaf35132e6302fa7e lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf1de expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.042885:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a3e0d5df/0xaf35132e6302fa7e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf1de expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.042921:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830de00, rc 0) 00000100:00100000:3.0:1780298639.042949:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343964700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777608996864:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 842us (920us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.042959:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7125 00000100:00100000:3.0:1780298639.043904:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7127 00000100:00100000:3.0:1780298639.043919:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343966a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777608997120:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.043927:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.043935:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001011cd71/0xaf35132e6302fa8c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf1ec expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.048459:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.050373:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.050453:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.052182:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.052202:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.053802:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.054574:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.055570:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.056910:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.058308:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.059272:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.060310:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.061240:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.064170:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.064188:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.066049:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.066179:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.067926:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.067943:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.069808:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.071766:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609000704 00000100:00100000:3.0:1780298639.071771:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609000704 00000100:00100000:3.0:1780298639.073655:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.073700:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.074473:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343967100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604633088:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.074490:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003b0cbb9d/0xaf35132e6302fb34 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf294 expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.074506:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b0cbb9d/0xaf35132e6302fb34 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf294 expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.074515:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b800, rc 0) 00000100:00100000:3.0:1780298639.074537:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609000960:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 784us (837us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.074542:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7151 00000100:00100000:3.0:1780298639.075396:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.076153:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24339ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604633216:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298639.076178:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000342a3832/0xaf35132e6302fb3b lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3616768->3624959] (req 3616768->3624959) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf29b expref: 8 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.076187:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244f3a600, rc 0) 00000100:00100000:3.0:1780298639.076207:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:29360:x1866777609001088:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 747us (811us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298639.076212:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7152 00000100:00100000:3.0:1780298639.078669:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24339f480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777604633344:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298639.078678:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298639.078742:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343966a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2399:x1866777609001344:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 767us (856us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298639.078753:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14639 00000100:00100000:3.0:1780298639.079981:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609001728 00000100:00100000:3.0:1780298639.079986:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609001728 00000100:00100000:3.0:1780298639.081741:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609001856 00000100:00100000:3.0:1780298639.081745:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609001856 00000100:00100000:3.0:1780298639.081779:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609001984 00000100:00100000:3.0:1780298639.081782:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609001984 00000100:00100000:3.0:1780298639.088757:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.090057:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.090080:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.090209:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5403 00000100:00100000:3.0:1780298639.090217:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609003392:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:3.0:1780298639.090529:0:19949:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4194304 -> 0 00002000:00010000:3.0:1780298639.090532:0:19949:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298639 -> 1780298639 00002000:00010000:3.0:1780298639.090534:0:19949:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298639 -> 1780298639 00002000:00010000:3.0:1780298639.090537:0:19949:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298639 -> 1780298639 00002000:00010000:3.0:1780298639.090539:0:19949:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 128 -> 0 00000100:00100000:3.0:1780298639.090560:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609003392:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 345us (504us total) trans 4294969040 rc 0/0 00000100:00100000:3.0:1780298639.090566:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5403 00002000:00010000:3.0:1780298639.090569:0:19949:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00002000:00100000:3.0:1780298639.090571:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fc700 with x1866777609003392 ext(0->18446744073709551615) 00002000:00010000:3.0:1780298639.090576:0:19949:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf3481fc700. 00000100:00100000:3.0:1780298639.092257:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609003648 00000100:00100000:3.0:1780298639.092263:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609003648 00000100:00100000:3.0:1780298639.092292:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7159 00000100:00100000:3.0:1780298639.092300:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609003648:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.092322:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.092331:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007dc48e1c/0xaf35132e6302fb88 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf2d3 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.092360:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481ff800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604635008:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.092399:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.093174:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343966a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604634880:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.093241:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e7fd0135/0xaf35132e6302fb81 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf2da expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.093251:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7fd0135/0xaf35132e6302fb81 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf2da expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.093258:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9be00, rc 0) 00000100:00100000:3.0:1780298639.093280:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609003776:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 977us (1041us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.093285:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7160 00000100:00100000:3.0:1780298639.094164:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.099041:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609004288 00000100:00100000:3.0:1780298639.099045:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609004288 00010000:00010000:3.0:1780298639.099067:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000b4561703/0xaf35132e6302fb65 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf2be expref: 10 pid: 8356 timeout: 928 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.099085:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14645 00000100:00100000:3.0:1780298639.099090:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f75a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609004288:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298639.099103:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298639.099117:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000b4561703/0xaf35132e6302fb65 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf2be expref: 10 pid: 8356 timeout: 928 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.099147:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000b4561703/0xaf35132e6302fb65 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf2be expref: 10 pid: 8356 timeout: 928 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.099163:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b4561703/0xaf35132e6302fb65 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf2be expref: 10 pid: 8356 timeout: 928 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.099171:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298639 v=(139 22) 00010000:00010000:3.0:1780298639.099173:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298639 v=(139 22) 00010000:00010000:3.0:1780298639.099179:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000003b0cbb9d/0xaf35132e6302fb9d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [5926912->39481343] (req 5926912->5931007) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf2ef expref: 12 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.099185:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003b0cbb9d/0xaf35132e6302fb9d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [5926912->39481343] (req 5926912->5931007) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf2ef expref: 12 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.099201:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000003b0cbb9d/0xaf35132e6302fb9d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [5926912->39481343] (req 5926912->5931007) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf2ef expref: 12 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.099208:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fca80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777604635520:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298639.099238:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733936c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.099872:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fca80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777604635520:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298639.099880:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298639.099905:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f75a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609004288:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 815us (900us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298639.099911:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14645 00000100:00100000:3.0:1780298639.101583:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609004544 00000100:00100000:3.0:1780298639.101588:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609004544 00000100:00100000:3.0:1780298639.101617:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7164 00000100:00100000:3.0:1780298639.101626:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f759880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609004544:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.101634:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.101640:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f8fcaf61/0xaf35132e6302fbb2 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf2fd expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.101667:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604635648:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.101700:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733936c0 going to sleep for 11 seconds 00002000:00010000:3.0:1780298639.102256:0:8358:0:(ofd_lvb.c:259:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size: 1253751 -> 5930564 00000100:00100000:3.0:1780298639.102266:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604635648:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.102278:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f8fcaf61/0xaf35132e6302fbb2 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf2fd expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.102288:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f8fcaf61/0xaf35132e6302fbb2 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf2fd expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.102294:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830c400, rc 0) 00000100:00100000:3.0:1780298639.102325:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f759880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609004544:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 702us (771us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.102333:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7164 00000100:00100000:3.0:1780298639.109309:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609005184 00000100:00100000:3.0:1780298639.109329:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609005184 00000100:00100000:3.0:1780298639.109355:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609005312 00000100:00100000:3.0:1780298639.109357:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609005312 00000100:00100000:3.0:1780298639.109385:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7169 00000100:00100000:3.0:1780298639.109399:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255512300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609005312:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.109409:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.109421:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a3e0d5df/0xaf35132e6302fbd5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf320 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.109450:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604636288:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.109487:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733936c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.110224:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604636288:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.110269:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a3e0d5df/0xaf35132e6302fbd5 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf320 expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.110279:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a3e0d5df/0xaf35132e6302fbd5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf320 expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.110285:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830de00, rc 0) 00000100:00100000:3.0:1780298639.110305:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255512300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609005312:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 912us (995us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.110310:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7169 00000100:00100000:3.0:1780298639.111215:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.113777:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.115051:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609005952 00000100:00100000:3.0:1780298639.115055:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609005952 00010000:00010000:3.0:1780298639.115074:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000003b0cbb9d/0xaf35132e6302fb9d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [5926912->39481343] (req 5926912->5931007) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf2ef expref: 15 pid: 12101 timeout: 928 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.115096:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14646 00000100:00100000:3.0:1780298639.115101:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2398:x1866777609005952:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298639.115108:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:3.0:1780298639.115115:0:6462:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 472 -> 480 00010000:00010000:3.0:1780298639.115119:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000003b0cbb9d/0xaf35132e6302fb9d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [5926912->39481343] (req 5926912->5931007) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf2ef expref: 15 pid: 12101 timeout: 928 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.115146:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000003b0cbb9d/0xaf35132e6302fb9d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [5926912->39481343] (req 5926912->5931007) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf2ef expref: 15 pid: 12101 timeout: 928 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.115159:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003b0cbb9d/0xaf35132e6302fb9d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [5926912->39481343] (req 5926912->5931007) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf2ef expref: 15 pid: 12101 timeout: 928 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.115167:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298639 v=(140 22) 00010000:00010000:3.0:1780298639.115170:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298639 v=(140 22) 00010000:00010000:3.0:1780298639.115175:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e6302fbea lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [5955584->39510015] (req 5955584->5963775) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf335 expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.115181:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e6302fbea lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [5955584->39510015] (req 5955584->5963775) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf335 expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.115197:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e6302fbea lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [5955584->39510015] (req 5955584->5963775) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf335 expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.115213:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fc380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777604636800:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298639.115244:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733936c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.116386:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609006208 00000100:00100000:3.0:1780298639.116391:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609006208 00000100:00100000:3.0:1780298639.116419:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7173 00000100:00100000:3.0:1780298639.116426:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343967100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609006208:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.116437:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.116448:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f8fcaf61/0xaf35132e6302fbf1 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf343 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.116520:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604636928:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.116560:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.117032:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604636928:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.117046:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f8fcaf61/0xaf35132e6302fbf1 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf343 expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.117059:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f8fcaf61/0xaf35132e6302fbf1 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf343 expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.117069:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830c400, rc 0) 00000100:00100000:3.0:1780298639.117160:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343967100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609006208:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 733us (851us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.117165:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7173 00000100:00100000:3.0:1780298639.118101:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.120621:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.121600:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.124369:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609007360 00000100:00100000:3.0:1780298639.124375:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609007360 00000100:00100000:3.0:1780298639.124411:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7181 00000100:00100000:3.0:1780298639.124419:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609007360:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.124428:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.124436:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f8fcaf61/0xaf35132e6302fc29 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf374 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.124462:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2556a9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604638080:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.124487:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.128022:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609007744 00000100:00100000:3.0:1780298639.128027:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609007744 00000100:00100000:3.0:1780298639.128049:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7184 00000100:00100000:3.0:1780298639.128055:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243cd5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:29360:x1866777609007744:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.128062:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.128068:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a3e0d5df/0xaf35132e6302fc3e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf389 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.128080:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298639 v=(140 22) 00010000:00010000:3.0:1780298639.128083:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298639 v=(140 22) 00010000:00010000:3.0:1780298639.128089:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a3e0d5df/0xaf35132e6302fc3e lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1331200->5955583] (req 1331200->1335295) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf389 expref: 15 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.128097:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a3e0d5df/0xaf35132e6302fc3e lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1331200->5955583] (req 1331200->1335295) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf389 expref: 15 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.128109:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830de00, rc 0) 00000100:00100000:3.0:1780298639.128150:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243cd5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:29360:x1866777609007744:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 95us (154us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298639.128156:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7184 00000100:00100000:3.0:1780298639.130047:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609008000 00000100:00100000:3.0:1780298639.130051:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609008000 00002000:00100000:3.0:1780298639.130067:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (1335296->1339391) req@ffff9cf243cd5c00 x1866777609008000/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:345/0 lens 488/0 e 0 to 0 dl 1780298650 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298639.130074:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243cd5c00 with x1866777609008000 ext(1335296->1339391) 00002000:00010000:3.0:1780298639.130076:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf243cd5c00 00000100:00100000:3.0:1780298639.130090:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5407 00000100:00100000:3.0:1780298639.130095:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243cd5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2397:x1866777609008000:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298639.131071:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609008256 00000100:00100000:3.0:1780298639.131075:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609008256 00000100:00100000:3.0:1780298639.132986:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7188 00000100:00100000:3.0:1780298639.132996:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343964700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609008512:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.133004:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.133011:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e6302fc5a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf3a5 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.133039:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2556ab480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604638848:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.134705:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.135534:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243cd7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604639104:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.135565:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e6302fc68 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf3b3 expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.135583:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e6302fc68 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf3b3 expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.135601:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3800, rc 0) 00000100:00100000:3.0:1780298639.135623:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609008768:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 871us (919us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.135629:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7190 00000100:00100000:3.0:1780298639.137256:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609009024 00000100:00100000:3.0:1780298639.137259:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609009024 00010000:00010000:3.0:1780298639.137273:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000991eb57d/0xaf35132e6302fc0d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1564672->35119103] (req 1564672->1576959) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf358 expref: 12 pid: 9128 timeout: 928 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.137291:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14648 00000100:00100000:3.0:1780298639.137296:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348318a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609009024:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298639.137302:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298639.137311:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000991eb57d/0xaf35132e6302fc0d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1564672->35119103] (req 1564672->1576959) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf358 expref: 12 pid: 9128 timeout: 928 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.137330:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000991eb57d/0xaf35132e6302fc0d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1564672->35119103] (req 1564672->1576959) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf358 expref: 12 pid: 9128 timeout: 928 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.137346:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000991eb57d/0xaf35132e6302fc0d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1564672->35119103] (req 1564672->1576959) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf358 expref: 12 pid: 9128 timeout: 928 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.137361:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298639 v=(100 16) 00010000:00010000:3.0:1780298639.137365:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298639 v=(100 16) 00010000:00010000:3.0:1780298639.137379:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000488a7b88/0xaf35132e6302fc6f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3284992->36839423] (req 3284992->3289087) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf3ba expref: 8 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.137388:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000488a7b88/0xaf35132e6302fc6f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3284992->36839423] (req 3284992->3289087) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf3ba expref: 8 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.137412:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000488a7b88/0xaf35132e6302fc6f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3284992->36839423] (req 3284992->3289087) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf3ba expref: 8 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.137421:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2556a8000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:7400:1866777604639360:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298639.137455:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733936c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.137992:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2556a8000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:7400:1866777604639360:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298639.137999:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298639.138017:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348318a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609009024:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 721us (788us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298639.138022:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14648 00000100:00100000:3.0:1780298639.142408:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609009536 00000100:00100000:3.0:1780298639.142412:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609009536 00000100:00100000:3.0:1780298639.142435:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7193 00000100:00100000:3.0:1780298639.142441:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348318700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2398:x1866777609009536:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.142448:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.142457:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ccb4ef2a/0xaf35132e6302fc76 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf3c1 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.142542:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609009664 00000100:00100000:3.0:1780298639.142547:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609009664 00000100:00100000:3.0:1780298639.142584:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7194 00000100:00100000:3.0:1780298639.149209:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.150275:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.151033:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2541ea680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777604640384:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298639.151046:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298639.151072:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b5ec50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609010816:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 720us (798us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298639.151083:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14651 00000100:00100000:3.0:1780298639.156749:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609011968 00000100:00100000:3.0:1780298639.156753:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609011968 00000100:00100000:3.0:1780298639.156776:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7199 00000100:00100000:3.0:1780298639.156781:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f75a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:29360:x1866777609011968:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.156787:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.156793:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4b625d7/0xaf35132e6302fcbc lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf3f2 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.156806:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c60b4886/0xaf35132e6302fc99 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf3dd expref: 12 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.156812:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298639 v=(142 22) 00010000:00010000:3.0:1780298639.156815:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298639 v=(143 22) 00010000:00010000:3.0:1780298639.156818:0:20252:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d4b625d7/0xaf35132e6302fcbc lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf3f2 expref: 15 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.156841:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c60b4886/0xaf35132e6302fc99 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf3dd expref: 12 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.156848:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c60b4886/0xaf35132e6302fc99 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf3dd expref: 12 pid: 8359 timeout: 928 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.156855:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2556a9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604640896:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298639.156883:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.157689:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609012096 00000100:00100000:3.0:1780298639.157693:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609012096 00010000:00010000:3.0:1780298639.157706:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c60b4886/0xaf35132e6302fc99 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf3dd expref: 13 pid: 8359 timeout: 928 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.157723:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14654 00000100:00100000:3.0:1780298639.157729:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f758700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2397:x1866777609012096:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298639.157734:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298639.157742:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c60b4886/0xaf35132e6302fc99 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf3dd expref: 13 pid: 8359 timeout: 928 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.157748:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c60b4886/0xaf35132e6302fc99 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf3dd expref: 13 pid: 8359 timeout: 928 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.157757:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c60b4886/0xaf35132e6302fc99 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf3dd expref: 13 pid: 8359 timeout: 928 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.157769:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298639 v=(143 22) 00010000:00010000:3.0:1780298639.157771:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298639 v=(143 22) 00010000:00010000:3.0:1780298639.157774:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000d4b625d7/0xaf35132e6302fcbc lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf3f2 expref: 14 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.157779:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d4b625d7/0xaf35132e6302fcbc lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf3f2 expref: 14 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.157794:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000d4b625d7/0xaf35132e6302fcbc lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf3f2 expref: 14 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.157800:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2556ab100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777604641024:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298639.157827:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.158427:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2556ab100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777604641024:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298639.158439:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298639.158534:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f758700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609012096:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 804us (868us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298639.158544:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14654 00000100:00100000:3.0:1780298639.165554:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.166404:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.167389:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.167459:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.171176:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.173528:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609014656 00000100:00100000:3.0:1780298639.173553:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609014656 00002000:00010000:3.0:1780298639.173596:0:8363:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (146707->18446744073709551615) 00002000:00100000:3.0:1780298639.173601:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255512a00 with x1866777609014656 ext(146707->18446744073709551615) 00002000:00010000:3.0:1780298639.173617:0:8363:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255512a00. 00000100:00100000:3.0:1780298639.173645:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5419 00000100:00100000:3.0:1780298639.173654:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255512a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609014656:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:3.0:1780298639.181341:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609015552 00000100:00100000:3.0:1780298639.181347:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609015552 00000100:00100000:3.0:1780298639.181387:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7205 00000100:00100000:3.0:1780298639.181397:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244811180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:29360:x1866777609015552:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.181410:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.181425:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a777b004/0xaf35132e6302fd25 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf431 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.181449:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d4b625d7/0xaf35132e6302fcbc lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf3f2 expref: 19 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.181459:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298639 v=(143 22) 00010000:00010000:3.0:1780298639.181463:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298639 v=(144 22) 00010000:00010000:3.0:1780298639.181469:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a777b004/0xaf35132e6302fd25 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf431 expref: 13 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.181497:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d4b625d7/0xaf35132e6302fcbc lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf3f2 expref: 19 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.181510:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d4b625d7/0xaf35132e6302fcbc lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf3f2 expref: 19 pid: 20252 timeout: 928 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.181533:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2556aad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604642176:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298639.181592:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.182195:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2556aad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604642176:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298639.182216:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a777b004/0xaf35132e6302fd25 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf431 expref: 13 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.182230:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830da00, rc 0) 00000100:00100000:3.0:1780298639.182331:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244811180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:29360:x1866777609015552:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 935us (1026us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298639.182342:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7205 00000100:00100000:3.0:1780298639.182480:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609015680 00000100:00100000:3.0:1780298639.182484:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609015680 00010000:00010000:3.0:1780298639.182498:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d4b625d7/0xaf35132e6302fcbc lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf3f2 expref: 20 pid: 20252 timeout: 928 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.182517:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14658 00000100:00100000:3.0:1780298639.182521:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b5cc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2399:x1866777609015680:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298639.182528:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298639.182542:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d4b625d7/0xaf35132e6302fcbc lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf3f2 expref: 20 pid: 20252 timeout: 928 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.182549:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d4b625d7/0xaf35132e6302fcbc lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf3f2 expref: 20 pid: 20252 timeout: 928 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.182569:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4b625d7/0xaf35132e6302fcbc lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf3f2 expref: 20 pid: 20252 timeout: 928 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.182577:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298639 v=(144 22) 00010000:00010000:3.0:1780298639.182579:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298639 v=(144 22) 00010000:00010000:3.0:1780298639.182582:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a777b004/0xaf35132e6302fd25 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf431 expref: 12 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.182588:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a777b004/0xaf35132e6302fd25 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf431 expref: 12 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.182603:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000a777b004/0xaf35132e6302fd25 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf431 expref: 12 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.182609:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2556aad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777604642304:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298639.182638:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.183349:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2556aad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777604642304:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298639.183365:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298639.183402:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b5cc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2399:x1866777609015680:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 877us (1005us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298639.183455:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14658 00000100:00100000:3.0:1780298639.185670:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.187264:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.190029:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.191236:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.194329:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.194352:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.194415:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7209 00000100:00100000:3.0:1780298639.194423:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609016960:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.194432:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.194446:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4b625d7/0xaf35132e6302fd48 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf446 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.194491:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2556ab480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604642688:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.194549:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.196401:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.197219:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244810000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604643072:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.197240:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a5a492f0/0xaf35132e6302fd56 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf45b expref: 14 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.197251:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5a492f0/0xaf35132e6302fd56 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf45b expref: 14 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.197259:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3600, rc 0) 00000100:00100000:3.0:1780298639.197281:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609017344:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 812us (883us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.197286:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7211 00000100:00100000:3.0:1780298639.199020:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609017600 00000100:00100000:3.0:1780298639.199030:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609017600 00010000:00010000:3.0:1780298639.199047:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e6302fd2c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf438 expref: 9 pid: 8359 timeout: 928 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.199069:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14660 00000100:00100000:3.0:1780298639.199075:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244812300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609017600:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298639.199082:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298639.199090:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e6302fd2c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf438 expref: 9 pid: 8359 timeout: 928 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.199097:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e6302fd2c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf438 expref: 9 pid: 8359 timeout: 928 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.199109:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e6302fd2c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf438 expref: 9 pid: 8359 timeout: 928 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.199117:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298639 v=(104 16) 00010000:00010000:3.0:1780298639.199120:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298639 v=(104 16) 00010000:00010000:3.0:1780298639.199142:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000039d55ad6/0xaf35132e6302fd5d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [503808->34058239] (req 503808->511999) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf462 expref: 13 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.199150:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000039d55ad6/0xaf35132e6302fd5d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [503808->34058239] (req 503808->511999) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf462 expref: 13 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.199168:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000039d55ad6/0xaf35132e6302fd5d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [503808->34058239] (req 503808->511999) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf462 expref: 13 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.199183:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2556ab480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:7400:1866777604643328:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298639.199211:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393c00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.199729:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2556ab480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:7400:1866777604643328:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298639.199737:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298639.199756:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244812300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609017600:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 682us (785us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298639.199761:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14660 00000100:00100000:3.0:1780298639.204804:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.205531:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8a5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604643584:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.205543:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004e9b43ec/0xaf35132e6302fd79 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf477 expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.205553:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e9b43ec/0xaf35132e6302fd79 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf477 expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.205560:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2a00, rc 0) 00000100:00100000:3.0:1780298639.205582:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556ab480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609018240:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 717us (779us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.205586:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7214 00000100:00100000:3.0:1780298639.206487:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609018368 00000100:00100000:3.0:1780298639.206491:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609018368 00000100:00100000:3.0:1780298639.206515:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609018496 00000100:00100000:3.0:1780298639.206517:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609018496 00000100:00100000:3.0:1780298639.206546:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7216 00000100:00100000:3.0:1780298639.206563:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255512300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609018496:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.206572:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.206585:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a3e0d5df/0xaf35132e6302fd87 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf485 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.206616:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f758700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604643840:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.206663:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393c00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.207337:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f758700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604643840:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.207357:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a3e0d5df/0xaf35132e6302fd87 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf485 expref: 15 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.207370:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a3e0d5df/0xaf35132e6302fd87 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf485 expref: 15 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.207377:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830de00, rc 0) 00000100:00100000:3.0:1780298639.207400:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255512300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609018496:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 840us (915us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.207405:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7216 00000100:00100000:3.0:1780298639.209479:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.211212:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609019136 00000100:00100000:3.0:1780298639.211217:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609019136 00000100:00100000:3.0:1780298639.211254:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7220 00000100:00100000:3.0:1780298639.211267:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2398:x1866777609019136:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.211277:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.211286:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ccb4ef2a/0xaf35132e6302fda3 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf4a1 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.211338:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f75aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604644352:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.211366:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393c00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.214664:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.215791:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.216567:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348005180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604644736:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.216583:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e7fd0135/0xaf35132e6302fdb8 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf4b6 expref: 16 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.216594:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7fd0135/0xaf35132e6302fdb8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf4b6 expref: 16 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.216602:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9be00, rc 0) 00000100:00100000:3.0:1780298639.216627:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2398:x1866777609019648:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 773us (837us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.216635:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7223 00000100:00100000:3.0:1780298639.219189:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609020032 00000100:00100000:3.0:1780298639.219193:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609020032 00000100:00100000:3.0:1780298639.219216:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7226 00000100:00100000:3.0:1780298639.219222:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609020032:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.219228:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.219235:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a3e0d5df/0xaf35132e6302fdcd lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf4cb expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.219255:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604645120:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.219290:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393c00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.219874:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604645120:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.219885:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a3e0d5df/0xaf35132e6302fdcd lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf4cb expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.219894:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a3e0d5df/0xaf35132e6302fdcd lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf4cb expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.219900:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830de00, rc 0) 00000100:00100000:3.0:1780298639.219920:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609020032:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 699us (758us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.219924:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7226 00000100:00100000:3.0:1780298639.224812:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609020672 00000100:00100000:3.0:1780298639.224816:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609020672 00000100:00100000:3.0:1780298639.224838:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7228 00000100:00100000:3.0:1780298639.224843:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609020672:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.224849:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.224856:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037a3562c/0xaf35132e6302fddb lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf4d9 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.224878:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604645504:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.224906:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393c00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.227804:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.227843:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.228668:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24339ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604645888:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.228698:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000342a3832/0xaf35132e6302fdf0 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf4ee expref: 17 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.228718:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000342a3832/0xaf35132e6302fdf0 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf4ee expref: 17 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.228725:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244f3a600, rc 0) 00000100:00100000:3.0:1780298639.228742:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609021056:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 834us (900us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.228748:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7231 00000100:00100000:3.0:1780298639.234143:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7234 00000100:00100000:3.0:1780298639.234156:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609021824:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.234167:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.234174:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a3e0d5df/0xaf35132e6302fe05 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf503 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.234205:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604646272:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.234250:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.234799:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604646272:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.234812:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a3e0d5df/0xaf35132e6302fe05 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf503 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.234820:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a3e0d5df/0xaf35132e6302fe05 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf503 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.234836:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830de00, rc 0) 00000100:00100000:3.0:1780298639.234856:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609021824:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 707us (794us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.234861:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7234 00000100:00100000:3.0:1780298639.235689:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609021952 00000100:00100000:3.0:1780298639.235692:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609021952 00000100:00100000:3.0:1780298639.235714:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7235 00000100:00100000:3.0:1780298639.235719:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609021952:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.235725:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.235731:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e6302fe0c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf50a expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.235750:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604646400:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.235778:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.236344:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604646400:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.236358:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e6302fe0c lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf50a expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.236370:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e6302fe0c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf50a expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.236376:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830d200, rc 0) 00000100:00100000:3.0:1780298639.236397:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609021952:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 678us (731us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.236406:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7235 00000100:00100000:3.0:1780298639.242674:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609022592 00000100:00100000:3.0:1780298639.242683:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609022592 00000100:00100000:3.0:1780298639.242704:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7238 00000100:00100000:3.0:1780298639.242709:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343966a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609022592:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.242715:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.242722:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f8fcaf61/0xaf35132e6302fe21 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf51f expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.242742:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604646784:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.242770:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.243350:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604646784:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.243363:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f8fcaf61/0xaf35132e6302fe21 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf51f expref: 17 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.243373:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f8fcaf61/0xaf35132e6302fe21 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf51f expref: 17 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.243380:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830c400, rc 0) 00000100:00100000:3.0:1780298639.243402:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343966a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609022592:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 693us (751us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.243407:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7238 00000100:00100000:3.0:1780298639.244200:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609022720 00000100:00100000:3.0:1780298639.244203:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609022720 00000100:00100000:3.0:1780298639.244224:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7239 00000100:00100000:3.0:1780298639.244235:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343967480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2398:x1866777609022720:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.244242:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.244247:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007dc48e1c/0xaf35132e6302fe28 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf526 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.244266:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604646912:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.244295:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.244910:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604646912:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.244921:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007dc48e1c/0xaf35132e6302fe28 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf526 expref: 17 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.244931:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007dc48e1c/0xaf35132e6302fe28 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf526 expref: 17 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.244938:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830c800, rc 0) 00000100:00100000:3.0:1780298639.244960:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343967480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2398:x1866777609022720:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 726us (788us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.244965:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7239 00000100:00100000:3.0:1780298639.249661:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609023232 00000100:00100000:3.0:1780298639.249666:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609023232 00000100:00100000:3.0:1780298639.249695:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7242 00000100:00100000:3.0:1780298639.249704:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2541e9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609023232:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.249711:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.249719:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e6302fe3d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf53b expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.249748:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604647296:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.249778:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.250273:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604647296:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.250284:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e6302fe3d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf53b expref: 17 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.250293:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e6302fe3d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf53b expref: 17 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.250299:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830d200, rc 0) 00000100:00100000:3.0:1780298639.250331:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2541e9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609023232:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 630us (695us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.250336:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7242 00000100:00100000:3.0:1780298639.253055:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609023616 00000100:00100000:3.0:1780298639.253059:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609023616 00000100:00100000:3.0:1780298639.253080:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7243 00000100:00100000:3.0:1780298639.253085:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2541e9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609023616:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.253091:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.253097:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a3e0d5df/0xaf35132e6302fe44 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf542 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.253116:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604647424:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.257517:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609024000 00000100:00100000:3.0:1780298639.257525:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609024000 00000100:00100000:3.0:1780298639.257549:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7246 00000100:00100000:3.0:1780298639.257558:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2541e8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609024000:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.257567:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.257575:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007dc48e1c/0xaf35132e6302fe59 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf557 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.257599:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604647808:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.257628:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.258270:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604647808:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.258285:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007dc48e1c/0xaf35132e6302fe59 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf557 expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.258299:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007dc48e1c/0xaf35132e6302fe59 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf557 expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.258322:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830c800, rc 0) 00000100:00100000:3.0:1780298639.258345:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2541e8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609024000:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 790us (852us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.258350:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7246 00000100:00100000:3.0:1780298639.265903:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609024768 00000100:00100000:3.0:1780298639.265907:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609024768 00000100:00100000:3.0:1780298639.265929:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7250 00000100:00100000:3.0:1780298639.265936:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609024768:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.265942:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.265953:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e6302fe75 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf573 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.265974:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604648320:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.266004:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.266603:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604648320:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.266614:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e6302fe75 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf573 expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.266624:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e6302fe75 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf573 expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.266630:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830d200, rc 0) 00000100:00100000:3.0:1780298639.266650:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609024768:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 716us (774us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.266655:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7250 00000100:00100000:3.0:1780298639.273759:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609025536 00000100:00100000:3.0:1780298639.273763:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609025536 00000100:00100000:3.0:1780298639.273785:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7254 00000100:00100000:3.0:1780298639.273790:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609025536:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.273796:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.273802:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007dc48e1c/0xaf35132e6302fe91 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf58f expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.273820:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604648704:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.273855:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.274463:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604648704:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.274474:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007dc48e1c/0xaf35132e6302fe91 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf58f expref: 17 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.274483:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007dc48e1c/0xaf35132e6302fe91 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf58f expref: 17 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.274489:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830c800, rc 0) 00000100:00100000:3.0:1780298639.274509:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609025536:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 720us (773us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.274514:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7254 00000100:00100000:3.0:1780298639.281085:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609026176 00000100:00100000:3.0:1780298639.281089:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609026176 00000100:00100000:3.0:1780298639.281110:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7258 00000100:00100000:3.0:1780298639.281115:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255493100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2397:x1866777609026176:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.281121:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.281143:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e6302fead lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf5ab expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.281163:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604649216:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.281193:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.281742:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604649216:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.281756:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e6302fead lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf5ab expref: 16 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.281782:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e6302fead lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf5ab expref: 16 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.281790:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830d200, rc 0) 00000100:00100000:3.0:1780298639.281818:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255493100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2397:x1866777609026176:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 702us (763us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.281823:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7258 00000100:00100000:3.0:1780298639.290117:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609027200 00000100:00100000:3.0:1780298639.290121:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609027200 00010000:00010000:3.0:1780298639.290150:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000a860e235/0xaf35132e6302fdfe lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf4fc expref: 7 pid: 9128 timeout: 928 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.290172:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14664 00000100:00100000:3.0:1780298639.290181:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f737b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609027200:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298639.290189:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298639.290202:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000a860e235/0xaf35132e6302fdfe lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf4fc expref: 7 pid: 9128 timeout: 928 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.290213:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000a860e235/0xaf35132e6302fdfe lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf4fc expref: 7 pid: 9128 timeout: 928 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.290228:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a860e235/0xaf35132e6302fdfe lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf4fc expref: 7 pid: 9128 timeout: 928 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.290248:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298639.290275:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f737b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609027200:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 96us (179us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298639.290282:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14664 00000100:00100000:3.0:1780298639.291477:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609027456 00000100:00100000:3.0:1780298639.291482:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609027456 00000100:00100000:3.0:1780298639.291512:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7262 00000100:00100000:3.0:1780298639.291520:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f736d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:29360:x1866777609027456:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.291528:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.291536:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e6302fed0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf5c7 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.291554:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000001d4afe0b/0xaf35132e6302fe7c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [139264->1835007] (req 139264->147455) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf57a expref: 14 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.291563:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000101214ae/0xaf35132e6302fdd4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1839104->35393535] (req 1839104->1843199) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf4d2 expref: 14 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.291570:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 2 to [0x280000401:0x12:0x0].0x0 time=1780298639 v=(105 16) 00010000:00010000:3.0:1780298639.291573:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298639 v=(107 16) 00010000:00010000:3.0:1780298639.291578:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e6302fed0 lrc: 5/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf5c7 expref: 17 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.291596:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000101214ae/0xaf35132e6302fdd4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1839104->35393535] (req 1839104->1843199) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf4d2 expref: 14 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.291612:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000101214ae/0xaf35132e6302fdd4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1839104->35393535] (req 1839104->1843199) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf4d2 expref: 14 pid: 8359 timeout: 928 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.291622:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604650112:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298639.291671:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000001d4afe0b/0xaf35132e6302fe7c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [139264->1835007] (req 139264->147455) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf57a expref: 14 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.291682:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000001d4afe0b/0xaf35132e6302fe7c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [139264->1835007] (req 139264->147455) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf57a expref: 14 pid: 8359 timeout: 928 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.291692:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604650240:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298639.291720:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.292184:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604650112:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298639.292190:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604650240:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298639.292200:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e6302fed0 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf5c7 expref: 17 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.292208:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830d200, rc 0) 00000100:00100000:3.0:1780298639.292227:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f736d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:29360:x1866777609027456:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 709us (774us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298639.292231:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7262 00000100:00100000:3.0:1780298639.292672:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609027712 00000100:00100000:3.0:1780298639.292681:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609027712 00010000:00010000:3.0:1780298639.292693:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000001d4afe0b/0xaf35132e6302fe7c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [139264->1835007] (req 139264->147455) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf57a expref: 16 pid: 8359 timeout: 928 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.292711:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14666 00000100:00100000:3.0:1780298639.292716:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f734700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609027712:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298639.292721:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298639.292729:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000001d4afe0b/0xaf35132e6302fe7c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [139264->1835007] (req 139264->147455) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf57a expref: 16 pid: 8359 timeout: 928 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.292735:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000001d4afe0b/0xaf35132e6302fe7c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [139264->1835007] (req 139264->147455) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf57a expref: 16 pid: 8359 timeout: 928 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.292744:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d4afe0b/0xaf35132e6302fe7c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [139264->1835007] (req 139264->147455) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf57a expref: 16 pid: 8359 timeout: 928 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.292752:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298639.292772:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f734700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2400:x1866777609027712:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 56us (114us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298639.292777:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14666 00000100:00100000:3.0:1780298639.297009:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7263 00000100:00100000:3.0:1780298639.297019:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f737100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2398:x1866777609028480:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.297067:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.297076:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007dc48e1c/0xaf35132e6302feec lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf5dc expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.297111:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604650624:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.297156:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.297783:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604650624:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.297802:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007dc48e1c/0xaf35132e6302feec lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf5dc expref: 16 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.297822:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007dc48e1c/0xaf35132e6302feec lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf5dc expref: 16 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.297845:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830c800, rc 0) 00000100:00100000:3.0:1780298639.297886:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f737100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2398:x1866777609028480:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 865us (947us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.297899:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7263 00000100:00100000:3.0:1780298639.298913:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609028736 00000100:00100000:3.0:1780298639.298918:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609028736 00000100:00100000:3.0:1780298639.303497:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.306005:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.309646:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.312559:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609030144 00000100:00100000:3.0:1780298639.312564:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609030144 00000100:00100000:3.0:1780298639.312595:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7271 00000100:00100000:3.0:1780298639.312603:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548bb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609030144:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.312611:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.312620:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007dc48e1c/0xaf35132e6302ff24 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf60d expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.312673:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604651520:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.312735:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.316008:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.316034:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.316870:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255680000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604651904:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.316887:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d71a6aba/0xaf35132e6302ff39 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf622 expref: 8 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.316900:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d71a6aba/0xaf35132e6302ff39 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf622 expref: 8 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.316908:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9bc00, rc 0) 00000100:00100000:3.0:1780298639.316936:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e4f1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609030528:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 812us (900us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.316947:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7274 00000100:00100000:3.0:1780298639.318799:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255682d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604652032:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.318820:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e6302ff40 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf629 expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.318843:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e6302ff40 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf629 expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.318855:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9aa00, rc 0) 00000100:00100000:3.0:1780298639.318897:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609030656:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 890us (980us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.318904:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7275 00000100:00100000:3.0:1780298639.319964:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609031040 00000100:00100000:3.0:1780298639.319968:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609031040 00000100:00100000:3.0:1780298639.320660:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8a5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604652416:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.320691:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004e9b43ec/0xaf35132e6302ff55 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf63e expref: 8 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.320711:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e9b43ec/0xaf35132e6302ff55 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf63e expref: 8 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.320725:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2a00, rc 0) 00000100:00100000:3.0:1780298639.320751:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255683800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609031040:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 747us (807us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.320756:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7278 00000100:00100000:3.0:1780298639.321584:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609031168 00000100:00100000:3.0:1780298639.321587:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609031168 00000100:00100000:3.0:1780298639.321607:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7279 00000100:00100000:3.0:1780298639.321611:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255683480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:29360:x1866777609031168:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.321617:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.321624:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f8fcaf61/0xaf35132e6302ff5c lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf645 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.321637:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000488a7b88/0xaf35132e6302fec9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf5c0 expref: 10 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.321652:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298639 v=(145 22) 00010000:00010000:3.0:1780298639.321655:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298639 v=(146 22) 00010000:00010000:3.0:1780298639.321658:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f8fcaf61/0xaf35132e6302ff5c lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [4444160->4448255] (req 4444160->4448255) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf645 expref: 7 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.321673:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000488a7b88/0xaf35132e6302fec9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf5c0 expref: 10 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.321683:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000488a7b88/0xaf35132e6302fec9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf5c0 expref: 10 pid: 19992 timeout: 928 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.321691:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604652544:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298639.321718:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.322258:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604652544:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298639.322269:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f8fcaf61/0xaf35132e6302ff5c lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [4444160->4448255] (req 4444160->4448255) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf645 expref: 7 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.322276:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830c400, rc 0) 00000100:00100000:3.0:1780298639.322296:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255683480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609031168:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 685us (733us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298639.322301:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7279 00000100:00100000:3.0:1780298639.324707:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609031680 00000100:00100000:3.0:1780298639.324712:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609031680 00000100:00100000:3.0:1780298639.326424:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.326442:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.327375:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255682a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604653184:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.327397:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003b0cbb9d/0xaf35132e6302ff78 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf661 expref: 8 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.327421:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b0cbb9d/0xaf35132e6302ff78 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf661 expref: 8 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.327436:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b800, rc 0) 00000100:00100000:3.0:1780298639.327460:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609031936:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 931us (1016us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.327465:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7283 00000100:00100000:3.0:1780298639.328464:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609032064 00000100:00100000:3.0:1780298639.328468:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609032064 00000100:00100000:3.0:1780298639.328499:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7284 00000100:00100000:3.0:1780298639.328509:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8a5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609032064:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.328517:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.328525:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e4811bb2/0xaf35132e6302ff7f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf668 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.328551:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604653312:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.328621:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.329261:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604653312:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.329282:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e4811bb2/0xaf35132e6302ff7f lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf668 expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.329310:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e4811bb2/0xaf35132e6302ff7f lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf668 expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.329329:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830ca00, rc 0) 00000100:00100000:3.0:1780298639.329356:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8a5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609032064:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 851us (936us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.329364:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7284 00000100:00100000:3.0:1780298639.330252:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609032320 00000100:00100000:3.0:1780298639.330257:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609032320 00000100:00100000:3.0:1780298639.330285:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7286 00000100:00100000:3.0:1780298639.330292:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:29360:x1866777609032320:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.330298:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.330304:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001011cd71/0xaf35132e6302ff8d lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf676 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.330328:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298639 v=(146 22) 00010000:00010000:3.0:1780298639.330332:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298639 v=(146 22) 00010000:00010000:3.0:1780298639.330339:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001011cd71/0xaf35132e6302ff8d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1949696->4444159] (req 1949696->1961983) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf676 expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.330349:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001011cd71/0xaf35132e6302ff8d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1949696->4444159] (req 1949696->1961983) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf676 expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.330355:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830c600, rc 0) 00000100:00100000:3.0:1780298639.330379:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:29360:x1866777609032320:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 89us (152us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298639.330383:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7286 00000100:00100000:3.0:1780298639.334119:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609032960 00000100:00100000:3.0:1780298639.334143:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609032960 00000100:00100000:3.0:1780298639.338027:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7292 00000100:00100000:3.0:1780298639.338040:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609033344:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.338052:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.338062:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e4811bb2/0xaf35132e6302ffb7 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf6a0 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.338105:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604654080:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.338190:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.338956:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604654080:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.338989:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e4811bb2/0xaf35132e6302ffb7 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf6a0 expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.339022:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e4811bb2/0xaf35132e6302ffb7 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf6a0 expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.339032:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830ca00, rc 0) 00000100:00100000:3.0:1780298639.339067:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609033344:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1032us (1111us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.339081:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7292 00000100:00100000:3.0:1780298639.342032:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609033856 00000100:00100000:3.0:1780298639.342043:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609033856 00000100:00100000:3.0:1780298639.342078:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609033984 00000100:00100000:3.0:1780298639.342082:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609033984 00000100:00100000:3.0:1780298639.344011:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.345114:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.350703:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609034880 00000100:00100000:3.0:1780298639.350709:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609034880 00000100:00100000:3.0:1780298639.350744:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7300 00000100:00100000:3.0:1780298639.350752:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609034880:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.350763:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.350778:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e4811bb2/0xaf35132e6302ffe8 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf6d8 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.350870:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304e680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604655104:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.350931:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.351738:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304e680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604655104:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.351762:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e4811bb2/0xaf35132e6302ffe8 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf6d8 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.351784:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e4811bb2/0xaf35132e6302ffe8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf6d8 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.351829:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830ca00, rc 0) 00000100:00100000:3.0:1780298639.351864:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609034880:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1111us (1214us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.351874:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7300 00000100:00100000:3.0:1780298639.354712:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609035392 00000100:00100000:3.0:1780298639.354717:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609035392 00000100:00100000:3.0:1780298639.354749:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7305 00000100:00100000:3.0:1780298639.354760:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343965f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609035392:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.354768:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.354777:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007dc48e1c/0xaf35132e6303000b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf6f4 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.354817:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304e680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604655744:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.354860:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.355539:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304e680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604655744:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.355555:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007dc48e1c/0xaf35132e6303000b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf6f4 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.355568:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007dc48e1c/0xaf35132e6303000b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf6f4 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.355586:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830c800, rc 0) 00000100:00100000:3.0:1780298639.355613:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343965f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609035392:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 856us (924us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.355626:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7305 00000100:00100000:3.0:1780298639.356612:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7306 00000100:00100000:3.0:1780298639.356622:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343967100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609035520:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.356667:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.356674:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002dc12793/0xaf35132e63030019 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf6fb expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.356701:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304e680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604656000:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.356737:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.357375:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304e680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604656000:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.357391:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002dc12793/0xaf35132e63030019 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf6fb expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.357406:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002dc12793/0xaf35132e63030019 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf6fb expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.357415:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830d200, rc 0) 00000100:00100000:3.0:1780298639.357442:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343967100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609035520:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 821us (904us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.357451:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7306 00000100:00100000:3.0:1780298639.358321:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609035776 00000100:00100000:3.0:1780298639.358326:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609035776 00000100:00100000:3.0:1780298639.358350:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7308 00000100:00100000:3.0:1780298639.358355:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343967480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609035776:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.358362:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.358368:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e4811bb2/0xaf35132e63030020 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf709 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.358383:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000008361787d/0xaf35132e6302ffb0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [557056->1949695] (req 557056->565247) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf699 expref: 10 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.358398:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298639 v=(147 22) 00010000:00010000:3.0:1780298639.358401:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298639 v=(148 22) 00010000:00010000:3.0:1780298639.358405:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e4811bb2/0xaf35132e63030020 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1089536->1097727] (req 1089536->1097727) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf709 expref: 8 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.358420:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000008361787d/0xaf35132e6302ffb0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [557056->1949695] (req 557056->565247) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf699 expref: 10 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.358430:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000008361787d/0xaf35132e6302ffb0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [557056->1949695] (req 557056->565247) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf699 expref: 10 pid: 19992 timeout: 928 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.358438:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304e680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604656128:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298639.358468:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.359036:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304e680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604656128:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298639.359051:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e4811bb2/0xaf35132e63030020 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1089536->1097727] (req 1089536->1097727) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf709 expref: 8 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.359062:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830ca00, rc 0) 00000100:00100000:3.0:1780298639.359086:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343967480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:29360:x1866777609035776:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 730us (798us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298639.359093:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7308 00000100:00100000:3.0:1780298639.363558:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609036544 00000100:00100000:3.0:1780298639.363564:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609036544 00000100:00100000:3.0:1780298639.363605:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7312 00000100:00100000:3.0:1780298639.363624:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2541ea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609036544:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298639.364532:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244811880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604656768:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.364595:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001d4afe0b/0xaf35132e6303003c lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf725 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.364609:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d4afe0b/0xaf35132e6303003c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf725 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.364620:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3000, rc 0) 00000100:00100000:3.0:1780298639.365466:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.366749:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.367733:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.369056:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604657280:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298639.369095:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c60b4886/0xaf35132e6303004a lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf72c expref: 11 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.369110:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244f3a800, rc 0) 00000100:00100000:3.0:1780298639.369224:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:29360:x1866777609037056:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1399us (1489us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298639.369232:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7313 00000100:00100000:3.0:1780298639.372349:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.373441:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.375900:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.375920:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.377855:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.377873:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.380329:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.381387:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.382306:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.383197:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.384068:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.384985:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.386158:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.386245:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5458 00000100:00100000:3.0:1780298639.386256:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348319180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609039872:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:3.0:1780298639.390935:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609040384 00000100:00100000:3.0:1780298639.390939:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609040384 00000100:00100000:3.0:1780298639.390962:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7321 00000100:00100000:3.0:1780298639.390967:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e424450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609040384:12345-192.168.203.25@tcp:16:fsx.0 00000100:00100000:3.0:1780298639.391653:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e424450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609040384:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 685us (747us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298639.391658:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7321 00000100:00100000:3.0:1780298639.393558:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609040768 00000100:00100000:3.0:1780298639.393562:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609040768 00000100:00100000:3.0:1780298639.393590:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7323 00000100:00100000:3.0:1780298639.393600:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609040768:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.393609:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.393620:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007dc48e1c/0xaf35132e630300ba lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf780 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.393663:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf348318700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604659072:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.393749:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.394294:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348318700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604659072:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.394331:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007dc48e1c/0xaf35132e630300ba lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf780 expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.394350:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007dc48e1c/0xaf35132e630300ba lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf780 expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.394361:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830c800, rc 0) 00000100:00100000:3.0:1780298639.394393:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609040768:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 796us (889us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.394401:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7323 00000100:00100000:3.0:1780298639.395219:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609041024 00000100:00100000:3.0:1780298639.395225:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609041024 00000100:00100000:3.0:1780298639.396965:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.399667:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609041664 00000100:00100000:3.0:1780298639.399672:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609041664 00000100:00100000:3.0:1780298639.399695:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7327 00000100:00100000:3.0:1780298639.399699:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8c8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:29360:x1866777609041664:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.399705:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.399712:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a3e0d5df/0xaf35132e630300dd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf79c expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.399732:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003b0cbb9d/0xaf35132e6303009e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf76b expref: 7 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.399738:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298639 v=(110 16) 00010000:00010000:3.0:1780298639.399740:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298639 v=(111 16) 00010000:00010000:3.0:1780298639.399742:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a3e0d5df/0xaf35132e630300dd lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf79c expref: 12 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.399754:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003b0cbb9d/0xaf35132e6303009e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf76b expref: 7 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.399760:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003b0cbb9d/0xaf35132e6303009e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf76b expref: 7 pid: 9128 timeout: 928 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.399767:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34831a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604659712:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298639.399794:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.400322:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34831a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604659712:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298639.400336:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a3e0d5df/0xaf35132e630300dd lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf79c expref: 12 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.400346:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830de00, rc 0) 00000100:00100000:3.0:1780298639.400374:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8c8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:29360:x1866777609041664:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 674us (741us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298639.400389:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7327 00000100:00100000:3.0:1780298639.401593:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609041920 00000100:00100000:3.0:1780298639.401599:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609041920 00002000:00010000:3.0:1780298639.401613:0:16972:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4612430->18446744073709551615) 00002000:00100000:3.0:1780298639.401617:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244811180 with x1866777609041920 ext(4612430->18446744073709551615) 00002000:00010000:3.0:1780298639.401622:0:16972:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf244811180. 00000100:00100000:3.0:1780298639.401643:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5464 00000100:00100000:3.0:1780298639.401654:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244811180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609041920:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:3.0:1780298639.401773:0:16972:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4221744 -> 4612430 00002000:00010000:3.0:1780298639.401779:0:16972:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298639 -> 1780298639 00002000:00010000:3.0:1780298639.401781:0:16972:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298639 -> 1780298639 00002000:00010000:3.0:1780298639.401783:0:16972:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298639 -> 1780298639 00002000:00010000:3.0:1780298639.401785:0:16972:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 368 -> 368 00000100:00100000:3.0:1780298639.401806:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244811180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609041920:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 156us (251us total) trans 4294969448 rc 0/0 00000100:00100000:3.0:1780298639.401811:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5464 00002000:00010000:3.0:1780298639.401814:0:16972:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4612430->18446744073709551615) 00002000:00100000:3.0:1780298639.401816:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244811180 with x1866777609041920 ext(4612430->18446744073709551615) 00002000:00010000:3.0:1780298639.401820:0:16972:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf244811180. 00000100:00100000:3.0:1780298639.403579:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609042304 00000100:00100000:3.0:1780298639.403583:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609042304 00000100:00100000:3.0:1780298639.403610:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609042432 00000100:00100000:3.0:1780298639.403612:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609042432 00000100:00100000:3.0:1780298639.403636:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7328 00000100:00100000:3.0:1780298639.403646:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609042304:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.403662:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.403670:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007dc48e1c/0xaf35132e630300f2 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf7aa expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.403699:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34831a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604659968:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.403737:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.404450:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34831a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604659968:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.404466:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007dc48e1c/0xaf35132e630300f2 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf7aa expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.404477:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007dc48e1c/0xaf35132e630300f2 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf7aa expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.404483:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830c800, rc 0) 00000100:00100000:3.0:1780298639.404508:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609042304:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 867us (966us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.404514:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7328 00000100:00100000:3.0:1780298639.405550:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609042688 00000100:00100000:3.0:1780298639.405555:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609042688 00000100:00100000:3.0:1780298639.405611:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7331 00000100:00100000:3.0:1780298639.405618:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8a5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609042688:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.405624:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.405630:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e63030107 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf7bf expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.405700:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34831a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604660352:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.405735:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.406355:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34831a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604660352:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.406367:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e63030107 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf7bf expref: 8 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.406378:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e63030107 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf7bf expref: 8 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.406387:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830d200, rc 0) 00000100:00100000:3.0:1780298639.406416:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8a5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609042688:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 798us (888us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.406423:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7331 00000100:00100000:3.0:1780298639.412219:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.412327:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.413234:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.416623:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.416641:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.425143:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609045376 00000100:00100000:3.0:1780298639.425148:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609045376 00002000:00100000:3.0:1780298639.425174:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x12:0x0] (626688->630783) req@ffff9cf36e5d9500 x1866777609045376/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:345/0 lens 488/0 e 0 to 0 dl 1780298650 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298639.425184:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e5d9500 with x1866777609045376 ext(626688->630783) 00002000:00010000:3.0:1780298639.425188:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf36e5d9500 00000100:00100000:3.0:1780298639.425212:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5470 00000100:00100000:3.0:1780298639.425220:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609045376:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298639.425356:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609045376:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 137us (265us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1780298639.425361:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5470 00002000:00100000:3.0:1780298639.425365:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x12:0x0] (626688->630783) req@ffff9cf36e5d9500 x1866777609045376/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:345/0 lens 488/4536 e 0 to 0 dl 1780298650 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298639.425372:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e5d9500 with x1866777609045376 ext(626688->630783) 00002000:00010000:3.0:1780298639.425376:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf36e5d9500 00000100:00100000:3.0:1780298639.429309:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609045760 00000100:00100000:3.0:1780298639.429324:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609045760 00000100:00100000:3.0:1780298639.429356:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7338 00000100:00100000:3.0:1780298639.429364:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609045760:12345-192.168.203.25@tcp:16:fsx.0 00000100:00100000:3.0:1780298639.430219:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609045760:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 857us (955us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298639.430224:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7338 00000100:00100000:3.0:1780298639.435171:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.436673:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7339 00000100:00100000:3.0:1780298639.436686:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609046272:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.436697:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.436710:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a777b004/0xaf35132e6303017e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf80c expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.436765:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf348318a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604661760:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.436841:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.437623:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348318a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604661760:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.437650:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a777b004/0xaf35132e6303017e lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf80c expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.437670:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a777b004/0xaf35132e6303017e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf80c expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.437681:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830da00, rc 0) 00000100:00100000:3.0:1780298639.437711:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609046272:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1029us (1170us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.437718:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7339 00000100:00100000:3.0:1780298639.438865:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7341 00000100:00100000:3.0:1780298639.438874:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255510a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609046528:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.438882:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.438892:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002dc12793/0xaf35132e6303018c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf81a expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.438925:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf348318a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604661888:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.438965:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.439692:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348318a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604661888:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.439706:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002dc12793/0xaf35132e6303018c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf81a expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.439726:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002dc12793/0xaf35132e6303018c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf81a expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.439734:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830d200, rc 0) 00000100:00100000:3.0:1780298639.439758:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255510a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609046528:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 887us (983us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.439763:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7341 00000100:00100000:3.0:1780298639.441877:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609046912 00000100:00100000:3.0:1780298639.441884:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609046912 00000100:00100000:3.0:1780298639.441934:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7344 00000100:00100000:3.0:1780298639.441948:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609046912:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.441959:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.441980:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007dc48e1c/0xaf35132e630301a1 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf82f expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.442021:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf348318a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604662272:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.442081:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393b40 going to sleep for 11 seconds 00002000:00010000:3.0:1780298639.442905:0:8359:0:(ofd_lvb.c:259:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size: 633823 -> 3150661 00000100:00100000:3.0:1780298639.442919:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348318a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604662272:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.442946:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007dc48e1c/0xaf35132e630301a1 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf82f expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.442967:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007dc48e1c/0xaf35132e630301a1 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf82f expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.442990:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830c800, rc 0) 00000100:00100000:3.0:1780298639.443033:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609046912:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1089us (1187us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.443040:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7344 00000100:00100000:3.0:1780298639.444088:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609047040 00000100:00100000:3.0:1780298639.444094:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609047040 00000100:00100000:3.0:1780298639.445190:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f75a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604662528:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.445213:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e630301af lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf83d expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.445233:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e630301af lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf83d expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.445248:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3800, rc 0) 00000100:00100000:3.0:1780298639.445275:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609047168:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1077us (1175us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.445296:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7346 00000100:00100000:3.0:1780298639.450710:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00002000:00010000:3.0:1780298639.459274:0:20252:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks: 368 -> 400 00000100:00100000:3.0:1780298639.459290:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255488a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604662784:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.459330:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e630301cb lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf84b expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.459366:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e630301cb lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf84b expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.459378:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9be00, rc 0) 00000100:00100000:3.0:1780298639.459407:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f75a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609048576:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1018us (1118us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.459414:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7348 00000100:00100000:3.0:1780298639.460207:0:3623:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1780298639.460219:0:3623:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3623:1866777604663040:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1780298639.460301:0:3623:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1780298639.460390:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609048832 00000100:00100000:3.0:1780298639.460394:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609048832 00000100:00100000:3.0:1780298639.460424:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609048960 00000100:00100000:3.0:1780298639.460427:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609048960 00000100:00100000:3.0:1780298639.460455:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7351 00000100:00100000:3.0:1780298639.460463:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255489180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609048960:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298639.460496:0:3623:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3623:1866777604663040:0@lo:41:osp-pre-1-0.0 00010000:00010000:3.0:1780298639.460521:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.460531:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007dc48e1c/0xaf35132e630301e0 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf860 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.460563:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34831aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604663296:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.461496:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34831aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604663296:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.461524:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007dc48e1c/0xaf35132e630301e0 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf860 expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.461575:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007dc48e1c/0xaf35132e630301e0 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf860 expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.461586:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830c800, rc 0) 00000100:00100000:3.0:1780298639.461616:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255489180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609048960:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1153us (1219us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.461623:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7351 00000100:00100000:3.0:1780298639.463868:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609049216 00000100:00100000:3.0:1780298639.463872:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609049216 00000100:00100000:3.0:1780298639.463900:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5479 00000100:00100000:3.0:1780298639.463905:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255489880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609049216:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:3.0:1780298639.464051:0:19943:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:3.0:1780298639.464054:0:19943:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298639 -> 1780298639 00002000:00010000:3.0:1780298639.464056:0:19943:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298639 -> 1780298639 00002000:00010000:3.0:1780298639.464058:0:19943:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298639 -> 1780298639 00002000:00010000:3.0:1780298639.464059:0:19943:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 16 -> 32 00000100:00100000:3.0:1780298639.464083:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255489880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609049216:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 178us (243us total) trans 4294969064 rc 0/0 00000100:00100000:3.0:1780298639.464087:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5479 00000100:00100000:3.0:1780298639.465337:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7353 00000100:00100000:3.0:1780298639.465348:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255489f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609049472:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.465356:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.465369:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e630301fc lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf875 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.465419:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34831aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604663552:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.465472:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.466190:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34831aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604663552:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.466210:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e630301fc lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf875 expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.466221:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e630301fc lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf875 expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.466229:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830d200, rc 0) 00000100:00100000:3.0:1780298639.466255:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255489f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609049472:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 909us (991us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.466261:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7353 00000100:00100000:3.0:1780298639.467097:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609049600 00000100:00100000:3.0:1780298639.467101:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609049600 00000100:00100000:3.0:1780298639.467148:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7354 00000100:00100000:3.0:1780298639.467155:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255488380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609049600:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.467161:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.467167:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a777b004/0xaf35132e63030203 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf87c expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.467188:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34831aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604663680:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.467261:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.467880:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34831aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604663680:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.467895:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a777b004/0xaf35132e63030203 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf87c expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.467905:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a777b004/0xaf35132e63030203 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf87c expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.467913:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830da00, rc 0) 00000100:00100000:3.0:1780298639.467938:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255488380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609049600:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 784us (876us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.467943:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7354 00000100:00100000:3.0:1780298639.468923:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.470360:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.472120:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609050368 00000100:00100000:3.0:1780298639.472153:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609050368 00000100:00100000:3.0:1780298639.472218:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7360 00000100:00100000:3.0:1780298639.472265:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609050368:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.472274:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.472284:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007dc48e1c/0xaf35132e63030234 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf8a6 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.472330:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34831b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604664576:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.472358:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.472893:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604664448:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.472912:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b4561703/0xaf35132e6303022d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf8ad expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.472922:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b4561703/0xaf35132e6303022d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf8ad expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.472929:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b600, rc 0) 00000100:00100000:3.0:1780298639.472953:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609050496:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 718us (764us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.472996:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7361 00000100:00100000:3.0:1780298639.473840:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.475550:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609050880 00000100:00100000:3.0:1780298639.475556:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609050880 00000100:00100000:3.0:1780298639.475625:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7364 00000100:00100000:3.0:1780298639.475632:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609050880:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.475639:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.475646:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a777b004/0xaf35132e63030249 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf8c2 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.475670:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34831b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604664960:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.475752:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.476262:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34831b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604664960:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.476279:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a777b004/0xaf35132e63030249 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf8c2 expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.476358:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a777b004/0xaf35132e63030249 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf8c2 expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.476367:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830da00, rc 0) 00000100:00100000:3.0:1780298639.476399:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609050880:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 767us (874us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.476406:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7364 00000100:00100000:3.0:1780298639.477361:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.479501:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.482185:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.483332:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609051776 00000100:00100000:3.0:1780298639.483338:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609051776 00000100:00100000:3.0:1780298639.483378:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5480 00000100:00100000:3.0:1780298639.483388:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f759c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609051776:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:3.0:1780298639.483540:0:8363:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:3.0:1780298639.483544:0:8363:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298639 -> 1780298639 00002000:00010000:3.0:1780298639.483547:0:8363:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298639 -> 1780298639 00002000:00010000:3.0:1780298639.483550:0:8363:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298639 -> 1780298639 00002000:00010000:3.0:1780298639.483552:0:8363:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 56 -> 56 00000100:00100000:3.0:1780298639.483583:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f759c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609051776:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 197us (346us total) trans 4294969066 rc 0/0 00000100:00100000:3.0:1780298639.483591:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5480 00000100:00100000:3.0:1780298639.490488:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.491789:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.498999:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.500179:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.500558:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.501693:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.505038:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.505093:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.507112:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609054080 00000100:00100000:3.0:1780298639.507118:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609054080 00000100:00100000:3.0:1780298639.507178:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7381 00000100:00100000:3.0:1780298639.507191:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34831e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609054080:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.507203:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.507214:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a777b004/0xaf35132e630302c7 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf939 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.507256:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255682300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604667648:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.507293:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.509343:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609054336 00000100:00100000:3.0:1780298639.509347:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609054336 00002000:00100000:3.0:1780298639.509378:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (1777664->1781759) req@ffff9cf34831ea00 x1866777609054336/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:345/0 lens 488/0 e 0 to 0 dl 1780298650 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298639.509386:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf34831ea00 with x1866777609054336 ext(1777664->1781759) 00002000:00010000:3.0:1780298639.509389:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf34831ea00 00000100:00100000:3.0:1780298639.509407:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5483 00000100:00100000:3.0:1780298639.509412:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34831ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609054336:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298639.509548:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34831ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609054336:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 134us (246us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1780298639.509564:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5483 00002000:00100000:3.0:1780298639.509572:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (1777664->1781759) req@ffff9cf34831ea00 x1866777609054336/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:345/0 lens 488/4536 e 0 to 0 dl 1780298650 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298639.509585:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf34831ea00 with x1866777609054336 ext(1777664->1781759) 00002000:00010000:3.0:1780298639.509588:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf34831ea00 00000100:00100000:3.0:1780298639.511926:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609054592 00000100:00100000:3.0:1780298639.511971:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609054592 00002000:00100000:3.0:1780298639.511994:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (1777664->1785855) req@ffff9cf348007800 x1866777609054592/t0(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:345/0 lens 8680/0 e 0 to 0 dl 1780298650 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298639.512003:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf348007800 with x1866777609054592 ext(1777664->1785855) 00002000:00010000:3.0:1780298639.512007:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf348007800 00000100:00100000:3.0:1780298639.512040:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5485 00000100:00100000:3.0:1780298639.512049:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348007800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609054592:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:3.0:1780298639.512559:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348007800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609054592:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 511us (674us total) trans 4294969455 rc 0/0 00000100:00100000:3.0:1780298639.512567:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5485 00002000:00100000:3.0:1780298639.512574:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (1777664->1785855) req@ffff9cf348007800 x1866777609054592/t4294969455(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:345/0 lens 8680/448 e 0 to 0 dl 1780298650 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298639.512583:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf348007800 with x1866777609054592 ext(1777664->1785855) 00002000:00010000:3.0:1780298639.512588:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf348007800 00000100:00100000:3.0:1780298639.513269:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609054720 00000100:00100000:3.0:1780298639.513274:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609054720 00000100:00100000:3.0:1780298639.513304:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7383 00000100:00100000:3.0:1780298639.513324:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34831d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609054720:12345-192.168.203.25@tcp:16:fsx.0 00000100:00100000:3.0:1780298639.514046:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34831d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609054720:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 734us (810us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298639.514055:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7383 00000100:00100000:3.0:1780298639.517086:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.517244:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609055232 00000100:00100000:3.0:1780298639.517248:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609055232 00000100:00100000:3.0:1780298639.517277:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7387 00000100:00100000:3.0:1780298639.517284:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348004000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609055232:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.517290:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.517296:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ccb4ef2a/0xaf35132e630302e3 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf95c expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.517360:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255683800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604668160:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.517386:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393c00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.519045:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609055360 00000100:00100000:3.0:1780298639.519049:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609055360 00000100:00100000:3.0:1780298639.519073:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609055488 00000100:00100000:3.0:1780298639.519076:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609055488 00000100:00100000:3.0:1780298639.519098:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7388 00000100:00100000:3.0:1780298639.519110:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348007480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609055360:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.519120:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.519164:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002dc12793/0xaf35132e630302f1 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf963 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.519204:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255683480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604668288:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.519243:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393c00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.521014:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.522655:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.525483:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.526519:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.527848:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.530916:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.531014:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.532951:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609057024 00000100:00100000:3.0:1780298639.532956:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609057024 00000100:00100000:3.0:1780298639.532993:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7399 00000100:00100000:3.0:1780298639.533006:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609057024:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.533015:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.533026:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4b625d7/0xaf35132e63030330 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cf9a2 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.533062:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24339ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604669696:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.533093:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.534982:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.538671:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.538776:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.540465:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.541221:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243edbb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604670592:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298639.541233:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a5a492f0/0xaf35132e63030361 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1826816->1830911] (req 1826816->1830911) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf9da expref: 7 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.541242:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3600, rc 0) 00000100:00100000:3.0:1780298639.541270:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609058048:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 729us (805us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298639.541275:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7406 00000100:00100000:3.0:1780298639.541438:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.542490:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.543787:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.546590:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.548440:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.548492:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.550363:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.551389:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.551522:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.553222:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.553267:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.555102:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.555881:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.556859:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.556876:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.558699:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.558720:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.560660:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.561655:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.562875:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.562923:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.564776:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.564886:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.566782:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609061376 00000100:00100000:3.0:1780298639.566786:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609061376 00000100:00100000:3.0:1780298639.566818:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7426 00000100:00100000:3.0:1780298639.566831:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243edad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609061376:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.566840:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.566849:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002dc12793/0xaf35132e630303e6 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cfa5f expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.566918:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2541e9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604672896:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.566956:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.567571:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2541e9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604672896:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.567587:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002dc12793/0xaf35132e630303e6 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfa5f expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.567602:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002dc12793/0xaf35132e630303e6 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfa5f expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.567656:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830d200, rc 0) 00000100:00100000:3.0:1780298639.567678:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243edad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609061376:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 852us (951us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.567683:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7426 00000100:00100000:3.0:1780298639.571383:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.572508:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.572552:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.573429:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243ed8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604673536:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.573449:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003bced65c/0xaf35132e63030410 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfa89 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.573483:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003bced65c/0xaf35132e63030410 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfa89 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.573493:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf3582cbe00, rc 0) 00000100:00100000:3.0:1780298639.573514:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2541ea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609062272:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 883us (962us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.573530:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7432 00000100:00100000:3.0:1780298639.574413:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.574431:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.574504:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7434 00000100:00100000:3.0:1780298639.574518:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2541e8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609062528:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.574527:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.574537:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ccb4ef2a/0xaf35132e6303041e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cfa97 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.574574:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243cd7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604673792:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.574601:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.576424:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609062656 00000100:00100000:3.0:1780298639.576429:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609062656 00000100:00100000:3.0:1780298639.576454:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609062784 00000100:00100000:3.0:1780298639.576457:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609062784 00000100:00100000:3.0:1780298639.578251:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.578376:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.580238:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.580281:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.582213:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.582308:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.584328:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609063680 00000100:00100000:3.0:1780298639.584333:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609063680 00000100:00100000:3.0:1780298639.584378:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609063808 00000100:00100000:3.0:1780298639.584381:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609063808 00000100:00100000:3.0:1780298639.584412:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7444 00000100:00100000:3.0:1780298639.584469:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243ed8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609063808:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.584478:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.584488:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e63030464 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cfadd expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.584522:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243cd5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604675072:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.584549:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.585341:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243cd5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604675072:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.585358:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e63030464 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfadd expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.585372:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e63030464 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfadd expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.585381:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830d200, rc 0) 00000100:00100000:3.0:1780298639.585406:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243ed8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609063808:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 943us (1048us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.585414:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7444 00000100:00100000:3.0:1780298639.586321:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609064064 00000100:00100000:3.0:1780298639.586333:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609064064 00000100:00100000:3.0:1780298639.588171:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.590158:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609064448 00000100:00100000:3.0:1780298639.590163:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609064448 00002000:00100000:3.0:1780298639.590189:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (663552->667647) req@ffff9cf2425f0000 x1866777609064448/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:345/0 lens 488/0 e 0 to 0 dl 1780298650 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298639.590200:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2425f0000 with x1866777609064448 ext(663552->667647) 00002000:00010000:3.0:1780298639.590205:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2425f0000 00000100:00100000:3.0:1780298639.590222:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5494 00000100:00100000:3.0:1780298639.590229:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609064448:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298639.590389:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609064448:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 160us (274us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1780298639.590394:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5494 00002000:00100000:3.0:1780298639.590398:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (663552->667647) req@ffff9cf2425f0000 x1866777609064448/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:345/0 lens 488/4536 e 0 to 0 dl 1780298650 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298639.590404:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2425f0000 with x1866777609064448 ext(663552->667647) 00002000:00010000:3.0:1780298639.590406:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2425f0000 00000100:00100000:3.0:1780298639.592906:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609064704 00000100:00100000:3.0:1780298639.592915:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609064704 00000100:00100000:3.0:1780298639.592998:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609064832 00000100:00100000:3.0:1780298639.593003:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609064832 00000100:00100000:3.0:1780298639.595373:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609064960 00000100:00100000:3.0:1780298639.595379:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609064960 00000100:00100000:3.0:1780298639.595432:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7451 00000100:00100000:3.0:1780298639.595448:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243ed8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609064960:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.595471:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.595499:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002dc12793/0xaf35132e6303049c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cfb0e expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.595566:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243cd5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604676096:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.595611:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.596361:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243cd5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604676096:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.596390:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002dc12793/0xaf35132e6303049c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfb0e expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.596405:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002dc12793/0xaf35132e6303049c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfb0e expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.596413:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830d200, rc 0) 00000100:00100000:3.0:1780298639.596439:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243ed8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609064960:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 994us (1102us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.596454:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7451 00000100:00100000:3.0:1780298639.597826:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609065344 00000100:00100000:3.0:1780298639.597834:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609065344 00000100:00100000:3.0:1780298639.605859:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609066112 00000100:00100000:3.0:1780298639.605869:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609066112 00000100:00100000:3.0:1780298639.605916:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7457 00000100:00100000:3.0:1780298639.605926:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2398:x1866777609066112:12345-192.168.203.25@tcp:16:fsx.0 00000100:00100000:3.0:1780298639.606772:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609066112:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 846us (951us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298639.606781:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7457 00000100:00100000:3.0:1780298639.607875:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7458 00000100:00100000:3.0:1780298639.607948:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2551c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609066240:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.607962:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.607986:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002dc12793/0xaf35132e630304bf lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cfb38 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.608067:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243cd5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604676480:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.608174:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.608967:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243cd5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604676480:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.609016:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002dc12793/0xaf35132e630304bf lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfb38 expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.609038:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002dc12793/0xaf35132e630304bf lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfb38 expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.609050:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830d200, rc 0) 00000100:00100000:3.0:1780298639.609091:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2551c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609066240:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1158us (1311us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.609099:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7458 00000100:00100000:3.0:1780298639.610234:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609066624 00000100:00100000:3.0:1780298639.610241:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609066624 00000100:00100000:3.0:1780298639.610324:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7461 00000100:00100000:3.0:1780298639.610338:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e426450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609066624:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298639.612018:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.614712:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.615743:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.617213:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.619082:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.619376:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609067648 00000100:00100000:3.0:1780298639.619386:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609067648 00000100:00100000:3.0:1780298639.619437:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7468 00000100:00100000:3.0:1780298639.619478:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2551c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609067648:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.619492:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.619506:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002dc12793/0xaf35132e63030505 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cfb77 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.619549:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243cd4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604677888:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.619600:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.620231:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243cd4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604677888:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.620251:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002dc12793/0xaf35132e63030505 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfb77 expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.620267:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002dc12793/0xaf35132e63030505 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfb77 expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.620278:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830d200, rc 0) 00000100:00100000:3.0:1780298639.620323:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2551c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609067648:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 866us (1041us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.620331:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7468 00000100:00100000:3.0:1780298639.625979:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.626010:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.630325:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609068800 00000100:00100000:3.0:1780298639.630345:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609068800 00000100:00100000:3.0:1780298639.630390:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7475 00000100:00100000:3.0:1780298639.630403:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:29360:x1866777609068800:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.630418:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.630432:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000037a3562c/0xaf35132e63030536 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cfbaf expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.630457:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000a5a492f0/0xaf35132e63030361 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [1826816->35381247] (req 1826816->1830911) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf9da expref: 7 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.630465:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298639 v=(156 22) 00010000:00010000:3.0:1780298639.630469:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298639 v=(157 22) 00010000:00010000:3.0:1780298639.630474:0:20252:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000037a3562c/0xaf35132e63030536 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [2576384->2580479] (req 2576384->2580479) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfbaf expref: 12 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.630500:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000a5a492f0/0xaf35132e63030361 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [1826816->35381247] (req 1826816->1830911) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf9da expref: 7 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.630514:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000a5a492f0/0xaf35132e63030361 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [1826816->35381247] (req 1826816->1830911) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cf9da expref: 7 pid: 19992 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.630526:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243cd5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604678784:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298639.630587:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.631221:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243cd5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604678784:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298639.631240:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000037a3562c/0xaf35132e63030536 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [2576384->2580479] (req 2576384->2580479) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfbaf expref: 12 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.631264:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830c000, rc 0) 00000100:00100000:3.0:1780298639.631354:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:29360:x1866777609068800:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 955us (1077us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298639.631363:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7475 00000100:00100000:3.0:1780298639.634230:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.634274:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.635286:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243edb800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604679040:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.635325:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e0cae770/0xaf35132e63030544 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfbbd expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.635344:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0cae770/0xaf35132e63030544 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfbbd expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.635357:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2600, rc 0) 00000100:00100000:3.0:1780298639.635396:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243cd7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609069312:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1018us (1121us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.635404:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7477 00000100:00100000:3.0:1780298639.638470:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.639442:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243ed9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604679296:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.639464:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008361787d/0xaf35132e63030560 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfbd9 expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.639478:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008361787d/0xaf35132e63030560 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfbd9 expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.639494:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2800, rc 0) 00000100:00100000:3.0:1780298639.639529:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243cd5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609069824:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 969us (1081us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.639535:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7481 00000100:00100000:3.0:1780298639.640506:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609069952 00000100:00100000:3.0:1780298639.640513:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609069952 00000100:00100000:3.0:1780298639.640567:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609070080 00000100:00100000:3.0:1780298639.640573:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609070080 00000100:00100000:3.0:1780298639.640682:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7483 00000100:00100000:3.0:1780298639.640701:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243edb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609070080:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.640722:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.640738:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e6303056e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cfbe7 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.640785:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604679424:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.640890:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.641559:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604679424:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.641577:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e6303056e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfbe7 expref: 12 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.641592:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e6303056e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfbe7 expref: 12 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.641602:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830d200, rc 0) 00000100:00100000:3.0:1780298639.641640:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243edb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609070080:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 950us (1095us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.641645:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7483 00000100:00100000:3.0:1780298639.644499:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609070464 00000100:00100000:3.0:1780298639.644508:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609070464 00000100:00100000:3.0:1780298639.644549:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7485 00000100:00100000:3.0:1780298639.644562:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609070464:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.644576:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.644588:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a777b004/0xaf35132e6303057c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cfbf5 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.644628:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604679552:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.644687:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.645444:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604679552:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.645463:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a777b004/0xaf35132e6303057c lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfbf5 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.645481:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a777b004/0xaf35132e6303057c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfbf5 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.645493:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830da00, rc 0) 00000100:00100000:3.0:1780298639.645524:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609070464:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 965us (1069us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.645529:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7485 00000100:00100000:3.0:1780298639.651047:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.652630:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609071360 00000100:00100000:3.0:1780298639.652636:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609071360 00000100:00100000:3.0:1780298639.652682:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7489 00000100:00100000:3.0:1780298639.652709:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609071360:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.652727:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.652746:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e63030598 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cfc18 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.652808:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604680064:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.652871:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.653595:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604680064:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.653618:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e63030598 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfc18 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.653632:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e63030598 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfc18 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.653640:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830d200, rc 0) 00000100:00100000:3.0:1780298639.653666:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609071360:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 977us (1118us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.653671:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7489 00000100:00100000:3.0:1780298639.654768:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7491 00000100:00100000:3.0:1780298639.654783:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243edbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609071488:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.654799:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.654810:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007dc48e1c/0xaf35132e630305a6 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cfc1f expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.654838:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007dc48e1c/0xaf35132e630305a6 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfc1f expref: 8 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.654852:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007dc48e1c/0xaf35132e630305a6 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfc1f expref: 8 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.654862:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830c800, rc 0) 00000100:00100000:3.0:1780298639.654904:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243edbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609071488:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 125us (283us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.654909:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7491 00000100:00100000:3.0:1780298639.658642:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.658669:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.660650:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.660686:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.662660:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609072512 00000100:00100000:3.0:1780298639.662667:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609072512 00000100:00100000:3.0:1780298639.662707:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7499 00000100:00100000:3.0:1780298639.662717:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609072512:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.662731:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.662744:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007dc48e1c/0xaf35132e630305de lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cfc57 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.662802:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007dc48e1c/0xaf35132e630305de lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfc57 expref: 13 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.662845:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007dc48e1c/0xaf35132e630305de lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfc57 expref: 13 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.662865:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830c800, rc 0) 00000100:00100000:3.0:1780298639.662910:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609072512:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 195us (332us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.662925:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7499 00000100:00100000:3.0:1780298639.664902:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.666745:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.667903:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.669108:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.670711:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.672112:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.674230:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.674264:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.676705:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.676742:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.678897:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.678964:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.681204:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609074432 00000100:00100000:3.0:1780298639.681210:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609074432 00000100:00100000:3.0:1780298639.681248:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7511 00000100:00100000:3.0:1780298639.681261:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609074432:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.681277:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.681300:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a3e0d5df/0xaf35132e6303062b lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cfca4 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.681373:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b42a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604681984:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.681441:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.682226:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b42a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604681984:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.682251:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a3e0d5df/0xaf35132e6303062b lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfca4 expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.682273:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a3e0d5df/0xaf35132e6303062b lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfca4 expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.682289:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830de00, rc 0) 00000100:00100000:3.0:1780298639.682340:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609074432:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1082us (1184us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.682349:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7511 00000100:00100000:3.0:1780298639.683417:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609074688 00000100:00100000:3.0:1780298639.683422:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609074688 00000100:00100000:3.0:1780298639.683455:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7513 00000100:00100000:3.0:1780298639.683466:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:29360:x1866777609074688:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.683477:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.683489:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007dc48e1c/0xaf35132e63030639 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cfcb2 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.683509:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298639 v=(116 16) 00010000:00010000:3.0:1780298639.683514:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298639 v=(116 16) 00010000:00010000:3.0:1780298639.683534:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007dc48e1c/0xaf35132e63030639 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [499712->831487] (req 499712->503807) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfcb2 expref: 11 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.683550:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007dc48e1c/0xaf35132e63030639 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [499712->831487] (req 499712->503807) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfcb2 expref: 11 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.683559:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830c800, rc 0) 00000100:00100000:3.0:1780298639.683591:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:29360:x1866777609074688:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 129us (205us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298639.683598:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7513 00000100:00100000:3.0:1780298639.687228:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604682240:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.687255:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001d4afe0b/0xaf35132e63030647 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfcc0 expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.687275:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d4afe0b/0xaf35132e63030647 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfcc0 expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.687291:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3000, rc 0) 00000100:00100000:3.0:1780298639.687349:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609075072:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1246us (1347us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.687359:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7515 00000100:00100000:3.0:1780298639.690612:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609075456 00000100:00100000:3.0:1780298639.690617:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609075456 00000100:00100000:3.0:1780298639.690674:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7518 00000100:00100000:3.0:1780298639.690686:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255488380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609075456:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.690706:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.690718:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a3e0d5df/0xaf35132e6303065c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cfcd5 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.690760:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b42a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604682752:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.690814:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.691596:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b42a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604682752:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.691611:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a3e0d5df/0xaf35132e6303065c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfcd5 expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.691622:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a3e0d5df/0xaf35132e6303065c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfcd5 expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.691629:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830de00, rc 0) 00000100:00100000:3.0:1780298639.691659:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255488380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609075456:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 976us (1075us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.691666:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7518 00000100:00100000:3.0:1780298639.695556:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609075968 00000100:00100000:3.0:1780298639.695561:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609075968 00000100:00100000:3.0:1780298639.695585:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7520 00000100:00100000:3.0:1780298639.695591:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609075968:12345-192.168.203.25@tcp:16:fsx.0 00000100:00100000:3.0:1780298639.696326:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609075968:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 733us (797us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298639.696335:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7520 00000100:00100000:3.0:1780298639.697237:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.697275:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.698560:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8a5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604683136:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.698595:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008746f5b4/0xaf35132e63030671 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfcea expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.698621:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008746f5b4/0xaf35132e63030671 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfcea expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.698634:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157200, rc 0) 00000100:00100000:3.0:1780298639.698679:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b41f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609076224:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1275us (1400us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.698690:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7522 00000100:00100000:3.0:1780298639.702582:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609076608 00000100:00100000:3.0:1780298639.702589:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609076608 00000100:00100000:3.0:1780298639.702630:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7525 00000100:00100000:3.0:1780298639.702639:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609076608:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.702650:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.702661:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a3e0d5df/0xaf35132e63030686 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cfcff expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.702741:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b41500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604683520:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.702799:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.706068:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.706163:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.707068:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2541e9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604683776:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.707102:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a5a492f0/0xaf35132e6303069b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfd14 expref: 13 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.707168:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5a492f0/0xaf35132e6303069b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfd14 expref: 13 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.707179:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3600, rc 0) 00000100:00100000:3.0:1780298639.707210:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609077120:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 978us (1047us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.707217:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7528 00000100:00100000:3.0:1780298639.708111:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.708182:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.711054:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.711982:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.713088:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609077888 00000100:00100000:3.0:1780298639.713093:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609077888 00000100:00100000:3.0:1780298639.713154:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609078016 00000100:00100000:3.0:1780298639.713158:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609078016 00000100:00100000:3.0:1780298639.713225:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7533 00000100:00100000:3.0:1780298639.713235:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2541e9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609078016:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.713244:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.713253:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a777b004/0xaf35132e630306be lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cfd37 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.713284:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604684672:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.713330:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.713998:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604684672:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.714055:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a777b004/0xaf35132e630306be lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfd37 expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.714075:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a777b004/0xaf35132e630306be lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfd37 expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.714085:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830da00, rc 0) 00000100:00100000:3.0:1780298639.714106:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2541e9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609078016:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 874us (992us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.714113:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7533 00000100:00100000:3.0:1780298639.715074:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609078272 00000100:00100000:3.0:1780298639.715079:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609078272 00000100:00100000:3.0:1780298639.715149:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7535 00000100:00100000:3.0:1780298639.715163:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2541e9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609078272:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.715172:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.715182:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e630306cc lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cfd45 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.715221:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604684928:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.715253:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.719556:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.719758:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.721957:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.721977:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.723877:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.724816:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.725932:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.725951:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.727832:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.727952:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.730633:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.731908:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.733283:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.737093:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.737944:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.739050:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.739082:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.739144:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7552 00000100:00100000:3.0:1780298639.739156:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8ec380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609081088:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.739168:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.739178:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a777b004/0xaf35132e63030743 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 8 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cfdbc expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.739216:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8ed180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604686848:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.739269:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.741081:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609081472 00000100:00100000:3.0:1780298639.741087:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609081472 00000100:00100000:3.0:1780298639.742973:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.743932:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.745320:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609081984 00000100:00100000:3.0:1780298639.745325:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609081984 00000100:00100000:3.0:1780298639.747353:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7559 00000100:00100000:3.0:1780298639.747365:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609082112:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.747379:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.747387:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001011cd71/0xaf35132e63030774 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 8 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cfded expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.747420:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8eed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604687616:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.747470:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393c00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.749357:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609082368 00000100:00100000:3.0:1780298639.749362:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609082368 00000100:00100000:3.0:1780298639.749389:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609082496 00000100:00100000:3.0:1780298639.749392:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609082496 00000100:00100000:3.0:1780298639.749423:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7562 00000100:00100000:3.0:1780298639.749436:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609082496:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.749449:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.749458:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e4811bb2/0xaf35132e63030789 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 8 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cfe02 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.749493:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8ee680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604688000:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.749524:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393c00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.750211:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8ee680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604688000:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.750229:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e4811bb2/0xaf35132e63030789 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfe02 expref: 13 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.750244:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e4811bb2/0xaf35132e63030789 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfe02 expref: 13 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.750272:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830ca00, rc 0) 00000100:00100000:3.0:1780298639.750299:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609082496:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 869us (950us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.750305:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7562 00000100:00100000:3.0:1780298639.754469:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.754587:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7565 00000100:00100000:3.0:1780298639.754602:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8ee680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2399:x1866777609083008:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.754611:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.754621:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a777b004/0xaf35132e6303079e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 8 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cfe10 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.754668:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8ef480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604688512:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.754710:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393c00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.755658:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8ef480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604688512:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.755670:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a777b004/0xaf35132e6303079e lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfe10 expref: 13 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.755680:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a777b004/0xaf35132e6303079e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfe10 expref: 13 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.755686:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830da00, rc 0) 00000100:00100000:3.0:1780298639.755709:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8ee680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2399:x1866777609083008:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1113us (1242us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.755727:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7565 00000100:00100000:3.0:1780298639.756528:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.756545:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.757535:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433ddf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604688768:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.757570:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000101214ae/0xaf35132e630307ac lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfe25 expref: 14 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.757593:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000101214ae/0xaf35132e630307ac lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfe25 expref: 14 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.757605:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3200, rc 0) 00000100:00100000:3.0:1780298639.757657:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8ee300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609083392:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1031us (1109us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.757674:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7567 00000100:00100000:3.0:1780298639.758801:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7569 00000100:00100000:3.0:1780298639.758814:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8a7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609083648:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.758827:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.758839:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e4811bb2/0xaf35132e630307ba lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 8 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cfe33 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.758881:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8ed500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604689024:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.758920:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393c00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.759792:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8ed500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604689024:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.759816:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e4811bb2/0xaf35132e630307ba lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfe33 expref: 14 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.759843:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e4811bb2/0xaf35132e630307ba lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfe33 expref: 14 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.759858:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830ca00, rc 0) 00000100:00100000:3.0:1780298639.759892:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8a7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609083648:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1081us (1179us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.759903:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7569 00000100:00100000:3.0:1780298639.763868:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609084032 00000100:00100000:3.0:1780298639.763875:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609084032 00000100:00100000:3.0:1780298639.763950:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609084160 00000100:00100000:3.0:1780298639.763953:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609084160 00000100:00100000:3.0:1780298639.766052:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.766072:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.768491:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.770476:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.771404:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.772553:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.773824:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.775632:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7580 00000100:00100000:3.0:1780298639.775654:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609085440:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.775664:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.775678:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e4811bb2/0xaf35132e63030807 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 8 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cfe79 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.775730:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8eed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604690304:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.775788:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393c00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.779252:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.783669:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.785205:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7585 00000100:00100000:3.0:1780298639.785220:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609086592:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.785229:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.785239:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f8fcaf61/0xaf35132e6303082a lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 8 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cfea3 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.785267:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8ed180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604690688:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.785316:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.786059:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8ed180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604690688:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.786077:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f8fcaf61/0xaf35132e6303082a lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfea3 expref: 13 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.786092:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f8fcaf61/0xaf35132e6303082a lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfea3 expref: 13 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.786101:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830c400, rc 0) 00000100:00100000:3.0:1780298639.786157:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609086592:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 941us (1044us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.786164:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7585 00000100:00100000:3.0:1780298639.787014:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609086848 00000100:00100000:3.0:1780298639.787019:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609086848 00000100:00100000:3.0:1780298639.787049:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609086976 00000100:00100000:3.0:1780298639.787059:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609086976 00000100:00100000:3.0:1780298639.789101:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7590 00000100:00100000:3.0:1780298639.789116:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24380ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609087232:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.789150:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298639.790948:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609087360 00000100:00100000:3.0:1780298639.790953:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609087360 00000100:00100000:3.0:1780298639.790991:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7591 00000100:00100000:3.0:1780298639.791001:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609087360:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.791011:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.791062:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a777b004/0xaf35132e63030854 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 8 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cfecd expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.791101:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8ed180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604691072:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.791189:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.791974:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8ed180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604691072:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.791993:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a777b004/0xaf35132e63030854 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfecd expref: 13 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.792008:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a777b004/0xaf35132e63030854 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfecd expref: 13 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.792018:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830da00, rc 0) 00000100:00100000:3.0:1780298639.792046:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609087360:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1048us (1123us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.792053:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7591 00000100:00100000:3.0:1780298639.795295:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.797383:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.797404:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.799207:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609088256 00000100:00100000:3.0:1780298639.799212:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609088256 00000100:00100000:3.0:1780298639.799251:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7597 00000100:00100000:3.0:1780298639.799265:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609088256:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.799274:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.799285:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a777b004/0xaf35132e6303087e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 8 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cfef7 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.799332:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25488a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604691456:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.799380:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.800895:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609088512 00000100:00100000:3.0:1780298639.800899:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609088512 00000100:00100000:3.0:1780298639.800919:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609088640 00000100:00100000:3.0:1780298639.800921:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609088640 00000100:00100000:3.0:1780298639.800950:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7600 00000100:00100000:3.0:1780298639.800962:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2397:x1866777609088640:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.800974:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.800982:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f8fcaf61/0xaf35132e63030893 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 8 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cff0c expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.801004:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f8fcaf61/0xaf35132e63030893 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cff0c expref: 15 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.801018:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f8fcaf61/0xaf35132e63030893 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cff0c expref: 15 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.801040:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830c400, rc 0) 00000100:00100000:3.0:1780298639.801068:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2397:x1866777609088640:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 110us (184us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.801077:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7600 00000100:00100000:3.0:1780298639.804356:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609089024 00000100:00100000:3.0:1780298639.804360:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609089024 00000100:00100000:3.0:1780298639.804384:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7602 00000100:00100000:3.0:1780298639.804390:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e427050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609089024:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.804398:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.804408:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002dc12793/0xaf35132e630308a1 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 9 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cff1a expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.804468:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254888380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604691712:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.804515:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.805111:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254888380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604691712:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.805162:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002dc12793/0xaf35132e630308a1 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cff1a expref: 13 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.805175:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002dc12793/0xaf35132e630308a1 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cff1a expref: 13 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.805184:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830d200, rc 0) 00000100:00100000:3.0:1780298639.805212:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e427050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609089024:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 821us (925us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.805225:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7602 00000100:00100000:3.0:1780298639.806000:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609089408 00000100:00100000:3.0:1780298639.806006:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609089408 00000100:00100000:3.0:1780298639.807738:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.807766:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.809334:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609089792 00000100:00100000:3.0:1780298639.809339:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609089792 00000100:00100000:3.0:1780298639.809360:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7608 00000100:00100000:3.0:1780298639.809367:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24380b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:29360:x1866777609089792:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.809373:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.809380:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f8fcaf61/0xaf35132e630308cb lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 9 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cff44 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.809392:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298639 v=(159 22) 00010000:00010000:3.0:1780298639.809395:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298639 v=(159 22) 00010000:00010000:3.0:1780298639.809402:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f8fcaf61/0xaf35132e630308cb lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 9 type: EXT [1970176->2097151] (req 1970176->1974271) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cff44 expref: 15 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.809411:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f8fcaf61/0xaf35132e630308cb lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 9 type: EXT [1970176->2097151] (req 1970176->1974271) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cff44 expref: 15 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.809417:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830c400, rc 0) 00000100:00100000:3.0:1780298639.809437:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24380b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:29360:x1866777609089792:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 72us (133us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298639.809441:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7608 00000100:00100000:3.0:1780298639.810149:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.814150:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609090560 00000100:00100000:3.0:1780298639.814156:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609090560 00000100:00100000:3.0:1780298639.814208:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7612 00000100:00100000:3.0:1780298639.814216:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24380bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609090560:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.814225:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.814234:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e630308e7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 8 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cff60 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.814253:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e630308e7 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cff60 expref: 15 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.814267:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e630308e7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cff60 expref: 15 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.814323:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830d200, rc 0) 00000100:00100000:3.0:1780298639.814346:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24380bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609090560:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 132us (245us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.814350:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7612 00000100:00100000:3.0:1780298639.822498:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609091584 00000100:00100000:3.0:1780298639.822503:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609091584 00002000:00100000:3.0:1780298639.822530:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (151552->159743) req@ffff9cf254918000 x1866777609091584/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:345/0 lens 488/0 e 0 to 0 dl 1780298650 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298639.822541:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf254918000 with x1866777609091584 ext(151552->159743) 00002000:00010000:3.0:1780298639.822546:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf254918000 00000100:00100000:3.0:1780298639.822566:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5528 00000100:00100000:3.0:1780298639.822574:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254918000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2398:x1866777609091584:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298639.822750:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254918000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2398:x1866777609091584:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 176us (283us total) trans 0 rc 8192/8192 00000100:00100000:3.0:1780298639.822757:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5528 00002000:00100000:3.0:1780298639.822764:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (151552->159743) req@ffff9cf254918000 x1866777609091584/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:345/0 lens 488/8632 e 0 to 0 dl 1780298650 ref 1 fl Complete:/600/0 rc 8192/8192 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298639.822773:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf254918000 with x1866777609091584 ext(151552->159743) 00002000:00010000:3.0:1780298639.822776:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf254918000 00000100:00100000:3.0:1780298639.823733:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609091712 00000100:00100000:3.0:1780298639.823738:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609091712 00000100:00100000:3.0:1780298639.823773:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609091840 00000100:00100000:3.0:1780298639.823777:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609091840 00000100:00100000:3.0:1780298639.827249:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609092352 00000100:00100000:3.0:1780298639.827255:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609092352 00000100:00100000:3.0:1780298639.827288:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7623 00000100:00100000:3.0:1780298639.827300:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243063800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2398:x1866777609092352:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.827321:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.827330:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e63030934 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 8 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cffad expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.827349:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e63030934 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cffad expref: 15 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.827358:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e63030934 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cffad expref: 15 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.827366:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830d200, rc 0) 00000100:00100000:3.0:1780298639.827395:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243063800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2398:x1866777609092352:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 99us (171us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.827400:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7623 00000100:00100000:3.0:1780298639.829794:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609092608 00000100:00100000:3.0:1780298639.829798:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609092608 00000100:00100000:3.0:1780298639.829850:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609092736 00000100:00100000:3.0:1780298639.829853:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609092736 00000100:00100000:3.0:1780298639.829877:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7625 00000100:00100000:3.0:1780298639.829888:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243063480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2397:x1866777609092736:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.829900:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.829907:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a3e0d5df/0xaf35132e63030942 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 8 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cffbb expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.829923:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a3e0d5df/0xaf35132e63030942 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cffbb expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.829937:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a3e0d5df/0xaf35132e63030942 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cffbb expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.829943:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830de00, rc 0) 00000100:00100000:3.0:1780298639.829963:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243063480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2397:x1866777609092736:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 80us (144us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.829970:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7625 00000100:00100000:3.0:1780298639.832974:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609093248 00000100:00100000:3.0:1780298639.832979:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609093248 00000100:00100000:3.0:1780298639.833006:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7629 00000100:00100000:3.0:1780298639.833013:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243060a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609093248:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.833025:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.833031:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037a3562c/0xaf35132e6303095e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 8 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cffd7 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.833046:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000037a3562c/0xaf35132e6303095e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cffd7 expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.833055:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037a3562c/0xaf35132e6303095e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cffd7 expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.833062:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830c000, rc 0) 00000100:00100000:3.0:1780298639.833087:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243060a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609093248:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 76us (139us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.833091:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7629 00000100:00100000:3.0:1780298639.834283:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609093376 00000100:00100000:3.0:1780298639.834286:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609093376 00000100:00100000:3.0:1780298639.834305:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7630 00000100:00100000:3.0:1780298639.834320:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:29360:x1866777609093376:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.834325:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.834331:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ccb4ef2a/0xaf35132e63030965 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 10 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cffde expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.834344:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f8fcaf61/0xaf35132e630308cb lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 10 type: EXT [1970176->2097151] (req 1970176->1974271) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cff44 expref: 17 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.834350:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298639 v=(160 22) 00010000:00010000:3.0:1780298639.834358:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298639 v=(161 22) 00010000:00010000:3.0:1780298639.834362:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ccb4ef2a/0xaf35132e63030965 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 10 type: EXT [1978368->1982463] (req 1978368->1982463) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cffde expref: 12 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.834379:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f8fcaf61/0xaf35132e630308cb lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 10 type: EXT [1970176->2097151] (req 1970176->1974271) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cff44 expref: 17 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.834389:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f8fcaf61/0xaf35132e630308cb lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 10 type: EXT [1970176->2097151] (req 1970176->1974271) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cff44 expref: 17 pid: 9128 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.834397:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254889180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604693632:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298639.834425:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.834900:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254889180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604693632:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298639.834910:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ccb4ef2a/0xaf35132e63030965 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 10 type: EXT [1978368->1982463] (req 1978368->1982463) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cffde expref: 12 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.834918:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830cc00, rc 0) 00000100:00100000:3.0:1780298639.834938:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:29360:x1866777609093376:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 628us (674us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298639.834942:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7630 00000100:00100000:3.0:1780298639.839655:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2399:x1866777609093888:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 1092us (1187us total) trans 4294969469 rc 0/0 00000100:00100000:3.0:1780298639.839664:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5532 00002000:00100000:3.0:1780298639.839672:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (1978368->5799935) req@ffff9cf244adbb80 x1866777609093888/t4294969469(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:345/0 lens 504/448 e 0 to 0 dl 1780298650 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298639.839693:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244adbb80 with x1866777609093888 ext(1978368->5799935) 00002000:00010000:3.0:1780298639.839700:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf244adbb80 00000100:00100000:3.0:1780298639.840364:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609094016 00000100:00100000:3.0:1780298639.840369:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609094016 00000100:00100000:3.0:1780298639.840401:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7631 00000100:00100000:3.0:1780298639.840408:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2398:x1866777609094016:12345-192.168.203.25@tcp:16:fsx.0 00000100:00100000:3.0:1780298639.841324:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609094016:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 915us (983us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298639.841332:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7631 00000100:00100000:3.0:1780298639.842302:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609094272 00000100:00100000:3.0:1780298639.842320:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609094272 00000100:00100000:3.0:1780298639.842361:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7633 00000100:00100000:3.0:1780298639.842371:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243061c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609094272:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.842380:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.842392:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037a3562c/0xaf35132e63030973 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 8 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cffec expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.842419:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000037a3562c/0xaf35132e63030973 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cffec expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.842438:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037a3562c/0xaf35132e63030973 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cffec expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.842453:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830c000, rc 0) 00000100:00100000:3.0:1780298639.842483:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243061c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609094272:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 115us (213us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.842491:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7633 00000100:00100000:3.0:1780298639.843986:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609094400 00000100:00100000:3.0:1780298639.843992:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609094400 00000100:00100000:3.0:1780298639.844026:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609094528 00000100:00100000:3.0:1780298639.844029:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609094528 00000100:00100000:3.0:1780298639.844062:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7635 00000100:00100000:3.0:1780298639.844072:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609094528:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.844084:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.844094:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a3e0d5df/0xaf35132e63030981 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 8 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194cfffa expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.844117:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a3e0d5df/0xaf35132e63030981 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfffa expref: 14 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.844155:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a3e0d5df/0xaf35132e63030981 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfffa expref: 14 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.844164:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830de00, rc 0) 00000100:00100000:3.0:1780298639.844193:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609094528:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 124us (202us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.844200:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7635 00000100:00100000:3.0:1780298639.845676:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609094784 00000100:00100000:3.0:1780298639.845682:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609094784 00000100:00100000:3.0:1780298639.845724:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7637 00000100:00100000:3.0:1780298639.845739:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243061880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609094784:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.845755:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.845765:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e6303098f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 8 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0008 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.845789:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e6303098f lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0008 expref: 15 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.845804:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e6303098f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0008 expref: 15 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.845814:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830d200, rc 0) 00000100:00100000:3.0:1780298639.845839:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243061880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609094784:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 105us (186us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.845848:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7637 00000100:00100000:3.0:1780298639.847431:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609094912 00000100:00100000:3.0:1780298639.847434:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609094912 00002000:00100000:3.0:1780298639.847454:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x12:0x0] (2953216->2957311) req@ffff9cf244ad9f80 x1866777609094912/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:345/0 lens 488/0 e 0 to 0 dl 1780298650 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298639.847461:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244ad9f80 with x1866777609094912 ext(2953216->2957311) 00002000:00010000:3.0:1780298639.847465:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf244ad9f80 00000100:00100000:3.0:1780298639.847479:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5533 00000100:00100000:3.0:1780298639.847483:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609094912:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298639.847600:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609094912:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 116us (191us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1780298639.847604:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5533 00002000:00100000:3.0:1780298639.847612:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x12:0x0] (2953216->2957311) req@ffff9cf244ad9f80 x1866777609094912/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:345/0 lens 488/4536 e 0 to 0 dl 1780298650 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298639.847618:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244ad9f80 with x1866777609094912 ext(2953216->2957311) 00002000:00010000:3.0:1780298639.847621:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf244ad9f80 00000100:00100000:3.0:1780298639.849845:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609095168 00000100:00100000:3.0:1780298639.849850:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609095168 00000100:00100000:3.0:1780298639.849921:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7638 00000100:00100000:3.0:1780298639.849934:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242861500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609095168:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.849946:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.849959:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4b625d7/0xaf35132e6303099d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 10 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d000f expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.850005:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254889180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604694272:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.850063:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.850737:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254889180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604694272:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.850751:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d4b625d7/0xaf35132e6303099d lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 10 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d000f expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.850761:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4b625d7/0xaf35132e6303099d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 10 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d000f expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.850767:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830ce00, rc 0) 00000100:00100000:3.0:1780298639.850789:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242861500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609095168:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 862us (1020us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.850794:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7638 00000100:00100000:3.0:1780298639.853115:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609095808 00000100:00100000:3.0:1780298639.853121:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609095808 00000100:00100000:3.0:1780298639.853163:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7642 00000100:00100000:3.0:1780298639.853170:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2551c4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609095808:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.853177:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.853184:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e630309b2 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 8 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0032 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.853206:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e630309b2 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0032 expref: 15 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.853220:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e630309b2 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0032 expref: 15 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.853227:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830d200, rc 0) 00000100:00100000:3.0:1780298639.853250:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2551c4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609095808:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 81us (157us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.853258:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7642 00000100:00100000:3.0:1780298639.856323:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609096192 00000100:00100000:3.0:1780298639.856329:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609096192 00000100:00100000:3.0:1780298639.856354:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609096320 00000100:00100000:3.0:1780298639.856358:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609096320 00000100:00100000:3.0:1780298639.862467:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609097088 00000100:00100000:3.0:1780298639.862474:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609097088 00000100:00100000:3.0:1780298639.862524:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7650 00000100:00100000:3.0:1780298639.862547:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243cd7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2400:x1866777609097088:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.862556:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.862585:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e630309ea lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 8 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0063 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.862614:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e630309ea lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0063 expref: 16 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.862634:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e630309ea lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0063 expref: 16 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.862644:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830d200, rc 0) 00000100:00100000:3.0:1780298639.862680:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243cd7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2400:x1866777609097088:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 137us (247us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.862688:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7650 00000100:00100000:3.0:1780298639.867038:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609097600 00000100:00100000:3.0:1780298639.867043:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609097600 00000100:00100000:3.0:1780298639.867078:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5537 00000100:00100000:3.0:1780298639.867085:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243cd5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609097600:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:3.0:1780298639.867306:0:19928:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:3.0:1780298639.867320:0:19928:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298640 -> 1780298640 00002000:00010000:3.0:1780298639.867322:0:19928:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298640 -> 1780298640 00002000:00010000:3.0:1780298639.867324:0:19928:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298640 -> 1780298640 00002000:00010000:3.0:1780298639.867327:0:19928:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 160 -> 168 00000100:00100000:3.0:1780298639.867358:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243cd5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609097600:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 273us (357us total) trans 4294969073 rc 0/0 00000100:00100000:3.0:1780298639.867365:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5537 00000100:00100000:3.0:1780298639.868636:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7654 00000100:00100000:3.0:1780298639.868654:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243cd7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2398:x1866777609097856:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.868668:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.868681:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d4b625d7/0xaf35132e63030a06 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 9 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0086 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.868712:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d4b625d7/0xaf35132e63030a06 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0086 expref: 16 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.868729:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d4b625d7/0xaf35132e63030a06 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0086 expref: 16 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.868740:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830ce00, rc 0) 00000100:00100000:3.0:1780298639.868777:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243cd7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2398:x1866777609097856:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 127us (234us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.868785:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7654 00000100:00100000:3.0:1780298639.870621:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7656 00000100:00100000:3.0:1780298639.870674:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242862d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609097984:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.870688:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.870699:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a3e0d5df/0xaf35132e63030a14 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 10 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d008d expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.870744:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254889180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604695552:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.870796:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.871528:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254889180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604695552:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.871545:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a3e0d5df/0xaf35132e63030a14 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 10 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d008d expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.871568:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a3e0d5df/0xaf35132e63030a14 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 10 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d008d expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.871578:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830de00, rc 0) 00000100:00100000:3.0:1780298639.871608:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242862d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609097984:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 939us (1090us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.871615:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7656 00000100:00100000:3.0:1780298639.877529:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609098880 00000100:00100000:3.0:1780298639.877543:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609098880 00010000:00010000:3.0:1780298639.877585:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000eae09c9b/0xaf35132e630305fa lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 11 type: EXT [5292032->38846463] (req 5292032->5296127) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfc73 expref: 12 pid: 8359 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.877752:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14707 00000100:00100000:3.0:1780298639.877781:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242861880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609098880:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298639.877805:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298639.877831:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000eae09c9b/0xaf35132e630305fa lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 8 type: EXT [5292032->38846463] (req 5292032->5296127) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfc73 expref: 10 pid: 8359 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.877860:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000eae09c9b/0xaf35132e630305fa lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 8 type: EXT [5292032->38846463] (req 5292032->5296127) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfc73 expref: 10 pid: 8359 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.877911:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000eae09c9b/0xaf35132e630305fa lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 8 type: EXT [5292032->38846463] (req 5292032->5296127) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfc73 expref: 9 pid: 8359 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.877943:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298639.877977:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242861880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609098880:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 205us (532us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298639.877990:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14707 00000100:00100000:3.0:1780298639.878337:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609099264 00000100:00100000:3.0:1780298639.878345:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609099264 00010000:00010000:3.0:1780298639.878374:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000001d4afe0b/0xaf35132e6303089a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2097152->2256895] (req 2097152->2109439) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cff13 expref: 14 pid: 12101 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.881325:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604697344:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298639.881544:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604697472:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298639.881568:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609100160 00000100:00100000:3.0:1780298639.881574:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609100160 00010000:00010000:3.0:1780298639.881600:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000002af870db/0xaf35132e630309dc lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [1662976->2813951] (req 1662976->1667071) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0055 expref: 17 pid: 12101 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.881965:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609100672 00000100:00100000:3.0:1780298639.881971:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609100672 00002000:00100000:3.0:1780298639.882049:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x12:0x0] (499712->503807) req@ffff9cf243ed9180 x1866777609100672/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:345/0 lens 4584/0 e 0 to 0 dl 1780298650 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298639.882066:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243ed9180 with x1866777609100672 ext(499712->503807) 00010000:00010000:3.0:1780298639.882080:0:19943:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000007dc48e1c/0xaf35132e63030639 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [499712->831487] (req 499712->503807) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfcb2 expref: 13 pid: 8356 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.882098:0:19943:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000007dc48e1c/0xaf35132e63030639 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [499712->831487] (req 499712->503807) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cfcb2 expref: 14 pid: 8356 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1780298639.882239:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9cf243ed9180 00000100:00100000:3.0:1780298639.882278:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 160 00000100:00100000:3.0:1780298639.882304:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243ed9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609100672:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:3.0:1780298639.884898:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609100928 00000100:00100000:3.0:1780298639.884905:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609100928 00000100:00100000:3.0:1780298639.891738:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609101696 00000100:00100000:3.0:1780298639.891744:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609101696 00002000:00100000:3.0:1780298639.891777:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (3080192->3084287) req@ffff9cf3481faa00 x1866777609101696/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:345/0 lens 488/0 e 0 to 0 dl 1780298650 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298639.891788:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481faa00 with x1866777609101696 ext(3080192->3084287) 00002000:00010000:3.0:1780298639.891792:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481faa00 00000100:00100000:3.0:1780298639.891815:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5541 00000100:00100000:3.0:1780298639.891823:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481faa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609101696:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298639.891981:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481faa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609101696:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 159us (330us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1780298639.891989:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5541 00002000:00100000:3.0:1780298639.891996:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (3080192->3084287) req@ffff9cf3481faa00 x1866777609101696/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:345/0 lens 488/4536 e 0 to 0 dl 1780298650 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298639.892006:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481faa00 with x1866777609101696 ext(3080192->3084287) 00002000:00010000:3.0:1780298639.892009:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481faa00 00000100:00100000:3.0:1780298639.892817:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.894403:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609101952 00000100:00100000:3.0:1780298639.894411:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609101952 00000100:00100000:3.0:1780298639.894511:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7661 00000100:00100000:3.0:1780298639.894528:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242861c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609101952:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.894558:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.894577:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4b625d7/0xaf35132e63030a61 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d00be expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.894632:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254888000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604698368:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.894692:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.895737:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254888000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604698368:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.895762:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d4b625d7/0xaf35132e63030a61 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d00be expref: 8 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.895821:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4b625d7/0xaf35132e63030a61 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d00be expref: 8 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.895833:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830ce00, rc 0) 00000100:00100000:3.0:1780298639.895870:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242861c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609101952:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1350us (1580us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.895878:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7661 00000100:00100000:3.0:1780298639.900053:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609102592 00000100:00100000:3.0:1780298639.900058:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609102592 00000100:00100000:3.0:1780298639.900100:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5542 00000100:00100000:3.0:1780298639.900109:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242860380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609102592:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:3.0:1780298639.900272:0:19943:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4007180 -> 4007180 00002000:00010000:3.0:1780298639.900276:0:19943:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298640 -> 1780298640 00002000:00010000:3.0:1780298639.900285:0:19943:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298640 -> 1780298640 00002000:00010000:3.0:1780298639.900287:0:19943:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298640 -> 1780298640 00002000:00010000:3.0:1780298639.900290:0:19943:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 520 -> 520 00000100:00100000:3.0:1780298639.900338:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242860380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609102592:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 230us (322us total) trans 4294969471 rc 0/0 00000100:00100000:3.0:1780298639.900345:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5542 00000100:00100000:3.0:1780298639.901184:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609102720 00000100:00100000:3.0:1780298639.901188:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609102720 00000100:00100000:3.0:1780298639.901212:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7666 00000100:00100000:3.0:1780298639.901218:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242861f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609102720:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.901225:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.901232:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a3e0d5df/0xaf35132e63030a7d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d00e1 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.901265:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254888000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604698752:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.901319:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.902051:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254888000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604698752:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.902073:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a3e0d5df/0xaf35132e63030a7d lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d00e1 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.902088:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a3e0d5df/0xaf35132e63030a7d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d00e1 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.902098:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830de00, rc 0) 00000100:00100000:3.0:1780298639.902145:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242861f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609102720:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 926us (980us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.902156:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7666 00000100:00100000:3.0:1780298639.908165:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609103360 00000100:00100000:3.0:1780298639.908169:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609103360 00000100:00100000:3.0:1780298639.908190:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7670 00000100:00100000:3.0:1780298639.908195:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242862a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609103360:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.908201:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.908207:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4b625d7/0xaf35132e63030a99 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d00fd expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.908232:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254888000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604699136:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.908266:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.908814:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254888000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604699136:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.908826:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d4b625d7/0xaf35132e63030a99 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d00fd expref: 13 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.908836:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4b625d7/0xaf35132e63030a99 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d00fd expref: 13 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.908843:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830ce00, rc 0) 00000100:00100000:3.0:1780298639.908864:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242862a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609103360:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 669us (723us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.908868:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7670 00000100:00100000:3.0:1780298639.911301:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609103744 00000100:00100000:3.0:1780298639.911319:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609103744 00000100:00100000:3.0:1780298639.913266:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609103872 00000100:00100000:3.0:1780298639.913271:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609103872 00000100:00100000:3.0:1780298639.913304:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7674 00000100:00100000:3.0:1780298639.913326:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243808700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:29360:x1866777609103872:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.913336:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.913346:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e63030ab5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0119 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.913366:0:19992:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000bf17d487/0xaf35132e63030a30 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d00a2 expref: 12 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.913375:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298639 v=(127 16) 00010000:00010000:3.0:1780298639.913379:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298639 v=(128 16) 00010000:00010000:3.0:1780298639.913384:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e63030ab5 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1826816->1835007] (req 1826816->1835007) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0119 expref: 14 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.913406:0:19992:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000bf17d487/0xaf35132e63030a30 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d00a2 expref: 12 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.913422:0:19992:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000bf17d487/0xaf35132e63030a30 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d00a2 expref: 12 pid: 12101 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.913433:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254888000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604699648:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298639.913476:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.914303:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609104000 00000100:00100000:3.0:1780298639.914344:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609104000 00010000:00010000:3.0:1780298639.914366:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000bf17d487/0xaf35132e63030a30 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d00a2 expref: 13 pid: 12101 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.915504:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.916782:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609104384 00000100:00100000:3.0:1780298639.916786:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609104384 00000100:00100000:3.0:1780298639.916822:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7676 00000100:00100000:3.0:1780298639.916834:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24380b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2398:x1866777609104384:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.916844:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.916852:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d4b625d7/0xaf35132e63030ac3 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0127 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.916880:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25488b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604700032:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.916918:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393c00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.917660:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25488b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604700032:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.917678:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d4b625d7/0xaf35132e63030ac3 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0127 expref: 16 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.917703:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d4b625d7/0xaf35132e63030ac3 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0127 expref: 16 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.917723:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830ce00, rc 0) 00000100:00100000:3.0:1780298639.917751:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24380b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2398:x1866777609104384:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 921us (1000us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.917757:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7676 00000100:00100000:3.0:1780298639.918718:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609104640 00000100:00100000:3.0:1780298639.918732:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609104640 00000100:00100000:3.0:1780298639.918778:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7677 00000100:00100000:3.0:1780298639.918787:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24380bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2399:x1866777609104640:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.918796:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.918804:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a3e0d5df/0xaf35132e63030aca lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0135 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.918831:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25488b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604700160:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.918875:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393c00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.919517:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25488b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604700160:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.919569:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a3e0d5df/0xaf35132e63030aca lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0135 expref: 16 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.919581:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a3e0d5df/0xaf35132e63030aca lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0135 expref: 16 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.919588:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830de00, rc 0) 00000100:00100000:3.0:1780298639.919616:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24380bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2399:x1866777609104640:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 832us (935us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.919621:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7677 00000100:00100000:3.0:1780298639.921943:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609104896 00000100:00100000:3.0:1780298639.921949:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609104896 00000100:00100000:3.0:1780298639.921996:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7680 00000100:00100000:3.0:1780298639.922007:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609104896:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.922016:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.922031:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000037a3562c/0xaf35132e63030adf lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d013c expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.922055:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25488b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604700544:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.922080:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393c00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.922838:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25488b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604700544:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.922855:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000037a3562c/0xaf35132e63030adf lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d013c expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.922868:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000037a3562c/0xaf35132e63030adf lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d013c expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.922880:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830c000, rc 0) 00000100:00100000:3.0:1780298639.922905:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609104896:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 901us (997us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.922917:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7680 00000100:00100000:3.0:1780298639.923880:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609105280 00000100:00100000:3.0:1780298639.923884:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609105280 00000100:00100000:3.0:1780298639.923913:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7682 00000100:00100000:3.0:1780298639.923923:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255489f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2398:x1866777609105280:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.923932:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.923940:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d4b625d7/0xaf35132e63030aed lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0151 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.923965:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25488b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604700800:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.923999:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393c00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.924692:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25488b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604700800:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.924714:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d4b625d7/0xaf35132e63030aed lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0151 expref: 16 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.924737:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d4b625d7/0xaf35132e63030aed lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0151 expref: 16 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.924752:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830ce00, rc 0) 00000100:00100000:3.0:1780298639.924778:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255489f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2398:x1866777609105280:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 856us (919us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.924786:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7682 00000100:00100000:3.0:1780298639.926996:0:6477:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255488380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6477:1866777604700928:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298639.927016:0:6477:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37c00/0xaf35132e63030af4 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 85 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.927253:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:3.0:1780298639.927450:0:6477:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37c00/0xaf35132e63030af4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 84 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.927456:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37c00/0xaf35132e63030af4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 84 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1780298639.927526:0:6477:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294990912 00000004:00010000:3.0:1780298639.927553:0:6477:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000d0d52ade reply state 00000000f5bc8bd0 transno 4294990913 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37c00/0xaf35132e63030af4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 84 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.927609:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37c00/0xaf35132e63030af4 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 84 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.927628:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298639.927634:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1780298639.927638:0:6477:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1780298639.927694:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254918a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+112:29360:x1866777609105536:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1619us (1696us total) trans 4294990913 rc 0/0 00000100:00100000:3.0:1780298639.927710:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61795 00000020:00080000:3.0:1780298639.927716:0:6477:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:3.0:1780298639.927748:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bee00/0xaf35132e6302c09b lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x31:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.927767:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bee00/0xaf35132e6302c09b lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x31:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.927778:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bee00/0xaf35132e6302c09b lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x31:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.927784:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf2445bee00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298639.927789:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bee00/0xaf35132e6302c09b lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x31:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.927800:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bee00/0xaf35132e6302c09b lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x31:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.927822:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x31:0x0].0x0 (ffff9cf2447a4200) 00010000:00010000:3.0:1780298639.927825:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298639.927829:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bee00/0xaf35132e6302c09b lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x31:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.927858:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bee00/0xaf35132e6302c09b lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x31:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.927880:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bec00/0xaf35132e6302c158 lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x5950b203 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.927890:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bec00/0xaf35132e6302c158 lrc: 2/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x5950b203 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.927907:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bec00/0xaf35132e6302c158 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x5950b203 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.927921:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf2445bec00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298639.927927:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bec00/0xaf35132e6302c158 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x5950b203 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.927942:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bec00/0xaf35132e6302c158 lrc: 4/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x5950b203 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.927960:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5950b203 (ffff9cf2447a4900) 00010000:00010000:3.0:1780298639.927964:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298639.927968:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bec00/0xaf35132e6302c158 lrc: 2/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x5950b203 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.927987:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bec00/0xaf35132e6302c158 lrc: 0/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x5950b203 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.927999:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d79a00/0xaf35132e6302c189 lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x5950b203 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928016:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d79a00/0xaf35132e6302c189 lrc: 2/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x5950b203 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928039:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d79a00/0xaf35132e6302c189 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x5950b203 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928047:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf375d79a00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298639.928050:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d79a00/0xaf35132e6302c189 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x5950b203 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928060:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d79a00/0xaf35132e6302c189 lrc: 4/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x5950b203 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928074:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5950b203 (ffff9cf2447a4900) 00010000:00010000:3.0:1780298639.928077:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298639.928079:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d79a00/0xaf35132e6302c189 lrc: 2/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x5950b203 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928091:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d79a00/0xaf35132e6302c189 lrc: 0/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x5950b203 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928112:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d79600/0xaf35132e6302c1f9 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 81 type: IBT flags: 0x50210000000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928117:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d79600/0xaf35132e6302c1f9 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 80 type: IBT flags: 0x50210400000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928121:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d79600/0xaf35132e6302c1f9 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 80 type: IBT flags: 0x40210400000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928150:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf375d79600 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298639.928156:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d79600/0xaf35132e6302c1f9 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 80 type: IBT flags: 0x40210400000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928179:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d79600/0xaf35132e6302c1f9 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 81 type: IBT flags: 0x40218400000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928214:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298639.928219:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298639.928224:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d79600/0xaf35132e6302c1f9 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 80 type: IBT flags: 0x44a19400000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928232:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d79600/0xaf35132e6302c1f9 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 80 type: IBT flags: 0x44a19400000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928258:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be000/0xaf35132e6302c277 lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xc9e3b019 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928279:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be000/0xaf35132e6302c277 lrc: 2/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xc9e3b019 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928298:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be000/0xaf35132e6302c277 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xc9e3b019 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928320:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf2445be000 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298639.928326:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be000/0xaf35132e6302c277 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xc9e3b019 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928344:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be000/0xaf35132e6302c277 lrc: 4/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0xc9e3b019 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928363:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xc9e3b019 (ffff9cf2447a4100) 00010000:00010000:3.0:1780298639.928369:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298639.928373:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be000/0xaf35132e6302c277 lrc: 2/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0xc9e3b019 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928391:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be000/0xaf35132e6302c277 lrc: 0/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0xc9e3b019 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928421:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d78400/0xaf35132e6302c2a8 lrc: 3/0,1 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x257bbe02 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928428:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d78400/0xaf35132e6302c2a8 lrc: 2/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x257bbe02 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928443:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d78400/0xaf35132e6302c2a8 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x257bbe02 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928450:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf375d78400 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298639.928455:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d78400/0xaf35132e6302c2a8 lrc: 3/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x257bbe02 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928459:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d78400/0xaf35132e6302c2a8 lrc: 4/0,0 mode: TXN/TXN res: [0x200000007:0x1:0x0].0x257bbe02 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928467:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x257bbe02 (ffff9cf2447a4600) 00010000:00010000:3.0:1780298639.928473:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298639.928477:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d78400/0xaf35132e6302c2a8 lrc: 2/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x257bbe02 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928494:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d78400/0xaf35132e6302c2a8 lrc: 0/0,0 mode: --/TXN res: [0x200000007:0x1:0x0].0x257bbe02 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928521:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d78600/0xaf35132e6302c2bd lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 76 type: IBT flags: 0x50210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928538:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d78600/0xaf35132e6302c2bd lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 77 type: IBT flags: 0x50210400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928548:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d78600/0xaf35132e6302c2bd lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 75 type: IBT flags: 0x40210400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928567:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf375d78600 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298639.928572:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d78600/0xaf35132e6302c2bd lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 76 type: IBT flags: 0x40210400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928581:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d78600/0xaf35132e6302c2bd lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 76 type: IBT flags: 0x40218400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928591:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298639.928594:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298639.928596:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d78600/0xaf35132e6302c2bd lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 75 type: IBT flags: 0x44a19400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928600:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d78600/0xaf35132e6302c2bd lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 75 type: IBT flags: 0x44a19400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928614:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d78a00/0xaf35132e6302c2d2 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 74 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928620:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d78a00/0xaf35132e6302c2d2 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 73 type: IBT flags: 0x50210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928624:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d78a00/0xaf35132e6302c2d2 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 73 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928638:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf375d78a00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298639.928642:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d78a00/0xaf35132e6302c2d2 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 74 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928664:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d78a00/0xaf35132e6302c2d2 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 74 type: IBT flags: 0x40218400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928674:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298639.928677:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298639.928679:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d78a00/0xaf35132e6302c2d2 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 73 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928682:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d78a00/0xaf35132e6302c2d2 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 73 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928694:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d79c00/0xaf35132e6302c2e7 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 73 type: IBT flags: 0x50210000000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928702:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d79c00/0xaf35132e6302c2e7 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 72 type: IBT flags: 0x50210400000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928705:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d79c00/0xaf35132e6302c2e7 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 71 type: IBT flags: 0x40210400000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928722:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf375d79c00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298639.928726:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d79c00/0xaf35132e6302c2e7 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 72 type: IBT flags: 0x40210400000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928740:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d79c00/0xaf35132e6302c2e7 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 71 type: IBT flags: 0x40218400000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928749:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298639.928753:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298639.928756:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d79c00/0xaf35132e6302c2e7 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 72 type: IBT flags: 0x44a19400000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928779:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d79c00/0xaf35132e6302c2e7 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 72 type: IBT flags: 0x44a19400000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928789:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d79800/0xaf35132e6302c2ee lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 70 type: IBT flags: 0x50210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928802:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d79800/0xaf35132e6302c2ee lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 71 type: IBT flags: 0x50210400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928818:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d79800/0xaf35132e6302c2ee lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 71 type: IBT flags: 0x40210400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928828:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf375d79800 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298639.928832:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d79800/0xaf35132e6302c2ee lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 71 type: IBT flags: 0x40210400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928836:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d79800/0xaf35132e6302c2ee lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 69 type: IBT flags: 0x40218400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928856:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298639.928861:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298639.928864:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d79800/0xaf35132e6302c2ee lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 70 type: IBT flags: 0x44a19400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928870:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d79800/0xaf35132e6302c2ee lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 69 type: IBT flags: 0x44a19400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928881:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841c600/0xaf35132e6302c2f5 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 69 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928894:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841c600/0xaf35132e6302c2f5 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 69 type: IBT flags: 0x50210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928903:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841c600/0xaf35132e6302c2f5 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 68 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928912:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf34841c600 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298639.928916:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841c600/0xaf35132e6302c2f5 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 69 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928928:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841c600/0xaf35132e6302c2f5 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 69 type: IBT flags: 0x40218400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928947:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298639.928949:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298639.928951:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841c600/0xaf35132e6302c2f5 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 67 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928955:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841c600/0xaf35132e6302c2f5 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 68 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928977:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d78000/0xaf35132e6302c2fc lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 67 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928989:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d78000/0xaf35132e6302c2fc lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 67 type: IBT flags: 0x50210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.928994:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d78000/0xaf35132e6302c2fc lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 66 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929001:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf375d78000 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298639.929006:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d78000/0xaf35132e6302c2fc lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 67 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929030:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d78000/0xaf35132e6302c2fc lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 67 type: IBT flags: 0x40218400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929046:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298639.929051:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298639.929054:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d78000/0xaf35132e6302c2fc lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 67 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929068:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d78000/0xaf35132e6302c2fc lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 66 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929083:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36400/0xaf35132e6302c381 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 65 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929097:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36400/0xaf35132e6302c381 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 64 type: IBT flags: 0x50210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929101:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36400/0xaf35132e6302c381 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 64 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929107:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf244b36400 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298639.929111:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36400/0xaf35132e6302c381 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 65 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929144:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36400/0xaf35132e6302c381 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 65 type: IBT flags: 0x40218400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929167:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298639.929170:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298639.929174:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36400/0xaf35132e6302c381 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 65 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929188:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36400/0xaf35132e6302c381 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 63 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929207:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841ca00/0xaf35132e6302c3c7 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 63 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929222:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841ca00/0xaf35132e6302c3c7 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 63 type: IBT flags: 0x50210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929228:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841ca00/0xaf35132e6302c3c7 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 62 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929234:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf34841ca00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298639.929237:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841ca00/0xaf35132e6302c3c7 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 63 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929253:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841ca00/0xaf35132e6302c3c7 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 63 type: IBT flags: 0x40218400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929320:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298639.929323:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298639.929325:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841ca00/0xaf35132e6302c3c7 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 61 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929334:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841ca00/0xaf35132e6302c3c7 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 62 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929348:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfa00/0xaf35132e6302c414 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 60 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929353:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfa00/0xaf35132e6302c414 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 60 type: IBT flags: 0x50210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929361:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfa00/0xaf35132e6302c414 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 60 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929370:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf2445bfa00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298639.929373:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfa00/0xaf35132e6302c414 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 61 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929389:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfa00/0xaf35132e6302c414 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 60 type: IBT flags: 0x40218400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929403:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298639.929405:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298639.929407:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfa00/0xaf35132e6302c414 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 59 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929412:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfa00/0xaf35132e6302c414 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 60 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929426:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841c000/0xaf35132e6302c499 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 59 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929435:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841c000/0xaf35132e6302c499 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 58 type: IBT flags: 0x50210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929441:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841c000/0xaf35132e6302c499 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 58 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929451:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf34841c000 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298639.929513:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841c000/0xaf35132e6302c499 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 58 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929550:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841c000/0xaf35132e6302c499 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 57 type: IBT flags: 0x40218400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929571:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298639.929576:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298639.929580:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841c000/0xaf35132e6302c499 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 57 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929592:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841c000/0xaf35132e6302c499 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 57 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929605:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf24450d200/0xaf35132e6302c652 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 56 type: IBT flags: 0x50210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929619:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf24450d200/0xaf35132e6302c652 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 56 type: IBT flags: 0x50210400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929625:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf24450d200/0xaf35132e6302c652 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 54 type: IBT flags: 0x40210400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929639:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf24450d200 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298639.929645:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf24450d200/0xaf35132e6302c652 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 55 type: IBT flags: 0x40210400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929660:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf24450d200/0xaf35132e6302c652 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 54 type: IBT flags: 0x40218400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929685:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298639.929688:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298639.929691:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf24450d200/0xaf35132e6302c652 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 54 type: IBT flags: 0x44a19400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929716:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf24450d200/0xaf35132e6302c652 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 55 type: IBT flags: 0x44a19400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929739:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243376400/0xaf35132e6302c6d7 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 53 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929752:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243376400/0xaf35132e6302c6d7 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 52 type: IBT flags: 0x50210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929758:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243376400/0xaf35132e6302c6d7 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 52 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929770:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf243376400 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298639.929774:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243376400/0xaf35132e6302c6d7 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 53 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929795:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243376400/0xaf35132e6302c6d7 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 52 type: IBT flags: 0x40218400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929871:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298639.929875:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298639.929879:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243376400/0xaf35132e6302c6d7 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 51 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929893:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243376400/0xaf35132e6302c6d7 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 51 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929919:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf24450d400/0xaf35132e6302c794 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 49 type: IBT flags: 0x50210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929927:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf24450d400/0xaf35132e6302c794 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 49 type: IBT flags: 0x50210400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929938:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf24450d400/0xaf35132e6302c794 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 49 type: IBT flags: 0x40210400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929951:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf24450d400 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298639.929957:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf24450d400/0xaf35132e6302c794 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 50 type: IBT flags: 0x40210400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.929973:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf24450d400/0xaf35132e6302c794 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 50 type: IBT flags: 0x40218400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.930081:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298639.930086:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298639.930090:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf24450d400/0xaf35132e6302c794 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 49 type: IBT flags: 0x44a19400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.930202:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf24450d400/0xaf35132e6302c794 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 48 type: IBT flags: 0x44a19400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.930216:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243377000/0xaf35132e6302cba0 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 47 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.930233:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243377000/0xaf35132e6302cba0 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 47 type: IBT flags: 0x50210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.930243:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243377000/0xaf35132e6302cba0 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 45 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.930268:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf243377000 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298639.930275:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243377000/0xaf35132e6302cba0 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 46 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.930383:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243377000/0xaf35132e6302cba0 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 45 type: IBT flags: 0x40218400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.930405:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298639.930411:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298639.930414:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243377000/0xaf35132e6302cba0 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 45 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.930433:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243377000/0xaf35132e6302cba0 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 45 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.930465:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf24450c000/0xaf35132e6302d070 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 43 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.930485:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf24450c000/0xaf35132e6302d070 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 44 type: IBT flags: 0x50210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.930506:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf24450c000/0xaf35132e6302d070 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 44 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.930520:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf24450c000 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298639.930562:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf24450c000/0xaf35132e6302d070 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 43 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.930577:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf24450c000/0xaf35132e6302d070 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 43 type: IBT flags: 0x40218400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.930593:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298639.930601:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298639.930605:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf24450c000/0xaf35132e6302d070 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 42 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.930643:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf24450c000/0xaf35132e6302d070 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 43 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.930661:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243376200/0xaf35132e6302d188 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 41 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.930677:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243376200/0xaf35132e6302d188 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 40 type: IBT flags: 0x50210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.930684:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243376200/0xaf35132e6302d188 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 40 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.930701:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf243376200 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298639.930707:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243376200/0xaf35132e6302d188 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 41 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.930715:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243376200/0xaf35132e6302d188 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 41 type: IBT flags: 0x40218400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.930735:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298639.930739:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298639.930742:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243376200/0xaf35132e6302d188 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 40 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.930779:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243376200/0xaf35132e6302d188 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 40 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.930795:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243376e00/0xaf35132e6302d4b4 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 39 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.930848:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243376e00/0xaf35132e6302d4b4 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 38 type: IBT flags: 0x50210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.930859:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243376e00/0xaf35132e6302d4b4 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 38 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.930875:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf243376e00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298639.930880:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243376e00/0xaf35132e6302d4b4 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 39 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.930895:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243376e00/0xaf35132e6302d4b4 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 39 type: IBT flags: 0x40218400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.930921:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298639.930928:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298639.930932:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243376e00/0xaf35132e6302d4b4 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 37 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.930952:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243376e00/0xaf35132e6302d4b4 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 38 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.930979:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93400/0xaf35132e6302d58d lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 36 type: IBT flags: 0x50210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.930984:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93400/0xaf35132e6302d58d lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 36 type: IBT flags: 0x50210400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.930991:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93400/0xaf35132e6302d58d lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 36 type: IBT flags: 0x40210400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.931008:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf375e93400 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298639.931012:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93400/0xaf35132e6302d58d lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 37 type: IBT flags: 0x40210400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.931050:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93400/0xaf35132e6302d58d lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 37 type: IBT flags: 0x40218400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.931071:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298639.931078:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298639.931082:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93400/0xaf35132e6302d58d lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 35 type: IBT flags: 0x44a19400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.931101:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93400/0xaf35132e6302d58d lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 36 type: IBT flags: 0x44a19400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.931144:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b200/0xaf35132e6302d5a9 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 35 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.931153:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b200/0xaf35132e6302d5a9 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 34 type: IBT flags: 0x50210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.931167:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b200/0xaf35132e6302d5a9 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 34 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.931183:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf243d9b200 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298639.931193:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b200/0xaf35132e6302d5a9 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 35 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.931211:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b200/0xaf35132e6302d5a9 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 34 type: IBT flags: 0x40218400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.931232:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298639.931237:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298639.931241:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b200/0xaf35132e6302d5a9 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 33 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.931281:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b200/0xaf35132e6302d5a9 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 34 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.931305:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92400/0xaf35132e6302d62e lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 33 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.931328:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92400/0xaf35132e6302d62e lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 32 type: IBT flags: 0x50210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.931332:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92400/0xaf35132e6302d62e lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 32 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.931341:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf375e92400 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298639.931347:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92400/0xaf35132e6302d62e lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 32 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.931357:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92400/0xaf35132e6302d62e lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 33 type: IBT flags: 0x40218400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.931377:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298639.931379:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298639.931382:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92400/0xaf35132e6302d62e lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 31 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.931388:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92400/0xaf35132e6302d62e lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 32 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.931413:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e6302db8a lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 31 type: IBT flags: 0x50210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.931458:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e6302db8a lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 30 type: IBT flags: 0x50210400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.931468:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e6302db8a lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 30 type: IBT flags: 0x40210400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.931476:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf243278800 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298639.931482:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e6302db8a lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 31 type: IBT flags: 0x40210400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.931500:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e6302db8a lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 31 type: IBT flags: 0x40218400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.931524:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298639.931529:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298639.931535:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e6302db8a lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 31 type: IBT flags: 0x44a19400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.931548:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e6302db8a lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 29 type: IBT flags: 0x44a19400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.931628:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279400/0xaf35132e6302e00d lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 28 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.931635:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279400/0xaf35132e6302e00d lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 28 type: IBT flags: 0x50210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.931641:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279400/0xaf35132e6302e00d lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 28 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.931649:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf243279400 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298639.931654:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279400/0xaf35132e6302e00d lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 29 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.931762:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279400/0xaf35132e6302e00d lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 27 type: IBT flags: 0x40218400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.931780:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298639.931787:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298639.931791:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279400/0xaf35132e6302e00d lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 26 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.931802:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279400/0xaf35132e6302e00d lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 27 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.931829:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279a00/0xaf35132e6302e23d lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 24 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.931840:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279a00/0xaf35132e6302e23d lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 24 type: IBT flags: 0x50210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.931852:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279a00/0xaf35132e6302e23d lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 24 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.931861:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf243279a00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298639.931866:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279a00/0xaf35132e6302e23d lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 24 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.931882:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279a00/0xaf35132e6302e23d lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 24 type: IBT flags: 0x40218400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.931892:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298639.931897:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298639.931901:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279a00/0xaf35132e6302e23d lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 24 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.931980:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279a00/0xaf35132e6302e23d lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 24 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.931992:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92c00/0xaf35132e6302e3e8 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 23 type: IBT flags: 0x50210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.931998:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92c00/0xaf35132e6302e3e8 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 23 type: IBT flags: 0x50210400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932003:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92c00/0xaf35132e6302e3e8 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 23 type: IBT flags: 0x40210400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932007:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf375e92c00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298639.932010:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92c00/0xaf35132e6302e3e8 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 23 type: IBT flags: 0x40210400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932015:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92c00/0xaf35132e6302e3e8 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 23 type: IBT flags: 0x40218400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932022:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298639.932025:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298639.932027:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92c00/0xaf35132e6302e3e8 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 23 type: IBT flags: 0x44a19400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932032:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92c00/0xaf35132e6302e3e8 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 23 type: IBT flags: 0x44a19400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932041:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92600/0xaf35132e6302e43c lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 22 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932046:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92600/0xaf35132e6302e43c lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 22 type: IBT flags: 0x50210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932057:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92600/0xaf35132e6302e43c lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 22 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932061:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf375e92600 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298639.932064:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92600/0xaf35132e6302e43c lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 22 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932156:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92600/0xaf35132e6302e43c lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 22 type: IBT flags: 0x40218400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932167:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298639.932171:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298639.932175:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92600/0xaf35132e6302e43c lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 22 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932180:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92600/0xaf35132e6302e43c lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 22 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932192:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92e00/0xaf35132e6302e70d lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 21 type: IBT flags: 0x50210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932198:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92e00/0xaf35132e6302e70d lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 21 type: IBT flags: 0x50210400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932205:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92e00/0xaf35132e6302e70d lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 21 type: IBT flags: 0x40210400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932210:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf375e92e00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298639.932214:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92e00/0xaf35132e6302e70d lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 21 type: IBT flags: 0x40210400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932231:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92e00/0xaf35132e6302e70d lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 21 type: IBT flags: 0x40218400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932319:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298639.932322:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298639.932326:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92e00/0xaf35132e6302e70d lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 21 type: IBT flags: 0x44a19400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932337:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92e00/0xaf35132e6302e70d lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 21 type: IBT flags: 0x44a19400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932358:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93c00/0xaf35132e6302e776 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 20 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932522:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93c00/0xaf35132e6302e776 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 20 type: IBT flags: 0x50210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932532:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93c00/0xaf35132e6302e776 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 20 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932538:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf375e93c00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298639.932542:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93c00/0xaf35132e6302e776 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 20 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932549:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93c00/0xaf35132e6302e776 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 20 type: IBT flags: 0x40218400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932559:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298639.932563:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298639.932567:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93c00/0xaf35132e6302e776 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 20 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932583:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93c00/0xaf35132e6302e776 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 20 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932603:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf36d5a8000/0xaf35132e6302e7e6 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 19 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932611:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf36d5a8000/0xaf35132e6302e7e6 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 19 type: IBT flags: 0x50210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932617:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf36d5a8000/0xaf35132e6302e7e6 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 19 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932624:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf36d5a8000 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298639.932631:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf36d5a8000/0xaf35132e6302e7e6 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 19 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932637:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf36d5a8000/0xaf35132e6302e7e6 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 19 type: IBT flags: 0x40218400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932705:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298639.932709:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298639.932713:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf36d5a8000/0xaf35132e6302e7e6 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 19 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932727:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf36d5a8000/0xaf35132e6302e7e6 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 19 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932753:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279800/0xaf35132e6302ee7d lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 18 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932763:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279800/0xaf35132e6302ee7d lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 18 type: IBT flags: 0x50210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932784:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279800/0xaf35132e6302ee7d lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 18 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932791:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf243279800 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298639.932795:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279800/0xaf35132e6302ee7d lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 18 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932801:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279800/0xaf35132e6302ee7d lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 18 type: IBT flags: 0x40218400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932820:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298639.932823:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298639.932826:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279800/0xaf35132e6302ee7d lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 18 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932832:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279800/0xaf35132e6302ee7d lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 18 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932847:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157c00/0xaf35132e6302ef9c lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 17 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932853:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157c00/0xaf35132e6302ef9c lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 17 type: IBT flags: 0x50210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932858:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157c00/0xaf35132e6302ef9c lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 17 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932863:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf243157c00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298639.932867:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157c00/0xaf35132e6302ef9c lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 17 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932881:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157c00/0xaf35132e6302ef9c lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 17 type: IBT flags: 0x40218400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932890:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298639.932892:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298639.932894:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157c00/0xaf35132e6302ef9c lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 17 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932900:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157c00/0xaf35132e6302ef9c lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 17 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932913:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279e00/0xaf35132e6302f075 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 16 type: IBT flags: 0x50210000000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932920:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279e00/0xaf35132e6302f075 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 16 type: IBT flags: 0x50210400000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932926:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279e00/0xaf35132e6302f075 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 16 type: IBT flags: 0x40210400000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932932:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf243279e00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298639.932972:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279e00/0xaf35132e6302f075 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 16 type: IBT flags: 0x40210400000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932980:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279e00/0xaf35132e6302f075 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 16 type: IBT flags: 0x40218400000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.932989:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298639.932994:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298639.932998:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279e00/0xaf35132e6302f075 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 16 type: IBT flags: 0x44a19400000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933020:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279e00/0xaf35132e6302f075 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 16 type: IBT flags: 0x44a19400000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933044:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156400/0xaf35132e6302f473 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 15 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933063:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156400/0xaf35132e6302f473 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 15 type: IBT flags: 0x50210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933072:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156400/0xaf35132e6302f473 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 15 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933087:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf243156400 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298639.933205:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156400/0xaf35132e6302f473 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 15 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933211:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156400/0xaf35132e6302f473 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 15 type: IBT flags: 0x40218400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933219:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298639.933222:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298639.933224:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156400/0xaf35132e6302f473 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 15 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933230:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156400/0xaf35132e6302f473 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 15 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933242:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278c00/0xaf35132e6302f561 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 14 type: IBT flags: 0x50210000000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933249:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278c00/0xaf35132e6302f561 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 14 type: IBT flags: 0x50210400000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933260:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278c00/0xaf35132e6302f561 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 14 type: IBT flags: 0x40210400000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933266:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf243278c00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298639.933269:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278c00/0xaf35132e6302f561 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 14 type: IBT flags: 0x40210400000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933277:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278c00/0xaf35132e6302f561 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 14 type: IBT flags: 0x40218400000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933285:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298639.933288:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298639.933291:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278c00/0xaf35132e6302f561 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 14 type: IBT flags: 0x44a19400000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933297:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278c00/0xaf35132e6302f561 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 14 type: IBT flags: 0x44a19400000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933321:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2000/0xaf35132e6302fcb5 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933328:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2000/0xaf35132e6302fcb5 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 13 type: IBT flags: 0x50210400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933334:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2000/0xaf35132e6302fcb5 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 13 type: IBT flags: 0x40210400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933340:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf344ff2000 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298639.933343:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2000/0xaf35132e6302fcb5 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 13 type: IBT flags: 0x40210400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933359:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2000/0xaf35132e6302fcb5 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 13 type: IBT flags: 0x40218400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933368:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298639.933370:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298639.933374:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2000/0xaf35132e6302fcb5 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 13 type: IBT flags: 0x44a19400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933380:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2000/0xaf35132e6302fcb5 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 13 type: IBT flags: 0x44a19400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933392:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a600/0xaf35132e6302fcf4 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 12 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933399:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a600/0xaf35132e6302fcf4 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 12 type: IBT flags: 0x50210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933405:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a600/0xaf35132e6302fcf4 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 12 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933411:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf243d9a600 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298639.933414:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a600/0xaf35132e6302fcf4 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 12 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933420:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a600/0xaf35132e6302fcf4 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 12 type: IBT flags: 0x40218400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933427:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298639.933429:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298639.933432:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a600/0xaf35132e6302fcf4 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 12 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933447:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a600/0xaf35132e6302fcf4 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 12 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933458:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2c00/0xaf35132e6302fd09 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 11 type: IBT flags: 0x50210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933464:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2c00/0xaf35132e6302fd09 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 11 type: IBT flags: 0x50210400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933471:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2c00/0xaf35132e6302fd09 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 11 type: IBT flags: 0x40210400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933476:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf344ff2c00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298639.933479:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2c00/0xaf35132e6302fd09 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 11 type: IBT flags: 0x40210400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933485:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2c00/0xaf35132e6302fd09 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 11 type: IBT flags: 0x40218400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933494:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298639.933497:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298639.933499:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2c00/0xaf35132e6302fd09 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 11 type: IBT flags: 0x44a19400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933505:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2c00/0xaf35132e6302fd09 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 11 type: IBT flags: 0x44a19400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933515:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3a00/0xaf35132e6302fec2 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 10 type: IBT flags: 0x50210000000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933522:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3a00/0xaf35132e6302fec2 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 10 type: IBT flags: 0x50210400000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933533:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3a00/0xaf35132e6302fec2 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 10 type: IBT flags: 0x40210400000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933538:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf344ff3a00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298639.933541:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3a00/0xaf35132e6302fec2 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 10 type: IBT flags: 0x40210400000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933548:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3a00/0xaf35132e6302fec2 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 10 type: IBT flags: 0x40218400000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933556:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298639.933559:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298639.933562:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3a00/0xaf35132e6302fec2 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 10 type: IBT flags: 0x44a19400000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933568:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3a00/0xaf35132e6302fec2 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 10 type: IBT flags: 0x44a19400000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933637:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2e00/0xaf35132e63030043 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x50210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933645:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2e00/0xaf35132e63030043 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x50210400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933651:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2e00/0xaf35132e63030043 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x40210400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933657:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf344ff2e00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298639.933661:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2e00/0xaf35132e63030043 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x40210400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933682:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2e00/0xaf35132e63030043 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x40218400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933693:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298639.933696:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298639.933700:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2e00/0xaf35132e63030043 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x44a19400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933709:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2e00/0xaf35132e63030043 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x44a19400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933725:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3e00/0xaf35132e630300cf lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933742:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3e00/0xaf35132e630300cf lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50210400000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933749:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3e00/0xaf35132e630300cf lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x40210400000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933755:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf344ff3e00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298639.933757:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3e00/0xaf35132e630300cf lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x40210400000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933767:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3e00/0xaf35132e630300cf lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x40218400000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933780:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298639.933785:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298639.933789:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3e00/0xaf35132e630300cf lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x44a19400000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933818:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3e00/0xaf35132e630300cf lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x44a19400000000 pid: 6478 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933842:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37e00/0xaf35132e63030a22 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933850:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37e00/0xaf35132e63030a22 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933858:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37e00/0xaf35132e63030a22 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933869:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf244b37e00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298639.933874:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37e00/0xaf35132e63030a22 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933883:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37e00/0xaf35132e63030a22 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933894:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298639.933897:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298639.933900:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37e00/0xaf35132e63030a22 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.933907:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37e00/0xaf35132e63030a22 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.934626:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609106688 00000100:00100000:3.0:1780298639.934631:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609106688 00000100:00100000:3.0:1780298639.934729:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7686 00000100:00100000:3.0:1780298639.934738:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:29360:x1866777609106688:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.934753:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.934763:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d4b625d7/0xaf35132e63030b10 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d015f expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.934782:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e63030ab5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1826816->35381247] (req 1826816->1835007) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0119 expref: 16 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.934790:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298639 v=(128 16) 00010000:00010000:3.0:1780298639.934794:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298639 v=(129 16) 00010000:00010000:3.0:1780298639.934798:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d4b625d7/0xaf35132e63030b10 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d015f expref: 16 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.934820:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e63030ab5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1826816->35381247] (req 1826816->1835007) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0119 expref: 16 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.934832:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000002dc12793/0xaf35132e63030ab5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1826816->35381247] (req 1826816->1835007) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0119 expref: 16 pid: 19992 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.934842:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25488b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604702080:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298639.934880:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.935425:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25488b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604702080:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298639.935439:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d4b625d7/0xaf35132e63030b10 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d015f expref: 16 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.935450:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830ce00, rc 0) 00000100:00100000:3.0:1780298639.935486:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:29360:x1866777609106688:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 750us (888us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298639.935492:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7686 00000100:00100000:3.0:1780298639.936630:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609107072 00000100:00100000:3.0:1780298639.936635:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609107072 00002000:00010000:3.0:1780298639.936653:0:19943:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (1105219->18446744073709551615) 00002000:00100000:3.0:1780298639.936658:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25586f800 with x1866777609107072 ext(1105219->18446744073709551615) 00002000:00010000:3.0:1780298639.936666:0:19943:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf25586f800. 00000100:00100000:3.0:1780298639.936695:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5546 00000100:00100000:3.0:1780298639.936703:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25586f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2399:x1866777609107072:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:3.0:1780298639.936868:0:19943:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 0 -> 1105219 00002000:00010000:3.0:1780298639.936873:0:19943:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298640 -> 1780298640 00002000:00010000:3.0:1780298639.936876:0:19943:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298640 -> 1780298640 00002000:00010000:3.0:1780298639.936879:0:19943:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298640 -> 1780298640 00002000:00010000:3.0:1780298639.936881:0:19943:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 16 -> 0 00000100:00100000:3.0:1780298639.936906:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25586f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2399:x1866777609107072:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 203us (326us total) trans 4294969078 rc 0/0 00000100:00100000:3.0:1780298639.936914:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5546 00002000:00010000:3.0:1780298639.936918:0:19943:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (1105219->18446744073709551615) 00002000:00100000:3.0:1780298639.936921:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25586f800 with x1866777609107072 ext(1105219->18446744073709551615) 00002000:00010000:3.0:1780298639.936926:0:19943:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf25586f800. 00000100:00100000:3.0:1780298639.944204:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609107840 00000100:00100000:3.0:1780298639.944210:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609107840 00000100:00100000:3.0:1780298639.944262:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7689 00000100:00100000:3.0:1780298639.944275:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609107840:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.944286:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.944322:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037a3562c/0xaf35132e63030b2c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0174 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.944361:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25488b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604702464:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.944437:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00002000:00010000:3.0:1780298639.945218:0:8358:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks: 0 -> 8 00000100:00100000:3.0:1780298639.945232:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25488b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604702464:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.945253:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000037a3562c/0xaf35132e63030b2c lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0174 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.945272:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037a3562c/0xaf35132e63030b2c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0174 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.945280:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830c000, rc 0) 00000100:00100000:3.0:1780298639.945317:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609107840:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1036us (1185us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.945330:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7689 00000100:00100000:3.0:1780298639.946399:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609107968 00000100:00100000:3.0:1780298639.946406:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609107968 00000100:00100000:3.0:1780298639.946446:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7690 00000100:00100000:3.0:1780298639.946456:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609107968:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.946467:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.946477:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a3e0d5df/0xaf35132e63030b33 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d017b expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.946513:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25488b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604702592:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.946576:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.947709:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25488b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604702592:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.947735:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a3e0d5df/0xaf35132e63030b33 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d017b expref: 12 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.947755:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a3e0d5df/0xaf35132e63030b33 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d017b expref: 12 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.947780:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830de00, rc 0) 00000100:00100000:3.0:1780298639.947820:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609107968:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1363us (1450us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.947831:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7690 00000100:00100000:3.0:1780298639.951765:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7692 00000100:00100000:3.0:1780298639.951812:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2541ea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609108480:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.951823:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.951834:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000037a3562c/0xaf35132e63030b4f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0190 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.951879:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25488b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604702848:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.951935:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.952717:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25488b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604702848:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.952739:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000037a3562c/0xaf35132e63030b4f lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0190 expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.952770:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000037a3562c/0xaf35132e63030b4f lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0190 expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.952789:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830c000, rc 0) 00000100:00100000:3.0:1780298639.952815:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2541ea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609108480:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1005us (1157us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.952821:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7692 00000100:00100000:3.0:1780298639.953844:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609108864 00000100:00100000:3.0:1780298639.953849:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609108864 00000100:00100000:3.0:1780298639.953876:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7694 00000100:00100000:3.0:1780298639.953883:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2541e8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609108864:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.953891:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.953900:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007dc48e1c/0xaf35132e63030b5d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d01a5 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.953930:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25488b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604703104:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.953966:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.954797:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25488b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604703104:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.954810:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007dc48e1c/0xaf35132e63030b5d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d01a5 expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.954821:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007dc48e1c/0xaf35132e63030b5d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d01a5 expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.954838:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830c800, rc 0) 00000100:00100000:3.0:1780298639.954892:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2541e8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609108864:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1008us (1093us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.954898:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7694 00000100:00100000:3.0:1780298639.965995:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609110016 00000100:00100000:3.0:1780298639.966000:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609110016 00000100:00100000:3.0:1780298639.966035:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5552 00000100:00100000:3.0:1780298639.966040:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b40a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609110016:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:3.0:1780298639.966206:0:19943:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:3.0:1780298639.966209:0:19943:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298640 -> 1780298640 00002000:00010000:3.0:1780298639.966210:0:19943:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298640 -> 1780298640 00002000:00010000:3.0:1780298639.966212:0:19943:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298640 -> 1780298640 00002000:00010000:3.0:1780298639.966213:0:19943:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 584 -> 608 00000100:00100000:3.0:1780298639.966235:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b40a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2399:x1866777609110016:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 195us (282us total) trans 4294969479 rc 0/0 00000100:00100000:3.0:1780298639.966239:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5552 00000100:00100000:3.0:1780298639.968717:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609110272 00000100:00100000:3.0:1780298639.968722:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609110272 00000100:00100000:3.0:1780298639.968752:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7703 00000100:00100000:3.0:1780298639.968761:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609110272:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.968769:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.968778:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007dc48e1c/0xaf35132e63030b9c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d01dd expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.968816:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25488b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604704256:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.968848:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298639.969596:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25488b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604704256:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.969609:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007dc48e1c/0xaf35132e63030b9c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d01dd expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.969619:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007dc48e1c/0xaf35132e63030b9c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d01dd expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.969627:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830c800, rc 0) 00000100:00100000:3.0:1780298639.969651:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609110272:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 892us (986us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298639.969656:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7703 00000100:00080000:3.0:1780298639.971367:0:6473:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1780298639.971538:0:3624:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25491ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3624:1866777604704512:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1780298639.971623:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609110528 00000100:00100000:3.0:1780298639.971628:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609110528 00000100:00100000:3.0:1780298639.971813:0:3624:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25586dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a32054e7-c858-4203-a2c6-5654368a105b:3624:1866777604704640:0@lo:400:kworker.0 00000100:00100000:3.0:1780298639.971871:0:3624:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1780298639.971937:0:3624:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25586c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3624:1866777604705152:0@lo:400:kworker.0 00000100:00100000:3.0:1780298639.972062:0:3624:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1780298639.972100:0:3624:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243cd4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3624:1866777604705792:0@lo:400:kworker.0 00000100:00100000:3.0:1780298639.972377:0:3624:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1780298639.972437:0:3624:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25586dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a32054e7-c858-4203-a2c6-5654368a105b:3624:1866777604704640:0@lo:400:kworker.0 00000100:00100000:3.0:1780298639.972493:0:6476:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777604704768 00000100:00100000:3.0:1780298639.972499:0:6476:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777604704768 00000100:00100000:3.0:1780298639.972540:0:6476:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777604705280 00000100:00100000:3.0:1780298639.972545:0:6476:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777604705280 00000100:00100000:3.0:1780298639.972584:0:6476:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1369 00000100:00100000:3.0:1780298639.972606:0:6476:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243cd7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3623:x1866777604705024:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1780298639.972679:0:6476:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243cd7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3623:x1866777604705024:12345-0@lo:400:kworker.0 Request processed in 80us (482us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298639.972697:0:6476:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1369 00000100:00100000:3.0:1780298639.972735:0:6476:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1372 00000100:00100000:3.0:1780298639.972752:0:6476:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3625:x1866777604705280:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1780298639.972800:0:6476:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3625:x1866777604705280:12345-0@lo:400:kworker.0 Request processed in 49us (305us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298639.972813:0:6476:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1372 00000100:00100000:3.0:1780298639.972892:0:3624:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243cd4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3624:1866777604705792:0@lo:400:kworker.0 00000100:00100000:3.0:1780298639.973233:0:3624:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25586c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3624:1866777604705152:0@lo:400:kworker.0 00000100:00100000:3.0:1780298639.974213:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.977776:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.982557:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.985548:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.987356:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.989785:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609111936 00000100:00100000:3.0:1780298639.989793:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609111936 00000100:00100000:3.0:1780298639.989852:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5555 00000100:00100000:3.0:1780298639.989864:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609111936:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:3.0:1780298639.990076:0:19928:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 1105219 -> 1105219 00002000:00010000:3.0:1780298639.990080:0:19928:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298640 -> 1780298640 00002000:00010000:3.0:1780298639.990083:0:19928:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298640 -> 1780298640 00002000:00010000:3.0:1780298639.990086:0:19928:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298640 -> 1780298640 00002000:00010000:3.0:1780298639.990088:0:19928:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 8 -> 8 00000100:00100000:3.0:1780298639.990160:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609111936:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 299us (448us total) trans 4294969080 rc 0/0 00000100:00100000:3.0:1780298639.990168:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5555 00000100:00100000:3.0:1780298639.991064:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.991231:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298639.995855:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609112576 00000100:00100000:3.0:1780298639.995865:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609112576 00000100:00100000:3.0:1780298639.995919:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7715 00000100:00100000:3.0:1780298639.995937:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609112576:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298639.995952:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298639.995968:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000037a3562c/0xaf35132e63030bf0 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0231 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.996091:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243cd7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604707712:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298639.996930:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243cd7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604707712:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298639.996963:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000037a3562c/0xaf35132e63030bf0 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0231 expref: 13 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.997030:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000037a3562c/0xaf35132e63030bf0 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0231 expref: 13 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.997061:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830c000, rc 0) 00000100:00100000:3.0:1780298639.999198:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609112960 00000100:00100000:3.0:1780298639.999205:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609112960 00010000:00010000:3.0:1780298639.999236:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000004f507fef/0xaf35132e63030b8e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [237568->1306623] (req 237568->249855) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d01cf expref: 16 pid: 8356 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.999272:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14732 00000100:00100000:3.0:1780298639.999284:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255091880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2399:x1866777609112960:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298639.999296:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298639.999323:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000004f507fef/0xaf35132e63030b8e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [237568->1306623] (req 237568->249855) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d01cf expref: 16 pid: 8356 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.999336:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000004f507fef/0xaf35132e63030b8e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [237568->1306623] (req 237568->249855) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d01cf expref: 16 pid: 8356 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.999353:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004f507fef/0xaf35132e63030b8e lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [237568->1306623] (req 237568->249855) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d01cf expref: 16 pid: 8356 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.999368:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298639 v=(177 22) 00010000:00010000:3.0:1780298639.999372:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298639 v=(177 22) 00010000:00010000:3.0:1780298639.999380:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000008676129/0xaf35132e63030bfe lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [823296->4112383] (req 823296->827391) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d023f expref: 12 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.999390:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000008676129/0xaf35132e63030bfe lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [823296->4112383] (req 823296->827391) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d023f expref: 12 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298639.999428:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000008676129/0xaf35132e63030bfe lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [823296->4112383] (req 823296->827391) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d023f expref: 12 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298639.999441:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243cd7b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777604707968:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298639.999504:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.001925:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609113216 00000100:00100000:3.0:1780298640.001930:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609113216 00002000:00100000:3.0:1780298640.001948:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (827392->831487) req@ffff9cf255091c00 x1866777609113216/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:346/0 lens 488/0 e 0 to 0 dl 1780298651 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298640.001955:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255091c00 with x1866777609113216 ext(827392->831487) 00002000:00010000:3.0:1780298640.001957:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255091c00 00000100:00100000:3.0:1780298640.001972:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5557 00000100:00100000:3.0:1780298640.001978:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255091c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2398:x1866777609113216:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298640.002040:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255091c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2398:x1866777609113216:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 62us (151us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1780298640.002045:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5557 00002000:00100000:3.0:1780298640.002050:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (827392->831487) req@ffff9cf255091c00 x1866777609113216/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:346/0 lens 488/4536 e 0 to 0 dl 1780298651 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298640.002056:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255091c00 with x1866777609113216 ext(827392->831487) 00002000:00010000:3.0:1780298640.002058:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255091c00 00000100:00100000:3.0:1780298640.003187:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.003210:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00080000:3.0:1780298640.003283:0:6473:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1780298640.007540:0:17578:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609113984 00000100:00100000:3.0:1780298640.007545:0:17578:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609113984 00000100:00100000:3.0:1780298640.007582:0:17578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61798 00000100:00100000:3.0:1780298640.007594:0:17578:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255092680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:29360:x1866777609113984:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:3.0:1780298640.007637:0:17578:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298640.007655:0:17578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36800/0xaf35132e63030b1e lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0166 expref: 22 pid: 17567 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.007666:0:17578:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298640.007669:0:17578:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1780298640.007805:0:17578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c000/0xaf35132e63030c21 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT flags: 0x40000000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.007816:0:17578:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3582cb000/0xaf35132e63030b48 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0189 expref: 27 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.007822:0:17578:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c000/0xaf35132e63030c21 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.007840:0:17578:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3582cb000/0xaf35132e63030b48 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0189 expref: 27 pid: 6478 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.007857:0:17578:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3582cb000/0xaf35132e63030b48 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0189 expref: 27 pid: 6478 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.007865:0:17578:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243cd7b80 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000_UUID:17578:1866777604708608:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298640.007898:0:17578:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.008602:0:17578:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243cd7b80 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000_UUID:17578:1866777604708608:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298640.008624:0:17578:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c000/0xaf35132e63030c21 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1780298640.008845:0:6473:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:3.0:1780298640.008960:0:17578:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c000/0xaf35132e63030c21 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.008966:0:17578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c000/0xaf35132e63030c21 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1780298640.009014:0:17578:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294990914 00000004:00010000:3.0:1780298640.009035:0:17578:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000c0d94f2f reply state 00000000e84ad986 transno 4294990915 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c000/0xaf35132e63030c21 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.009043:0:17578:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c000/0xaf35132e63030c21 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.009048:0:17578:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298640.009050:0:17578:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1780298640.009052:0:17578:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1780298640.009100:0:17578:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255092680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+24:29360:x1866777609113984:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1511us (1595us total) trans 4294990915 rc 0/0 00000100:00100000:3.0:1780298640.009187:0:17578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61798 00000020:00080000:3.0:1780298640.009191:0:17578:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1780298640.009851:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609114240 00000100:00100000:3.0:1780298640.009855:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609114240 00000100:00100000:3.0:1780298640.009885:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7722 00000100:00100000:3.0:1780298640.009892:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255093100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:29360:x1866777609114240:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.009900:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.009909:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007dc48e1c/0xaf35132e63030c28 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0262 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.009932:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000ce272b4a/0xaf35132e63030bb8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [4112384->37666815] (req 4112384->4116479) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d01f9 expref: 16 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.009939:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000008676129/0xaf35132e63030bfe lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [823296->4112383] (req 823296->827391) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d023f expref: 12 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.009944:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 2 to [0x2c0000401:0x11:0x0].0x0 time=1780298639 v=(177 22) 00010000:00010000:3.0:1780298640.009947:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298640 v=(2 177) 00010000:00010000:3.0:1780298640.009949:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007dc48e1c/0xaf35132e63030c28 lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0262 expref: 16 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.009963:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000008676129/0xaf35132e63030bfe lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [823296->4112383] (req 823296->827391) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d023f expref: 12 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.009970:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000008676129/0xaf35132e63030bfe lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [823296->4112383] (req 823296->827391) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d023f expref: 12 pid: 8359 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.009978:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243cd7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604708736:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298640.010017:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000ce272b4a/0xaf35132e63030bb8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [4112384->37666815] (req 4112384->4116479) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d01f9 expref: 16 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.010025:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000ce272b4a/0xaf35132e63030bb8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [4112384->37666815] (req 4112384->4116479) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d01f9 expref: 16 pid: 8356 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.010041:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25586f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604708864:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298640.010059:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.010705:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243cd7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604708736:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298640.010780:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25586f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604708864:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298640.010835:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007dc48e1c/0xaf35132e63030c28 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0262 expref: 16 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.010848:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830c800, rc 0) 00000100:00100000:3.0:1780298640.010876:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255093100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:29360:x1866777609114240:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 984us (1047us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298640.010882:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7722 00000100:00100000:3.0:1780298640.011151:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.016589:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.016799:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.020220:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609115520 00000100:00100000:3.0:1780298640.020227:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609115520 00000100:00100000:3.0:1780298640.020272:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7726 00000100:00100000:3.0:1780298640.020283:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242861f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609115520:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.020298:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.020323:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a3e0d5df/0xaf35132e63030c59 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0285 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.020369:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25586ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604709760:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.020423:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.023624:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.024912:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.028429:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.030388:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.034723:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.037530:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.042170:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.044249:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609117312 00000100:00100000:3.0:1780298640.044253:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609117312 00002000:00100000:3.0:1780298640.044287:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (122880->126975) req@ffff9cf3481f9c00 x1866777609117312/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:346/0 lens 488/0 e 0 to 0 dl 1780298651 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298640.044303:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481f9c00 with x1866777609117312 ext(122880->126975) 00002000:00010000:3.0:1780298640.044317:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481f9c00 00000100:00100000:3.0:1780298640.044341:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5562 00000100:00100000:3.0:1780298640.044350:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609117312:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298640.044485:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609117312:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 136us (269us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1780298640.044494:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5562 00002000:00100000:3.0:1780298640.044502:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (122880->126975) req@ffff9cf3481f9c00 x1866777609117312/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:346/0 lens 488/4536 e 0 to 0 dl 1780298651 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298640.044509:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481f9c00 with x1866777609117312 ext(122880->126975) 00002000:00010000:3.0:1780298640.044514:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481f9c00 00000100:00100000:3.0:1780298640.045832:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.054653:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609118336 00000100:00100000:3.0:1780298640.054665:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609118336 00010000:00010000:3.0:1780298640.054686:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000bf17d487/0xaf35132e63030c60 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2220032->2224127) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0293 expref: 13 pid: 20252 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.054725:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14740 00000100:00100000:3.0:1780298640.054738:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f2680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2399:x1866777609118336:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298640.054751:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298640.054765:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000bf17d487/0xaf35132e63030c60 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2220032->2224127) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0293 expref: 13 pid: 20252 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.054782:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000bf17d487/0xaf35132e63030c60 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2220032->2224127) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0293 expref: 13 pid: 20252 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.054803:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bf17d487/0xaf35132e63030c60 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2220032->2224127) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0293 expref: 13 pid: 20252 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.054815:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298640 v=(3 177) 00010000:00010000:3.0:1780298640.054819:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298640 v=(3 177) 00010000:00010000:3.0:1780298640.054830:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000008676129/0xaf35132e63030cb4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d02d9 expref: 16 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.054846:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000008676129/0xaf35132e63030cb4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d02d9 expref: 15 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.054874:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000008676129/0xaf35132e63030cb4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d02d9 expref: 15 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.054898:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f1880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777604711424:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298640.054944:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.055735:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f1880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777604711424:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298640.055753:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298640.055786:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f2680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609118336:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1053us (1205us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298640.055793:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14740 00000100:00100000:3.0:1780298640.057519:0:17578:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609118720 00000100:00100000:3.0:1780298640.057525:0:17578:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609118720 00000100:00100000:3.0:1780298640.057570:0:17578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61802 00000100:00100000:3.0:1780298640.057578:0:17578:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242862a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+27:29360:x1866777609118720:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.057593:0:17578:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.057609:0:17578:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830cc00/0xaf35132e63030cbb lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 9 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d02e0 expref: -99 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.057679:0:17578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c400/0xaf35132e63030cc2 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 10 type: IBT flags: 0x40000000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.057701:0:17578:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c400/0xaf35132e63030cc2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.057715:0:17578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c400/0xaf35132e63030cc2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1780298640.057738:0:17578:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 8932069 00000004:00200000:3.0:1780298640.057742:0:17578:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1780298640.057768:0:17578:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c400/0xaf35132e63030cc2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.057792:0:17578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830cc00/0xaf35132e63030cbb lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 10 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d02e0 expref: 29 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.057806:0:17578:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c400/0xaf35132e63030cc2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d02e0 expref: 28 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.057814:0:17578:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298640.057817:0:17578:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1780298640.057820:0:17578:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830c400, rc 0) 00000100:00100000:3.0:1780298640.057858:0:17578:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242862a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+28:29360:x1866777609118720:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 281us (369us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298640.057865:0:17578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61802 00000100:00100000:3.0:1780298640.059315:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609118976 00000100:00100000:3.0:1780298640.059322:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609118976 00000100:00100000:3.0:1780298640.067920:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.069302:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609120128 00000100:00100000:3.0:1780298640.069316:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609120128 00000100:00100000:3.0:1780298640.069355:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7740 00000100:00100000:3.0:1780298640.069371:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609120128:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.069381:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.069395:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ccb4ef2a/0xaf35132e63030ce5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d02f5 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.069435:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604711680:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.069518:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.070325:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604711680:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.070349:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ccb4ef2a/0xaf35132e63030ce5 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d02f5 expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.070379:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ccb4ef2a/0xaf35132e63030ce5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d02f5 expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.070399:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830cc00, rc 0) 00000100:00100000:3.0:1780298640.070437:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609120128:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1068us (1166us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.070446:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7740 00000100:00100000:3.0:1780298640.079524:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.081379:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.084320:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.086216:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.086234:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.086332:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7748 00000100:00100000:3.0:1780298640.086343:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609121792:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.086351:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.086367:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a777b004/0xaf35132e63030d1d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d032d expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.086422:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433dca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604712576:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.086482:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00002000:00010000:3.0:1780298640.087170:0:12101:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks: 8 -> 24 00000100:00100000:3.0:1780298640.087186:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433dca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604712576:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.087210:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a777b004/0xaf35132e63030d1d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d032d expref: 8 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.087228:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a777b004/0xaf35132e63030d1d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d032d expref: 8 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.087238:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830da00, rc 0) 00000100:00100000:3.0:1780298640.088278:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609122048 00000100:00100000:3.0:1780298640.088282:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609122048 00000100:00100000:3.0:1780298640.088329:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7750 00000100:00100000:3.0:1780298640.088339:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24380bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609122048:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.088350:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.088358:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ccb4ef2a/0xaf35132e63030d32 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d033b expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.088386:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433dca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604712960:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.088412:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.090221:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.090397:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.092055:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.092080:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.094101:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.095201:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.096292:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.099220:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.099261:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.101147:0:17578:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609123712 00000100:00100000:3.0:1780298640.101152:0:17578:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609123712 00010000:00010000:3.0:1780298640.102370:0:17578:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3821e6e00/0xaf35132e63030d78 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.102377:0:17578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3821e6e00/0xaf35132e63030d78 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1780298640.102435:0:17578:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294990920 00000004:00010000:3.0:1780298640.102461:0:17578:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 000000004e74e5a2 reply state 0000000012642a22 transno 4294990921 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3821e6e00/0xaf35132e63030d78 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.102510:0:17578:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3821e6e00/0xaf35132e63030d78 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 10 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.102516:0:17578:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298640.102518:0:17578:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1780298640.102520:0:17578:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1780298640.102569:0:17578:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243808700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+33:29360:x1866777609123712:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1361us (1463us total) trans 4294990921 rc 0/0 00000100:00100000:3.0:1780298640.102576:0:17578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61806 00000020:00080000:3.0:1780298640.102580:0:17578:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1780298640.105039:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.107526:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.108749:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.130543:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609125120 00000100:00100000:3.0:1780298640.130548:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609125120 00000100:00100000:3.0:1780298640.130588:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7762 00000100:00100000:3.0:1780298640.130599:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243061880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609125120:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.130609:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.130630:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007dbcc772/0xaf35132e63030da9 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d039d expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.130692:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8a6680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604715008:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.130789:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.131565:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8a6680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604715008:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.131591:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007dbcc772/0xaf35132e63030da9 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d039d expref: 8 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.131616:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007dbcc772/0xaf35132e63030da9 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d039d expref: 8 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.131632:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf3582cb400, rc 0) 00000100:00100000:3.0:1780298640.131662:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243061880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609125120:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1064us (1169us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.131677:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7762 00000100:00100000:3.0:1780298640.132746:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609125376 00000100:00100000:3.0:1780298640.132751:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609125376 00000100:00100000:3.0:1780298640.132785:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7765 00000100:00100000:3.0:1780298640.132797:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609125376:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.132804:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.132814:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007544ea9c/0xaf35132e63030dbe lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d03ab expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.132855:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8a6680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604715264:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.132915:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.133666:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8a6680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604715264:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.133697:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007544ea9c/0xaf35132e63030dbe lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d03ab expref: 8 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.133717:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007544ea9c/0xaf35132e63030dbe lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d03ab expref: 8 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.133724:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf3582cba00, rc 0) 00000100:00100000:3.0:1780298640.133751:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609125376:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 957us (1033us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.133756:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7765 00000100:00100000:3.0:1780298640.135012:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7767 00000100:00100000:3.0:1780298640.135024:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609125760:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.135034:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.135046:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000b8a2561/0xaf35132e63030dcc lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d03c0 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.135077:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8a6680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604715648:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.135107:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.135847:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8a6680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604715648:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.135869:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000b8a2561/0xaf35132e63030dcc lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d03c0 expref: 7 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.135906:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000b8a2561/0xaf35132e63030dcc lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d03c0 expref: 7 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.135915:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf3582cb000, rc 0) 00000100:00100000:3.0:1780298640.135939:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609125760:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 916us (1010us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.135947:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7767 00000100:00100000:3.0:1780298640.140417:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.141647:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f2680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777604715904:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298640.141669:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298640.141743:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8a6680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609126016:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1195us (1329us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298640.141752:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14746 00000100:00100000:3.0:1780298640.146834:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609126528 00000100:00100000:3.0:1780298640.146841:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609126528 00000100:00100000:3.0:1780298640.146889:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609126656 00000100:00100000:3.0:1780298640.146892:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609126656 00000100:00100000:3.0:1780298640.152917:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609126784 00000100:00100000:3.0:1780298640.152928:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609126784 00000100:00100000:3.0:1780298640.152971:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7773 00000100:00100000:3.0:1780298640.152979:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2541ea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609126784:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.152992:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.153006:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000b8a2561/0xaf35132e63030df6 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d03ea expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.153065:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8a5f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604716544:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.153149:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.153944:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8a5f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604716544:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.153963:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000b8a2561/0xaf35132e63030df6 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d03ea expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.153977:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000b8a2561/0xaf35132e63030df6 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d03ea expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.153986:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf3582cb000, rc 0) 00000100:00100000:3.0:1780298640.154017:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2541ea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609126784:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1040us (1167us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.154023:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7773 00000100:00100000:3.0:1780298640.158757:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609127424 00000100:00100000:3.0:1780298640.158762:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609127424 00000100:00100000:3.0:1780298640.158790:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7777 00000100:00100000:3.0:1780298640.158798:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254888380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609127424:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.158804:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.158812:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007544ea9c/0xaf35132e63030e12 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0406 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.158838:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8a5f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604717056:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.158862:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.161617:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8a5f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604717056:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.161661:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007544ea9c/0xaf35132e63030e12 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0406 expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.161686:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007544ea9c/0xaf35132e63030e12 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0406 expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.161696:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf3582cba00, rc 0) 00000100:00100000:3.0:1780298640.161779:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254888380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609127424:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 2982us (3064us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.161785:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7777 00000100:00100000:3.0:1780298640.163216:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609127552 00000100:00100000:3.0:1780298640.163225:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609127552 00002000:00100000:3.0:1780298640.163267:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x12:0x0] (1892352->1900543) req@ffff9cf25488bb80 x1866777609127552/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:346/0 lens 488/0 e 0 to 0 dl 1780298651 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298640.163281:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25488bb80 with x1866777609127552 ext(1892352->1900543) 00002000:00010000:3.0:1780298640.163287:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf25488bb80 00000100:00100000:3.0:1780298640.163326:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5580 00000100:00100000:3.0:1780298640.163338:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25488bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609127552:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298640.163530:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25488bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609127552:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 193us (353us total) trans 0 rc 8192/8192 00000100:00100000:3.0:1780298640.163538:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5580 00002000:00100000:3.0:1780298640.163547:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x12:0x0] (1892352->1900543) req@ffff9cf25488bb80 x1866777609127552/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:346/0 lens 488/8632 e 0 to 0 dl 1780298651 ref 1 fl Complete:/600/0 rc 8192/8192 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298640.163569:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25488bb80 with x1866777609127552 ext(1892352->1900543) 00002000:00010000:3.0:1780298640.163574:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf25488bb80 00000100:00100000:3.0:1780298640.164634:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609127680 00000100:00100000:3.0:1780298640.164644:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609127680 00000100:00100000:3.0:1780298640.164712:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7778 00000100:00100000:3.0:1780298640.164723:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25488a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609127680:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.164739:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.164758:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000b8a2561/0xaf35132e63030e19 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d040d expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.164819:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8a5f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604717184:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.164916:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.165750:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8a5f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604717184:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.165788:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000b8a2561/0xaf35132e63030e19 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d040d expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.165808:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000b8a2561/0xaf35132e63030e19 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d040d expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.165824:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf3582cb000, rc 0) 00000100:00100000:3.0:1780298640.165867:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25488a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609127680:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1144us (1275us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.165877:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7778 00000100:00100000:3.0:1780298640.171142:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609128192 00000100:00100000:3.0:1780298640.171153:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609128192 00000100:00100000:3.0:1780298640.171192:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609128320 00000100:00100000:3.0:1780298640.171195:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609128320 00000100:00100000:3.0:1780298640.173192:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.176190:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.176242:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.177118:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b41500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604718080:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.177166:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000522865f2/0xaf35132e63030e51 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0445 expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.177191:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000522865f2/0xaf35132e63030e51 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0445 expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.177219:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d09800, rc 0) 00000100:00100000:3.0:1780298640.177255:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8a7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609128832:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 947us (1012us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.177264:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7786 00000100:00100000:3.0:1780298640.178527:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609129088 00000100:00100000:3.0:1780298640.178532:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609129088 00000100:00100000:3.0:1780298640.178602:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7788 00000100:00100000:3.0:1780298640.178611:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e426c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609129088:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.178619:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.178628:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007544ea9c/0xaf35132e63030e5f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0453 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.178694:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8a5500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604718464:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.178738:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.179216:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604718336:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.179241:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e63030e58 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d044c expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.179259:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e63030e58 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d044c expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.179354:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37000, rc 0) 00000100:00100000:3.0:1780298640.179401:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b00a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609128960:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 963us (1057us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.179414:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7787 00000100:00100000:3.0:1780298640.180573:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609129216 00000100:00100000:3.0:1780298640.180579:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609129216 00000100:00100000:3.0:1780298640.180617:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7789 00000100:00100000:3.0:1780298640.180625:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b00e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609129216:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.180635:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.180644:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000b8a2561/0xaf35132e63030e66 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d045a expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.184376:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.184579:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609129728 00000100:00100000:3.0:1780298640.184587:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609129728 00000100:00100000:3.0:1780298640.184658:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7791 00000100:00100000:3.0:1780298640.184667:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b03480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609129728:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.184717:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.184725:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007dbcc772/0xaf35132e63030e74 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0468 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.184748:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8a4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604718976:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.184775:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.185464:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8a4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604718976:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.185479:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007dbcc772/0xaf35132e63030e74 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0468 expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.185493:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007dbcc772/0xaf35132e63030e74 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0468 expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.185503:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf3582cb400, rc 0) 00000100:00100000:3.0:1780298640.185532:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b03480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609129728:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 867us (984us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.185539:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7791 00000100:00100000:3.0:1780298640.186460:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609129856 00000100:00100000:3.0:1780298640.186466:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609129856 00000100:00100000:3.0:1780298640.186499:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7792 00000100:00100000:3.0:1780298640.186512:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b41180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609129856:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.186521:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.186530:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003bced65c/0xaf35132e63030e7b lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d046f expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.186611:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8a4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604719104:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.186660:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.187319:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8a4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604719104:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.187342:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003bced65c/0xaf35132e63030e7b lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d046f expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.187362:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003bced65c/0xaf35132e63030e7b lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d046f expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.187369:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf3582cbe00, rc 0) 00000100:00100000:3.0:1780298640.187396:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b41180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609129856:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 890us (1026us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.187401:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7792 00000100:00100000:3.0:1780298640.192014:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609130496 00000100:00100000:3.0:1780298640.192019:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609130496 00000100:00100000:3.0:1780298640.192042:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7795 00000100:00100000:3.0:1780298640.192049:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609130496:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.192056:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.192077:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007dbcc772/0xaf35132e63030e90 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0484 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.192114:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8a4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604719616:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.195485:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.198646:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609131136 00000100:00100000:3.0:1780298640.198651:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609131136 00002000:00100000:3.0:1780298640.198673:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (3284992->3289087) req@ffff9cf25599ca80 x1866777609131136/t0(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:346/0 lens 4584/0 e 0 to 0 dl 1780298651 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298640.198683:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25599ca80 with x1866777609131136 ext(3284992->3289087) 00002000:00010000:3.0:1780298640.198686:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf25599ca80 00000100:00100000:3.0:1780298640.198705:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5585 00000100:00100000:3.0:1780298640.198713:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25599ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609131136:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:3.0:1780298640.199818:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.203648:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.207446:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.212605:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.214554:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.216960:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.218495:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.221609:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.222870:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.223735:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.224832:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609133696 00000100:00100000:3.0:1780298640.224836:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609133696 00000100:00100000:3.0:1780298640.226269:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609133824 00000100:00100000:3.0:1780298640.226274:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609133824 00000100:00100000:3.0:1780298640.226314:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7813 00000100:00100000:3.0:1780298640.226325:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2397:x1866777609133824:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.226335:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.226345:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003bced65c/0xaf35132e63030f07 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d04fb expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.226398:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8ef480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604722048:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.226437:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.227052:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8ef480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604722048:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.227069:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003bced65c/0xaf35132e63030f07 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d04fb expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.227083:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003bced65c/0xaf35132e63030f07 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d04fb expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.227093:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf3582cbe00, rc 0) 00000100:00100000:3.0:1780298640.227184:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2397:x1866777609133824:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 859us (951us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.227195:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7813 00000100:00100000:3.0:1780298640.228930:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609134336 00000100:00100000:3.0:1780298640.228934:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609134336 00000100:00100000:3.0:1780298640.228956:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7815 00000100:00100000:3.0:1780298640.228962:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609134336:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.228969:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.228977:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004f507fef/0xaf35132e63030f15 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0510 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.229030:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8ef480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604722304:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.229077:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.229654:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8ef480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604722304:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.229668:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004f507fef/0xaf35132e63030f15 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0510 expref: 13 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.229678:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004f507fef/0xaf35132e63030f15 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0510 expref: 13 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.229685:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf3582ca800, rc 0) 00000100:00100000:3.0:1780298640.229710:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609134336:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 750us (822us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.229742:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7815 00000100:00100000:3.0:1780298640.230658:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609134592 00000100:00100000:3.0:1780298640.230663:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609134592 00000100:00100000:3.0:1780298640.230688:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7818 00000100:00100000:3.0:1780298640.230693:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b43100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609134592:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.230699:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.230705:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b8cdae1/0xaf35132e63030f2a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d051e expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.230725:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8ef480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604722688:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.230795:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.231284:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8ef480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604722688:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.231301:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001b8cdae1/0xaf35132e63030f2a lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d051e expref: 13 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.231364:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b8cdae1/0xaf35132e63030f2a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d051e expref: 13 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.231371:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf3582cae00, rc 0) 00000100:00100000:3.0:1780298640.231392:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609134592:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 700us (791us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.231396:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7818 00000100:00100000:3.0:1780298640.232082:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609134720 00000100:00100000:3.0:1780298640.232086:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609134720 00000100:00100000:3.0:1780298640.232107:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7819 00000100:00100000:3.0:1780298640.232111:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:29360:x1866777609134720:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.232117:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.232144:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000b8a2561/0xaf35132e63030f31 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0525 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.232156:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298640 v=(8 177) 00010000:00010000:3.0:1780298640.232159:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298640 v=(8 177) 00010000:00010000:3.0:1780298640.232165:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000b8a2561/0xaf35132e63030f31 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [872448->2559999] (req 872448->880639) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0525 expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.232179:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000b8a2561/0xaf35132e63030f31 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [872448->2559999] (req 872448->880639) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0525 expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.232188:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf3582cb000, rc 0) 00000100:00100000:3.0:1780298640.232223:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609134720:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 110us (160us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298640.232231:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7819 00000100:00100000:3.0:1780298640.234146:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609135104 00000100:00100000:3.0:1780298640.234151:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609135104 00000100:00100000:3.0:1780298640.234216:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7821 00000100:00100000:3.0:1780298640.234224:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242861c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609135104:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.234232:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.234239:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003bced65c/0xaf35132e63030f3f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0533 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.234264:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8ef480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604722944:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.234300:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.234803:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8ef480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604722944:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.234814:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003bced65c/0xaf35132e63030f3f lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0533 expref: 13 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.234823:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003bced65c/0xaf35132e63030f3f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0533 expref: 13 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.234830:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf3582cbe00, rc 0) 00000100:00100000:3.0:1780298640.234850:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242861c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609135104:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 628us (762us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.234855:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7821 00000100:00100000:3.0:1780298640.235608:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609135232 00000100:00100000:3.0:1780298640.235618:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609135232 00000100:00100000:3.0:1780298640.235647:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7822 00000100:00100000:3.0:1780298640.235657:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609135232:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.235663:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.235670:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007dbcc772/0xaf35132e63030f46 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d053a expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.235734:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8ef480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604723072:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.235766:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.236335:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8ef480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604723072:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.236347:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007dbcc772/0xaf35132e63030f46 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d053a expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.236358:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007dbcc772/0xaf35132e63030f46 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d053a expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.236365:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf3582cb400, rc 0) 00000100:00100000:3.0:1780298640.236420:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609135232:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 766us (854us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.236429:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7822 00000100:00100000:3.0:1780298640.237287:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609135616 00000100:00100000:3.0:1780298640.237292:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609135616 00000100:00100000:3.0:1780298640.237374:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7825 00000100:00100000:3.0:1780298640.237384:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343964e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609135616:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.237392:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.237406:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004f507fef/0xaf35132e63030f5b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d054f expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.237468:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8ef480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604723456:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.237502:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.238006:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8ef480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604723456:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.238072:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004f507fef/0xaf35132e63030f5b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d054f expref: 13 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.238082:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004f507fef/0xaf35132e63030f5b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d054f expref: 13 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.238088:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf3582ca800, rc 0) 00000100:00100000:3.0:1780298640.238109:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343964e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609135616:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 730us (847us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.238114:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7825 00000100:00100000:3.0:1780298640.238813:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609135872 00000100:00100000:3.0:1780298640.238817:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609135872 00000100:00100000:3.0:1780298640.240349:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.240449:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.241861:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.243467:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.245170:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.246285:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.247325:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7834 00000100:00100000:3.0:1780298640.247333:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25599d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609137024:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.247351:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.247361:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007dbcc772/0xaf35132e63030f9a lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d058e expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.247392:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8ed880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604724608:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.247439:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.249028:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.249047:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.250870:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.251741:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.251845:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.253560:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.253605:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.255204:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.255318:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.256738:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.256755:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.258589:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.258625:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.260584:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.262028:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.263266:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.266353:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.267960:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.268039:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.269960:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.269979:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.271968:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.273047:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.274265:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.275634:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.275662:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.275732:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7855 00000100:00100000:3.0:1780298640.275746:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8cad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609140608:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.275755:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.275767:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003bced65c/0xaf35132e63031026 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d061a expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.275804:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8c8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604727168:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.275845:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.278942:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.280513:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.282037:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.283347:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.283363:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.283388:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.283460:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14754 00000100:00100000:3.0:1780298640.283474:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a8a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609141504:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298640.283488:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298640.283498:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000cd74c683/0xaf35132e6303101f lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0613 expref: 9 pid: 8359 timeout: 929 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.283520:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000cd74c683/0xaf35132e6303101f lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0613 expref: 9 pid: 8359 timeout: 929 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.283563:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000cd74c683/0xaf35132e6303101f lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0613 expref: 9 pid: 8359 timeout: 929 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.283576:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298640.283603:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a8a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609141504:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 133us (236us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298640.283619:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14754 00000100:00100000:3.0:1780298640.284513:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.285605:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.287285:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609142144 00000100:00100000:3.0:1780298640.287326:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609142144 00000100:00100000:3.0:1780298640.287351:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7863 00000100:00100000:3.0:1780298640.287357:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25599dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609142144:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.287363:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.287370:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007544ea9c/0xaf35132e6303105e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0652 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.292287:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7865 00000100:00100000:3.0:1780298640.292299:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25599ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609142656:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.292318:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.292327:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004f507fef/0xaf35132e6303106c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0660 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.292359:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255511500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604728960:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.292413:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.295900:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.297552:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.300963:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.304756:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609144064 00000100:00100000:3.0:1780298640.304761:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609144064 00000100:00100000:3.0:1780298640.304817:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609144192 00000100:00100000:3.0:1780298640.304821:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609144192 00000100:00100000:3.0:1780298640.309067:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.312053:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.312076:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.313870:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.314864:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609145344 00000100:00100000:3.0:1780298640.314868:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609145344 00010000:00010000:3.0:1780298640.314884:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000003e31d8c5/0xaf35132e63030ef2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1003520->34557951] (req 1003520->1007615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d04e6 expref: 7 pid: 8359 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.314905:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14757 00000100:00100000:3.0:1780298640.314909:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25599f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609145344:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298640.314917:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298640.314924:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000003e31d8c5/0xaf35132e63030ef2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1003520->34557951] (req 1003520->1007615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d04e6 expref: 7 pid: 8359 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.314932:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000003e31d8c5/0xaf35132e63030ef2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1003520->34557951] (req 1003520->1007615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d04e6 expref: 7 pid: 8359 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.314944:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e31d8c5/0xaf35132e63030ef2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1003520->34557951] (req 1003520->1007615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d04e6 expref: 7 pid: 8359 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.314959:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298640 v=(5 130) 00010000:00010000:3.0:1780298640.314962:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298640 v=(5 130) 00010000:00010000:3.0:1780298640.314971:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e630310d5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [995328->34549759] (req 995328->1007615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d06c9 expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.314980:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e630310d5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [995328->34549759] (req 995328->1007615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d06c9 expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.315004:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e630310d5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [995328->34549759] (req 995328->1007615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d06c9 expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.315017:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f735180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:7400:1866777604730880:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298640.315046:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.321052:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.321255:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609146240 00000100:00100000:3.0:1780298640.321303:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609146240 00000100:00100000:3.0:1780298640.321346:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7887 00000100:00100000:3.0:1780298640.321354:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609146240:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.321439:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.321449:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004f507fef/0xaf35132e630310ff lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d06f3 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.321478:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f737100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604731648:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.321519:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.322231:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f737100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604731648:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.322243:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004f507fef/0xaf35132e630310ff lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d06f3 expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.322253:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004f507fef/0xaf35132e630310ff lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d06f3 expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.322260:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf3582ca800, rc 0) 00000100:00100000:3.0:1780298640.322281:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609146240:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 929us (1073us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.322287:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7887 00000100:00100000:3.0:1780298640.323034:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609146368 00000100:00100000:3.0:1780298640.323038:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609146368 00000100:00100000:3.0:1780298640.323060:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7888 00000100:00100000:3.0:1780298640.323065:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:29360:x1866777609146368:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.323072:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.323079:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001b8cdae1/0xaf35132e63031106 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d06fa expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.323093:0:19992:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f68a0cb8/0xaf35132e63031065 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3948544->37502975] (req 3948544->3952639) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0659 expref: 8 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.323100:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298640 v=(11 177) 00010000:00010000:3.0:1780298640.323103:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298640 v=(12 177) 00010000:00010000:3.0:1780298640.323106:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001b8cdae1/0xaf35132e63031106 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3989504->3993599] (req 3989504->3993599) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d06fa expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.323150:0:19992:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f68a0cb8/0xaf35132e63031065 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3948544->37502975] (req 3948544->3952639) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0659 expref: 8 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.323162:0:19992:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f68a0cb8/0xaf35132e63031065 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3948544->37502975] (req 3948544->3952639) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0659 expref: 8 pid: 8359 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.323170:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f737100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604731776:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298640.323202:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.323796:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f737100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604731776:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298640.323806:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001b8cdae1/0xaf35132e63031106 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3989504->3993599] (req 3989504->3993599) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d06fa expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.323814:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf3582cae00, rc 0) 00000100:00100000:3.0:1780298640.323833:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609146368:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 769us (821us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298640.323838:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7888 00000100:00100000:3.0:1780298640.332298:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609147264 00000100:00100000:3.0:1780298640.332316:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609147264 00000100:00100000:3.0:1780298640.334338:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.334687:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609147648 00000100:00100000:3.0:1780298640.334691:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609147648 00000100:00100000:3.0:1780298640.334717:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7894 00000100:00100000:3.0:1780298640.334725:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b01f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609147648:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.334734:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.334749:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004f507fef/0xaf35132e63031129 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d071d expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.334774:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f734000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604732416:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.334806:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.336692:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609147776 00000100:00100000:3.0:1780298640.336697:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609147776 00000100:00100000:3.0:1780298640.336738:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7895 00000100:00100000:3.0:1780298640.336751:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b00a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609147776:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.336766:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.336778:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007544ea9c/0xaf35132e63031130 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0724 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.336827:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f737b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604732544:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.336879:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.337618:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f737b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604732544:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.337639:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007544ea9c/0xaf35132e63031130 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0724 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.337656:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007544ea9c/0xaf35132e63031130 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0724 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.337680:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf3582cba00, rc 0) 00000100:00100000:3.0:1780298640.337717:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b00a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609147776:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 968us (1066us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.337740:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7895 00000100:00100000:3.0:1780298640.341733:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.344463:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609148672 00000100:00100000:3.0:1780298640.344468:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609148672 00000100:00100000:3.0:1780298640.344501:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7899 00000100:00100000:3.0:1780298640.344510:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609148672:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.344521:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.344533:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007544ea9c/0xaf35132e63031153 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0740 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.344556:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000577d6b77/0xaf35132e63031096 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3096576->3948543] (req 3096576->3100671) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d068a expref: 6 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.344566:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000008676129/0xaf35132e6303110d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3985408->3989503] (req 3985408->3989503) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0701 expref: 11 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.344574:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000001b8cdae1/0xaf35132e63031106 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3989504->37543935] (req 3989504->3993599) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d06fa expref: 11 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.344583:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 3 to [0x2c0000401:0x11:0x0].0x0 time=1780298640 v=(12 177) 00010000:00010000:3.0:1780298640.344588:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298640 v=(15 177) 00010000:00010000:3.0:1780298640.344593:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007544ea9c/0xaf35132e63031153 lrc: 6/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0740 expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.344616:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000001b8cdae1/0xaf35132e63031106 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3989504->37543935] (req 3989504->3993599) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d06fa expref: 11 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.344636:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000001b8cdae1/0xaf35132e63031106 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3989504->37543935] (req 3989504->3993599) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d06fa expref: 11 pid: 19992 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.344647:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f734000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604733184:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298640.344705:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000008676129/0xaf35132e6303110d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3985408->3989503] (req 3985408->3989503) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0701 expref: 11 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.344717:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000008676129/0xaf35132e6303110d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3985408->3989503] (req 3985408->3989503) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0701 expref: 11 pid: 8356 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.344727:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f735880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604733312:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298640.344764:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000577d6b77/0xaf35132e63031096 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3096576->3948543] (req 3096576->3100671) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d068a expref: 6 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.344842:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000577d6b77/0xaf35132e63031096 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3096576->3948543] (req 3096576->3100671) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d068a expref: 6 pid: 8358 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.344855:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604733440:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298640.344923:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.345504:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f734000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604733184:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298640.345519:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f735880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604733312:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298640.345620:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604733440:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298640.345647:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007544ea9c/0xaf35132e63031153 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0740 expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.345723:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf3582cba00, rc 0) 00000100:00100000:3.0:1780298640.345761:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:29360:x1866777609148672:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1251us (1330us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298640.345824:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7899 00000100:00100000:3.0:1780298640.345940:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609149056 00000100:00100000:3.0:1780298640.345946:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609149056 00010000:00010000:3.0:1780298640.345969:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000577d6b77/0xaf35132e63031096 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3096576->3948543] (req 3096576->3100671) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d068a expref: 7 pid: 8358 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.346007:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14762 00000100:00100000:3.0:1780298640.346022:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343964e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609149056:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298640.346034:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298640.346048:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000577d6b77/0xaf35132e63031096 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3096576->3948543] (req 3096576->3100671) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d068a expref: 7 pid: 8358 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.346104:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000577d6b77/0xaf35132e63031096 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3096576->3948543] (req 3096576->3100671) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d068a expref: 7 pid: 8358 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.346119:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000577d6b77/0xaf35132e63031096 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3096576->3948543] (req 3096576->3100671) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d068a expref: 7 pid: 8358 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.346198:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298640 v=(15 177) 00010000:00010000:3.0:1780298640.346201:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298640 v=(15 177) 00010000:00010000:3.0:1780298640.346214:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000007544ea9c/0xaf35132e63031153 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0740 expref: 8 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.346224:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007544ea9c/0xaf35132e63031153 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0740 expref: 8 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.346248:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000007544ea9c/0xaf35132e63031153 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0740 expref: 8 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.346259:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777604733568:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298640.346363:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.346992:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777604733568:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298640.347004:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298640.347035:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343964e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609149056:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1019us (1139us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298640.347044:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14762 00000100:00100000:3.0:1780298640.347177:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609149184 00000100:00100000:3.0:1780298640.347182:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609149184 00000100:00100000:3.0:1780298640.347213:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7900 00000100:00100000:3.0:1780298640.347221:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:29360:x1866777609149184:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.347230:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.347242:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000b8a2561/0xaf35132e6303115a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0747 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.347262:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e630310d5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [995328->34549759] (req 995328->1007615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d06c9 expref: 12 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.347281:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298640 v=(5 130) 00010000:00010000:3.0:1780298640.347285:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298640 v=(6 130) 00010000:00010000:3.0:1780298640.347290:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000b8a2561/0xaf35132e6303115a lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0747 expref: 12 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.347327:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e630310d5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [995328->34549759] (req 995328->1007615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d06c9 expref: 12 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.347339:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e630310d5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [995328->34549759] (req 995328->1007615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d06c9 expref: 12 pid: 19992 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.347349:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604733696:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298640.347386:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.347883:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604733696:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298640.347896:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000b8a2561/0xaf35132e6303115a lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0747 expref: 12 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.347908:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf3582cb000, rc 0) 00000100:00100000:3.0:1780298640.347935:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:29360:x1866777609149184:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 716us (849us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298640.347941:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7900 00000100:00100000:3.0:1780298640.349249:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.349276:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.349392:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5610 00000100:00100000:3.0:1780298640.349403:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f735880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609149568:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:3.0:1780298640.349532:0:19943:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 2396203 -> 4194304 00002000:00010000:3.0:1780298640.349536:0:19943:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298640 -> 1780298640 00002000:00010000:3.0:1780298640.349540:0:19943:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298640 -> 1780298640 00002000:00010000:3.0:1780298640.349546:0:19943:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298640 -> 1780298640 00002000:00010000:3.0:1780298640.349553:0:19943:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 56 -> 56 00000100:00100000:3.0:1780298640.349589:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f735880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609149568:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 188us (311us total) trans 4294969091 rc 0/0 00000100:00100000:3.0:1780298640.349604:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5610 00002000:00010000:3.0:1780298640.349608:0:19943:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4194304->18446744073709551615) 00002000:00100000:3.0:1780298640.349613:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f735880 with x1866777609149568 ext(4194304->18446744073709551615) 00002000:00010000:3.0:1780298640.349621:0:19943:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f735880. 00000100:00100000:3.0:1780298640.350360:0:17578:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609149696 00000100:00100000:3.0:1780298640.350365:0:17578:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609149696 00000100:00100000:3.0:1780298640.350397:0:17578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61810 00000100:00100000:3.0:1780298640.350406:0:17578:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343967100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+36:29360:x1866777609149696:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.350418:0:17578:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.350432:0:17578:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3582cbe00/0xaf35132e63031161 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 12 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d074e expref: -99 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.350498:0:17578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3582cb400/0xaf35132e63031168 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.350513:0:17578:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3582cb400/0xaf35132e63031168 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.350534:0:17578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3582cb400/0xaf35132e63031168 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1780298640.350558:0:17578:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 9529568 00000004:00200000:3.0:1780298640.350562:0:17578:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1780298640.350576:0:17578:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3582cb400/0xaf35132e63031168 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.350597:0:17578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3582cbe00/0xaf35132e63031161 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 13 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d074e expref: 38 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.350610:0:17578:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3582cb400/0xaf35132e63031168 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d074e expref: 37 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.350617:0:17578:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298640.350620:0:17578:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1780298640.350623:0:17578:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf3582cb400, rc 0) 00000100:00100000:3.0:1780298640.350652:0:17578:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343967100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+37:29360:x1866777609149696:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 247us (318us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298640.350659:0:17578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61810 00000100:00100000:3.0:1780298640.359538:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343967b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604733952:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298640.359560:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000021dd6ca7/0xaf35132e6303116f lrc: 3/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [393216->397311] (req 393216->397311) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0755 expref: 7 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.359571:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36200, rc 0) 00000100:00100000:3.0:1780298640.359596:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609150336:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 905us (983us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298640.359614:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7902 00000100:00100000:3.0:1780298640.362266:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609150720 00000100:00100000:3.0:1780298640.362271:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609150720 00000100:00100000:3.0:1780298640.362293:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7903 00000100:00100000:3.0:1780298640.362298:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609150720:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.362314:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.362323:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003bced65c/0xaf35132e63031176 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d075c expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.362335:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298640 v=(7 130) 00010000:00010000:3.0:1780298640.362338:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298640 v=(7 130) 00010000:00010000:3.0:1780298640.362344:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003bced65c/0xaf35132e63031176 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d075c expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.362358:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003bced65c/0xaf35132e63031176 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d075c expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.362368:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf3582cbe00, rc 0) 00000100:00100000:3.0:1780298640.362399:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609150720:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 100us (184us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298640.362406:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7903 00000100:00100000:3.0:1780298640.365824:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609151104 00000100:00100000:3.0:1780298640.365829:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609151104 00000100:00100000:3.0:1780298640.365861:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7904 00000100:00100000:3.0:1780298640.365871:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548bb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609151104:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.365887:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.365898:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b8cdae1/0xaf35132e6303117d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0763 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.365918:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000021dd6ca7/0xaf35132e6303116f lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 393216->397311) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0755 expref: 6 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.365928:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003bced65c/0xaf35132e63031176 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d075c expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.365935:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298640 v=(7 130) 00010000:00010000:3.0:1780298640.365938:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298640 v=(7 130) 00010000:00010000:3.0:1780298640.365941:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001b8cdae1/0xaf35132e6303117d lrc: 5/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3723264->3731455] (req 3723264->3731455) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0763 expref: 11 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.365961:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003bced65c/0xaf35132e63031176 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d075c expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.365988:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003bced65c/0xaf35132e63031176 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d075c expref: 11 pid: 9128 timeout: 929 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.366000:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f734000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604734208:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298640.366055:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000021dd6ca7/0xaf35132e6303116f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 393216->397311) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0755 expref: 6 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.366076:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000021dd6ca7/0xaf35132e6303116f lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 393216->397311) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0755 expref: 6 pid: 8358 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.366086:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f734700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604734336:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298640.366106:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.366703:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f734000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604734208:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298640.366851:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f734700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604734336:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298640.366912:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001b8cdae1/0xaf35132e6303117d lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3723264->3731455] (req 3723264->3731455) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0763 expref: 11 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.366924:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf3582cae00, rc 0) 00000100:00100000:3.0:1780298640.366959:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548bb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:29360:x1866777609151104:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1090us (1172us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298640.366967:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7904 00000100:00100000:3.0:1780298640.368055:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.370241:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.370265:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.372296:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609152000 00000100:00100000:3.0:1780298640.372301:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609152000 00000100:00100000:3.0:1780298640.372339:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7907 00000100:00100000:3.0:1780298640.372348:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343964700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609152000:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.372355:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.372363:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000b8a2561/0xaf35132e630311a0 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d077f expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.372401:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f737800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604734848:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.372444:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.374135:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609152256 00000100:00100000:3.0:1780298640.374139:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609152256 00000100:00100000:3.0:1780298640.375557:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.376198:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.378899:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.380508:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.381404:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.382371:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.383349:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.383379:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.384251:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548b9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604735744:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.384274:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000008dd0310/0xaf35132e630311d8 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d07b7 expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.384300:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000008dd0310/0xaf35132e630311d8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d07b7 expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.384330:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243376400, rc 0) 00000100:00100000:3.0:1780298640.384367:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609153408:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 912us (987us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.384381:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7915 00000100:00100000:3.0:1780298640.385239:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.385277:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.386153:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343967b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604736128:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.386185:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000097580845/0xaf35132e630311e6 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d07c5 expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.386203:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000097580845/0xaf35132e630311e6 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d07c5 expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.386215:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37600, rc 0) 00000100:00100000:3.0:1780298640.386243:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609153664:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 884us (966us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.386251:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7917 00000100:00100000:3.0:1780298640.391301:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.392507:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609154304 00000100:00100000:3.0:1780298640.392511:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609154304 00000100:00100000:3.0:1780298640.392541:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7920 00000100:00100000:3.0:1780298640.392550:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609154304:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.392558:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.392567:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004f507fef/0xaf35132e630311fb lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d07da expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.392597:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604736512:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.392636:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.393241:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604736512:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.393253:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004f507fef/0xaf35132e630311fb lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d07da expref: 14 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.393280:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004f507fef/0xaf35132e630311fb lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d07da expref: 14 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.393287:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf3582ca800, rc 0) 00000100:00100000:3.0:1780298640.393319:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609154304:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 771us (839us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.393325:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7920 00000100:00100000:3.0:1780298640.394029:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609154432 00000100:00100000:3.0:1780298640.394032:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609154432 00000100:00100000:3.0:1780298640.394052:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7921 00000100:00100000:3.0:1780298640.394057:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255491f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609154432:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.394063:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.394069:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000b8a2561/0xaf35132e63031202 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d07e1 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.394088:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604736640:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.394115:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.394706:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604736640:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.394719:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000b8a2561/0xaf35132e63031202 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d07e1 expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.394731:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000b8a2561/0xaf35132e63031202 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d07e1 expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.394752:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf3582cb000, rc 0) 00000100:00100000:3.0:1780298640.394778:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255491f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609154432:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 720us (775us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.394784:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7921 00000100:00100000:3.0:1780298640.398781:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609155072 00000100:00100000:3.0:1780298640.398787:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609155072 00000100:00100000:3.0:1780298640.398830:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7925 00000100:00100000:3.0:1780298640.398844:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255493b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609155072:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.398858:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.398868:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003bced65c/0xaf35132e6303121e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d07fd expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.398907:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604737152:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.398941:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.399612:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604737152:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.399636:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003bced65c/0xaf35132e6303121e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d07fd expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.399663:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003bced65c/0xaf35132e6303121e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d07fd expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.399682:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf3582cbe00, rc 0) 00000100:00100000:3.0:1780298640.399711:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255493b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609155072:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 871us (962us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.399727:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7925 00000100:00100000:3.0:1780298640.400670:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609155328 00000100:00100000:3.0:1780298640.400674:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609155328 00000100:00100000:3.0:1780298640.400700:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609155200 00000100:00100000:3.0:1780298640.400703:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609155200 00000100:00100000:3.0:1780298640.400729:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7926 00000100:00100000:3.0:1780298640.400736:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609155328:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.400746:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.400757:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e63031225 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d080b expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.400791:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604737280:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.400828:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.401647:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604737280:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.401666:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e63031225 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d080b expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.401683:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e63031225 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d080b expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.401696:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2000, rc 0) 00000100:00100000:3.0:1780298640.401774:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609155328:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1035us (1133us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.401789:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7926 00000100:00100000:3.0:1780298640.402852:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609155456 00000100:00100000:3.0:1780298640.402863:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609155456 00000100:00100000:3.0:1780298640.402896:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7928 00000100:00100000:3.0:1780298640.402905:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b43100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609155456:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.402915:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.402941:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e63031233 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0812 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.402975:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604737536:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.403021:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.403706:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604737536:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.403723:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e63031233 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0812 expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.403738:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e63031233 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0812 expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.403749:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2e00, rc 0) 00000100:00100000:3.0:1780298640.403779:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609155456:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 875us (953us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.403787:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7928 00000100:00100000:3.0:1780298640.404704:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609155584 00000100:00100000:3.0:1780298640.404709:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609155584 00000100:00100000:3.0:1780298640.404741:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7929 00000100:00100000:3.0:1780298640.404747:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e425450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609155584:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.404757:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.404773:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e6303123a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0819 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.404800:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604737664:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.404838:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.405472:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604737664:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.405488:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e6303123a lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0819 expref: 14 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.405502:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e6303123a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0819 expref: 14 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.405511:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2400, rc 0) 00000100:00100000:3.0:1780298640.405538:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e425450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609155584:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 791us (869us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.405545:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7929 00000100:00100000:3.0:1780298640.406460:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.408413:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609155968 00000100:00100000:3.0:1780298640.408420:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609155968 00000100:00100000:3.0:1780298640.408458:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7932 00000100:00100000:3.0:1780298640.408471:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343965500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:29360:x1866777609155968:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.408480:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.408490:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e6303124f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d082e expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.408511:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000007544ea9c/0xaf35132e63031153 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0740 expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.408527:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298640 v=(15 177) 00010000:00010000:3.0:1780298640.408531:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298640 v=(16 177) 00010000:00010000:3.0:1780298640.408537:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e6303124f lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [5156864->5160959] (req 5156864->5160959) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d082e expref: 7 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.408560:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000007544ea9c/0xaf35132e63031153 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0740 expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.408574:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000007544ea9c/0xaf35132e63031153 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0740 expref: 10 pid: 9128 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.408586:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fe680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604738048:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298640.408635:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.410773:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.413510:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609156480 00000100:00100000:3.0:1780298640.413516:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609156480 00000100:00100000:3.0:1780298640.413599:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7933 00000100:00100000:3.0:1780298640.413610:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255493100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609156480:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.413620:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.413638:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e63031256 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0835 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.413671:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298640 v=(16 177) 00010000:00010000:3.0:1780298640.413676:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298640 v=(16 177) 00010000:00010000:3.0:1780298640.413691:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e63031256 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0835 expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.413711:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e63031256 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0835 expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.413722:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2e00, rc 0) 00000100:00100000:3.0:1780298640.413756:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255493100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609156480:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 150us (289us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298640.413764:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7933 00000100:00100000:3.0:1780298640.417942:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.422275:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.423745:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.428298:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.430302:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.431769:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.432841:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.434096:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.434209:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.435281:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.436489:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.437938:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.439113:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.439154:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.440348:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.442071:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.443456:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.444711:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.447820:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.449796:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.451969:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609160064 00000100:00100000:3.0:1780298640.451979:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609160064 00002000:00100000:3.0:1780298640.452022:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (1568768->1572863) req@ffff9cf343964380 x1866777609160064/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:346/0 lens 488/0 e 0 to 0 dl 1780298651 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298640.452030:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf343964380 with x1866777609160064 ext(1568768->1572863) 00002000:00010000:3.0:1780298640.452035:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf343964380 00000100:00100000:3.0:1780298640.452060:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5631 00000100:00100000:3.0:1780298640.452067:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343964380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609160064:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298640.453611:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.455254:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.459696:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.461702:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.463705:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.466936:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.468214:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.470195:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.473866:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.475193:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.476431:0:17578:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609161856 00000100:00100000:3.0:1780298640.476436:0:17578:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609161856 00000100:00100000:3.0:1780298640.476472:0:17578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61814 00000100:00100000:3.0:1780298640.476480:0:17578:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255491c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+36:29360:x1866777609161856:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.476493:0:17578:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.476519:0:17578:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2800/0xaf35132e630312e9 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 14 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d08ba expref: -99 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.476596:0:17578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3000/0xaf35132e630312f0 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 15 type: IBT flags: 0x40000000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.476611:0:17578:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3000/0xaf35132e630312f0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 15 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.476622:0:17578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3000/0xaf35132e630312f0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 15 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1780298640.476644:0:17578:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 8516605 00000004:00200000:3.0:1780298640.476648:0:17578:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1780298640.476662:0:17578:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3000/0xaf35132e630312f0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 15 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.476683:0:17578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2800/0xaf35132e630312e9 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 15 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d08ba expref: 38 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.476700:0:17578:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3000/0xaf35132e630312f0 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 14 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d08ba expref: 37 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.476707:0:17578:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298640.476710:0:17578:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1780298640.476713:0:17578:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3000, rc 0) 00000100:00100000:3.0:1780298640.476744:0:17578:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255491c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+37:29360:x1866777609161856:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 265us (363us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298640.476750:0:17578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61814 00000100:00100000:3.0:1780298640.477842:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.477925:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.480288:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.483000:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609162496 00000100:00100000:3.0:1780298640.483010:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609162496 00000100:00100000:3.0:1780298640.483061:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7956 00000100:00100000:3.0:1780298640.483074:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255493b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:29360:x1866777609162496:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.483088:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.483103:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e63031313 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d08dd expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.483161:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000006dfdf24b/0xaf35132e630312cd lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2506752->3997695] (req 2506752->2510847) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d089e expref: 7 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.483173:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298640 v=(17 177) 00010000:00010000:3.0:1780298640.483178:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298640 v=(18 177) 00010000:00010000:3.0:1780298640.483185:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e63031313 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3850240->3858431] (req 3850240->3858431) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d08dd expref: 14 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.483219:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000006dfdf24b/0xaf35132e630312cd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2506752->3997695] (req 2506752->2510847) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d089e expref: 7 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.483240:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000006dfdf24b/0xaf35132e630312cd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2506752->3997695] (req 2506752->2510847) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d089e expref: 7 pid: 8359 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.483257:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f737b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604741632:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298640.483360:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393c00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.484024:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f737b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604741632:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298640.484041:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e63031313 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3850240->3858431] (req 3850240->3858431) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d08dd expref: 14 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.484052:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2800, rc 0) 00000100:00100000:3.0:1780298640.484087:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255493b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:29360:x1866777609162496:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1017us (1136us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298640.484094:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7956 00000100:00100000:3.0:1780298640.489158:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.489248:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.496392:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.501377:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609164032 00000100:00100000:3.0:1780298640.501389:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609164032 00000100:00100000:3.0:1780298640.501442:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7963 00000100:00100000:3.0:1780298640.501453:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2397:x1866777609164032:12345-192.168.203.25@tcp:16:fsx.0 00000100:00100000:3.0:1780298640.502444:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609164032:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 992us (1126us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298640.502452:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7963 00002000:00010000:3.0:1780298640.504365:0:8358:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks: 720 -> 768 00000100:00100000:3.0:1780298640.504380:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343964000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604742656:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.504416:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000085c2bc23/0xaf35132e63031344 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d090e expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.504438:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000085c2bc23/0xaf35132e63031344 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d090e expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.504466:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37a00, rc 0) 00000100:00100000:3.0:1780298640.504504:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609164160:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 978us (1073us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.504511:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7964 00000100:00100000:3.0:1780298640.508505:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609164544 00000100:00100000:3.0:1780298640.508514:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609164544 00010000:00010000:3.0:1780298640.508543:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000042649f54/0xaf35132e6303128e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0866 expref: 7 pid: 8359 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.508575:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14776 00000100:00100000:3.0:1780298640.508584:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b42680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609164544:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298640.508597:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298640.508612:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000042649f54/0xaf35132e6303128e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0866 expref: 7 pid: 8359 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.508625:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000042649f54/0xaf35132e6303128e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0866 expref: 7 pid: 8359 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.508643:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042649f54/0xaf35132e6303128e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0866 expref: 7 pid: 8359 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.508656:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298640 v=(11 130) 00010000:00010000:3.0:1780298640.508660:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298640 v=(11 130) 00010000:00010000:3.0:1780298640.508668:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000008676129/0xaf35132e63031352 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3821568->37375999] (req 3821568->3825663) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d091c expref: 9 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.508686:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000008676129/0xaf35132e63031352 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3821568->37375999] (req 3821568->3825663) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d091c expref: 9 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.508714:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000008676129/0xaf35132e63031352 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3821568->37375999] (req 3821568->3825663) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d091c expref: 9 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.508727:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f737100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:20253:1866777604743040:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298640.508786:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393c00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.509620:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f737100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:20253:1866777604743040:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298640.509634:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298640.509700:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b42680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609164544:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1117us (1237us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298640.509711:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14776 00000100:00100000:3.0:1780298640.511196:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609164800 00000100:00100000:3.0:1780298640.511203:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609164800 00000100:00100000:3.0:1780298640.511253:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609164928 00000100:00100000:3.0:1780298640.511258:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609164928 00000100:00100000:3.0:1780298640.511359:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7968 00000100:00100000:3.0:1780298640.511374:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255492300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609164928:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.511393:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.511407:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e63031360 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d092a expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.511451:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f737100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604743296:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.511510:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393c00 going to sleep for 11 seconds 00002000:00010000:3.0:1780298640.512234:0:19992:0:(ofd_lvb.c:265:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime: 1780298640 -> 1780298641 00002000:00010000:3.0:1780298640.512245:0:19992:0:(ofd_lvb.c:271:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime: 1780298640 -> 1780298641 00002000:00010000:3.0:1780298640.512256:0:19992:0:(ofd_lvb.c:277:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime: 1780298640 -> 1780298641 00000100:00100000:3.0:1780298640.512279:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f737100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604743296:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.512315:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e63031360 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d092a expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.512337:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e63031360 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d092a expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.512351:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2400, rc 0) 00000100:00100000:3.0:1780298640.512395:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255492300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609164928:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1025us (1167us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.512404:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7968 00000100:00100000:3.0:1780298640.513385:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609165184 00000100:00100000:3.0:1780298640.513392:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609165184 00000100:00100000:3.0:1780298640.514374:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255492d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604743424:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.514406:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000577d6b77/0xaf35132e63031367 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0931 expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.514423:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000577d6b77/0xaf35132e63031367 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0931 expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.514476:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243377e00, rc 0) 00000100:00100000:3.0:1780298640.514529:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b41500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609165056:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1102us (1223us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.514537:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7969 00000100:00100000:3.0:1780298640.519777:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609166080 00000100:00100000:3.0:1780298640.519785:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609166080 00010000:00010000:3.0:1780298640.519800:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000fba32a6f/0xaf35132e630312e2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [1478656->2506751] (req 1478656->1490943) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d08b3 expref: 14 pid: 20252 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.519854:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14778 00000100:00100000:3.0:1780298640.519864:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b43800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2398:x1866777609166080:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298640.519873:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298640.519893:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000fba32a6f/0xaf35132e630312e2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [1478656->2506751] (req 1478656->1490943) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d08b3 expref: 14 pid: 20252 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.519907:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000fba32a6f/0xaf35132e630312e2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 8 type: EXT [1478656->2506751] (req 1478656->1490943) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d08b3 expref: 14 pid: 20252 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.519943:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fba32a6f/0xaf35132e630312e2 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 8 type: EXT [1478656->2506751] (req 1478656->1490943) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d08b3 expref: 14 pid: 20252 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.519956:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298640.519995:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609166080:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 132us (244us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298640.520020:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14778 00000100:00100000:3.0:1780298640.520809:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f2680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777604744192:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298640.520875:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298640.520922:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b42680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609165952:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1005us (1231us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298640.520932:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14780 00000100:00100000:3.0:1780298640.523079:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609166464 00000100:00100000:3.0:1780298640.523085:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609166464 00002000:00010000:3.0:1780298640.523104:0:16972:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (3282448->18446744073709551615) 00002000:00100000:3.0:1780298640.523109:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf343964a80 with x1866777609166464 ext(3282448->18446744073709551615) 00002000:00010000:3.0:1780298640.523117:0:16972:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf343964a80. 00000100:00100000:3.0:1780298640.523183:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5643 00000100:00100000:3.0:1780298640.523194:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343964a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609166464:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:3.0:1780298640.523675:0:16972:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4322301 -> 3282448 00002000:00010000:3.0:1780298640.523680:0:16972:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298641 -> 1780298641 00002000:00010000:3.0:1780298640.523683:0:16972:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298641 -> 1780298641 00002000:00010000:3.0:1780298640.523686:0:16972:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298641 -> 1780298641 00002000:00010000:3.0:1780298640.523689:0:16972:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 768 -> 736 00000100:00100000:3.0:1780298640.523725:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343964a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609166464:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 534us (678us total) trans 4294969504 rc 0/0 00000100:00100000:3.0:1780298640.523733:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5643 00002000:00010000:3.0:1780298640.523737:0:16972:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (3282448->18446744073709551615) 00002000:00100000:3.0:1780298640.523741:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf343964a80 with x1866777609166464 ext(3282448->18446744073709551615) 00002000:00010000:3.0:1780298640.523746:0:16972:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf343964a80. 00000100:00100000:3.0:1780298640.524545:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.525763:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.527559:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609167104 00000100:00100000:3.0:1780298640.527567:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609167104 00000100:00100000:3.0:1780298640.527618:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7976 00000100:00100000:3.0:1780298640.527638:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e425c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609167104:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.527651:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.527660:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e630313ad lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0962 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.527695:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f734000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604744960:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.527728:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393c00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.529291:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.530343:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.531442:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.534610:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.536329:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.536360:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.538392:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.543551:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.545253:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.548399:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.551336:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609169408 00000100:00100000:3.0:1780298640.551347:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609169408 00000100:00100000:3.0:1780298640.551404:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7987 00000100:00100000:3.0:1780298640.551415:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:29360:x1866777609169408:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.551432:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.551448:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63031401 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d09b6 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.551476:0:19992:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000522865f2/0xaf35132e630313b4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1130496->34684927] (req 1130496->1134591) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0970 expref: 6 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.551503:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298640 v=(22 177) 00010000:00010000:3.0:1780298640.551507:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298640 v=(23 177) 00010000:00010000:3.0:1780298640.551513:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63031401 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [4268032->4276223] (req 4268032->4276223) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d09b6 expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.551554:0:19992:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000522865f2/0xaf35132e630313b4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1130496->34684927] (req 1130496->1134591) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0970 expref: 6 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.551569:0:19992:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000522865f2/0xaf35132e630313b4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1130496->34684927] (req 1130496->1134591) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0970 expref: 6 pid: 12101 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.551583:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548b9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604746368:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298640.551657:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.553977:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.557701:0:17578:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609170048 00000100:00100000:3.0:1780298640.557708:0:17578:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609170048 00000100:00100000:3.0:1780298640.557744:0:17578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61818 00000100:00100000:3.0:1780298640.557750:0:17578:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+31:29360:x1866777609170048:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:3.0:1780298640.557796:0:17578:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298640.557819:0:17578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09e00/0xaf35132e63031391 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 16 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d094d expref: 31 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.557837:0:17578:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298640.557842:0:17578:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1780298640.557980:0:17578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2c00/0xaf35132e63031416 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 15 type: IBT flags: 0x40000000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.557991:0:17578:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09000/0xaf35132e630313ec lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 15 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d09a1 expref: 39 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.557999:0:17578:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2c00/0xaf35132e63031416 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 15 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.558020:0:17578:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09000/0xaf35132e630313ec lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 15 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d09a1 expref: 39 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.558029:0:17578:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09000/0xaf35132e630313ec lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 15 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d09a1 expref: 39 pid: 9976 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.558037:0:17578:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548ba680 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000_UUID:17578:1866777604746880:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298640.558088:0:17578:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.558749:0:17578:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548ba680 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000_UUID:17578:1866777604746880:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298640.558765:0:17578:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2c00/0xaf35132e63031416 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 15 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.559100:0:17578:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2c00/0xaf35132e63031416 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 14 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.559107:0:17578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2c00/0xaf35132e63031416 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 14 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1780298640.559201:0:17578:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294990928 00000004:00010000:3.0:1780298640.559257:0:17578:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 000000007169db92 reply state 00000000ea4b7bf0 transno 4294990929 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2c00/0xaf35132e63031416 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 14 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.559279:0:17578:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2c00/0xaf35132e63031416 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 14 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.559285:0:17578:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298640.559288:0:17578:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1780298640.559290:0:17578:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1780298640.559358:0:17578:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+33:29360:x1866777609170048:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1607us (1697us total) trans 4294990929 rc 0/0 00000100:00100000:3.0:1780298640.559363:0:17578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61818 00000020:00080000:3.0:1780298640.559366:0:17578:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1780298640.562388:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609170560 00000100:00100000:3.0:1780298640.562393:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609170560 00010000:00010000:3.0:1780298640.562409:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63031401 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [4268032->37822463] (req 4268032->4276223) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d09b6 expref: 11 pid: 19992 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.562430:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14785 00000100:00100000:3.0:1780298640.562435:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609170560:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298640.562442:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:3.0:1780298640.562451:0:6462:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 760 -> 776 00010000:00010000:3.0:1780298640.562456:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63031401 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [4268032->37822463] (req 4268032->4276223) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d09b6 expref: 11 pid: 19992 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.562463:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63031401 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [4268032->37822463] (req 4268032->4276223) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d09b6 expref: 11 pid: 19992 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.562482:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63031401 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [4268032->37822463] (req 4268032->4276223) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d09b6 expref: 11 pid: 19992 timeout: 929 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.562496:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298640 v=(24 177) 00010000:00010000:3.0:1780298640.562499:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298640 v=(24 177) 00010000:00010000:3.0:1780298640.562503:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000bf17d487/0xaf35132e6303141d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d09cb expref: 6 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.562509:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000bf17d487/0xaf35132e6303141d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d09cb expref: 6 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.562526:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000bf17d487/0xaf35132e6303141d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d09cb expref: 6 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.562533:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548ba680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777604747136:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298640.562566:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.563289:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548ba680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777604747136:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298640.563299:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298640.563340:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609170560:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 903us (975us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298640.563350:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14785 00000100:00100000:3.0:1780298640.564390:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.565259:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f2a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:20253:1866777604747392:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298640.565271:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298640.565298:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548ba680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609170816:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 807us (909us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298640.565327:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14786 00000100:00100000:3.0:1780298640.567595:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609171456 00000100:00100000:3.0:1780298640.567600:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609171456 00000100:00100000:3.0:1780298640.567646:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 7993 00000100:00100000:3.0:1780298640.567658:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b41180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609171456:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.567665:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.567675:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000039c5f58c/0xaf35132e63031440 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d09e7 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.567715:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548ba300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604747520:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.567754:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00002000:00010000:3.0:1780298640.568449:0:8356:0:(ofd_lvb.c:271:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime: 1780298640 -> 1780298641 00000100:00100000:3.0:1780298640.568461:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548ba300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604747520:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.568482:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000039c5f58c/0xaf35132e63031440 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d09e7 expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.568503:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000039c5f58c/0xaf35132e63031440 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d09e7 expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.568515:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3200, rc 0) 00000100:00100000:3.0:1780298640.568549:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b41180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609171456:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 895us (978us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.568557:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7993 00000100:00100000:3.0:1780298640.569631:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.570466:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b42680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604747904:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.570531:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003e31d8c5/0xaf35132e6303144e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d09f5 expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.570541:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e31d8c5/0xaf35132e6303144e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d09f5 expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.570547:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d08a00, rc 0) 00000100:00100000:3.0:1780298640.570573:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609171712:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 859us (948us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.570589:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7995 00000100:00100000:3.0:1780298640.578445:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609172608 00000100:00100000:3.0:1780298640.578450:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609172608 00000100:00100000:3.0:1780298640.578479:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609172736 00000100:00100000:3.0:1780298640.578481:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609172736 00000100:00100000:3.0:1780298640.579396:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b41500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604748544:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.579413:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000522865f2/0xaf35132e6303146a lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0a11 expref: 12 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.579428:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000522865f2/0xaf35132e6303146a lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0a11 expref: 12 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.579436:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d09800, rc 0) 00000100:00100000:3.0:1780298640.579463:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609172608:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 956us (1038us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.579475:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 7999 00000100:00100000:3.0:1780298640.580395:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.582678:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.583676:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.584783:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.588330:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.592252:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.592280:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.594368:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.597557:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.601026:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609175296 00000100:00100000:3.0:1780298640.601030:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609175296 00000100:00100000:3.0:1780298640.601053:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8013 00000100:00100000:3.0:1780298640.601057:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f737480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609175296:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.601064:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.601070:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e630314da lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0a7a expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.601090:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548b8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604750336:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.601144:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.601757:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548b8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604750336:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.601768:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e630314da lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0a7a expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.601777:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e630314da lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0a7a expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.601793:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3200, rc 0) 00000100:00100000:3.0:1780298640.601815:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f737480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609175296:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 757us (820us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.601819:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8013 00000100:00100000:3.0:1780298640.602617:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609175424 00000100:00100000:3.0:1780298640.602621:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609175424 00000100:00100000:3.0:1780298640.602640:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8014 00000100:00100000:3.0:1780298640.602644:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255490700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:29360:x1866777609175424:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.602650:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.602656:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000077813dbd/0xaf35132e630314e1 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0a81 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.602668:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000522865f2/0xaf35132e6303149b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [245760->2183167] (req 245760->258047) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0a42 expref: 13 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.602674:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298640 v=(26 177) 00010000:00010000:3.0:1780298640.602678:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298640 v=(27 177) 00010000:00010000:3.0:1780298640.602681:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000077813dbd/0xaf35132e630314e1 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1904640->1908735] (req 1904640->1908735) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0a81 expref: 10 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.602694:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000522865f2/0xaf35132e6303149b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [245760->2183167] (req 245760->258047) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0a42 expref: 13 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.602703:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000522865f2/0xaf35132e6303149b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [245760->2183167] (req 245760->258047) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0a42 expref: 13 pid: 12101 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.602715:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548b8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604750464:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298640.602747:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.603259:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548b8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604750464:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298640.603269:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000077813dbd/0xaf35132e630314e1 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1904640->1908735] (req 1904640->1908735) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0a81 expref: 10 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.603276:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2200, rc 0) 00000100:00100000:3.0:1780298640.603296:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255490700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:29360:x1866777609175424:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 652us (702us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298640.603300:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8014 00000100:00100000:3.0:1780298640.607663:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.612197:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.612230:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.612312:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8017 00000100:00100000:3.0:1780298640.612324:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548bb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2397:x1866777609176320:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.612331:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.612340:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e630314ef lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0a88 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.612401:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548b8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604750848:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.612465:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00002000:00010000:3.0:1780298640.613215:0:9128:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks: 808 -> 832 00000100:00100000:3.0:1780298640.613228:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548b8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604750848:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.613259:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e630314ef lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0a88 expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.613288:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e630314ef lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0a88 expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.613298:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3200, rc 0) 00000100:00100000:3.0:1780298640.613338:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548bb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2397:x1866777609176320:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1018us (1108us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.613347:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8017 00000100:00100000:3.0:1780298640.614381:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609176576 00000100:00100000:3.0:1780298640.614385:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609176576 00000100:00100000:3.0:1780298640.614421:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8018 00000100:00100000:3.0:1780298640.614430:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609176576:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.614440:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.614449:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e630314f6 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0a96 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.614484:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548b8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604750976:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.614525:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.615237:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255490e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604751104:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.615253:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008d64202a/0xaf35132e630314fd lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0a9d expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.615299:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008d64202a/0xaf35132e630314fd lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0a9d expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.615324:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d09e00, rc 0) 00000100:00100000:3.0:1780298640.616385:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609176960 00000100:00100000:3.0:1780298640.616392:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609176960 00000100:00100000:3.0:1780298640.616425:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8020 00000100:00100000:3.0:1780298640.616431:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609176960:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.616438:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.616446:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e63031504 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0aab expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.616476:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548b9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604751232:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.621244:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.621299:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00002000:00010000:3.0:1780298640.622082:0:8358:0:(ofd_lvb.c:271:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime: 1780298640 -> 1780298641 00000100:00100000:3.0:1780298640.622093:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255491880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604751872:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.622110:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007ba4a4a6/0xaf35132e63031520 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0ac0 expref: 8 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.622143:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ba4a4a6/0xaf35132e63031520 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0ac0 expref: 8 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.622158:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d09c00, rc 0) 00000100:00100000:3.0:1780298640.622193:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609177600:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 822us (894us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.622218:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8024 00000100:00100000:3.0:1780298640.623143:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.623176:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.626038:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.627267:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.628294:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609178368 00000100:00100000:3.0:1780298640.628299:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609178368 00000100:00100000:3.0:1780298640.628334:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8027 00000100:00100000:3.0:1780298640.628339:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255491500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:29360:x1866777609178368:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.628346:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.628354:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e6303153c lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0ad5 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.628371:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000077813dbd/0xaf35132e630314e1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1904640->2183167] (req 1904640->1908735) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0a81 expref: 7 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.628377:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000085c2bc23/0xaf35132e63031478 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2183168->35737599] (req 2183168->2191359) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0a1f expref: 7 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.628381:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 2 to [0x2c0000401:0x11:0x0].0x0 time=1780298640 v=(27 177) 00010000:00010000:3.0:1780298640.628385:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298640 v=(29 177) 00010000:00010000:3.0:1780298640.628387:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e6303153c lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0ad5 expref: 9 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.628400:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000085c2bc23/0xaf35132e63031478 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2183168->35737599] (req 2183168->2191359) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0a1f expref: 7 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.628416:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000085c2bc23/0xaf35132e63031478 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2183168->35737599] (req 2183168->2191359) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0a1f expref: 7 pid: 8358 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.628423:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548ba300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604752384:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298640.628456:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000077813dbd/0xaf35132e630314e1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1904640->2183167] (req 1904640->1908735) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0a81 expref: 7 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.628462:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000077813dbd/0xaf35132e630314e1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1904640->2183167] (req 1904640->1908735) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0a81 expref: 7 pid: 12101 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.628469:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548baa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604752512:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298640.628488:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393c00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.629213:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548baa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604752512:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298640.629230:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e6303153c lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0ad5 expref: 9 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.629243:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3200, rc 0) 00000100:00100000:3.0:1780298640.629273:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255491500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609178368:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 933us (1013us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298640.629280:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8027 00000100:00100000:3.0:1780298640.629454:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.629489:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.630444:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.632650:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609179136 00000100:00100000:3.0:1780298640.632655:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609179136 00002000:00010000:3.0:1780298640.632669:0:16972:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4194304->18446744073709551615) 00002000:00100000:3.0:1780298640.632673:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255493480 with x1866777609179136 ext(4194304->18446744073709551615) 00002000:00010000:3.0:1780298640.632678:0:16972:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf255493480. 00000100:00100000:3.0:1780298640.632699:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5662 00000100:00100000:3.0:1780298640.632709:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255493480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609179136:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:3.0:1780298640.632814:0:16972:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 0 -> 4194304 00002000:00010000:3.0:1780298640.632816:0:16972:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298641 -> 1780298641 00002000:00010000:3.0:1780298640.632818:0:16972:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298641 -> 1780298640 00002000:00010000:3.0:1780298640.632820:0:16972:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298641 -> 1780298641 00002000:00010000:3.0:1780298640.632821:0:16972:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 24 -> 24 00000100:00100000:3.0:1780298640.632847:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255493480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609179136:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 142us (259us total) trans 4294969103 rc 0/0 00000100:00100000:3.0:1780298640.632852:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5662 00002000:00010000:3.0:1780298640.632855:0:16972:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4194304->18446744073709551615) 00002000:00100000:3.0:1780298640.632857:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255493480 with x1866777609179136 ext(4194304->18446744073709551615) 00002000:00010000:3.0:1780298640.632861:0:16972:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf255493480. 00000100:00100000:3.0:1780298640.633815:0:17578:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609179264 00000100:00100000:3.0:1780298640.633820:0:17578:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609179264 00000100:00100000:3.0:1780298640.633856:0:17578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61822 00000100:00100000:3.0:1780298640.633864:0:17578:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255490700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+42:29360:x1866777609179264:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.633879:0:17578:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.633892:0:17578:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3a00/0xaf35132e6303154a lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 16 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0ae3 expref: -99 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.633973:0:17578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3600/0xaf35132e63031551 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 17 type: IBT flags: 0x40000000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.633986:0:17578:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3600/0xaf35132e63031551 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 17 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.633995:0:17578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3600/0xaf35132e63031551 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 17 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1780298640.634018:0:17578:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 8713424 00000004:00200000:3.0:1780298640.634022:0:17578:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1780298640.634038:0:17578:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3600/0xaf35132e63031551 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 17 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.634059:0:17578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3a00/0xaf35132e6303154a lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 17 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0ae3 expref: 44 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.634073:0:17578:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3600/0xaf35132e63031551 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 16 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0ae3 expref: 43 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.634081:0:17578:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298640.634083:0:17578:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1780298640.634086:0:17578:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3600, rc 0) 00000100:00100000:3.0:1780298640.634116:0:17578:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255490700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+43:29360:x1866777609179264:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 253us (334us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298640.634151:0:17578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61822 00000100:00100000:3.0:1780298640.639023:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609179648 00000100:00100000:3.0:1780298640.639030:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609179648 00000100:00100000:3.0:1780298640.639086:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8029 00000100:00100000:3.0:1780298640.639095:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609179648:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.639106:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.639118:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e63031566 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0af1 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.639198:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548b8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604753024:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.639270:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393c00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.641361:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609179904 00000100:00100000:3.0:1780298640.641366:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609179904 00000100:00100000:3.0:1780298640.641399:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8031 00000100:00100000:3.0:1780298640.641406:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255491880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:29360:x1866777609179904:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.641416:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.641426:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005c47b04a/0xaf35132e63031574 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0aff expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.641445:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e6303153c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0ad5 expref: 9 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.641454:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298640 v=(29 177) 00010000:00010000:3.0:1780298640.641457:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298640 v=(30 177) 00010000:00010000:3.0:1780298640.641462:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005c47b04a/0xaf35132e63031574 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [4407296->4415487] (req 4407296->4415487) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0aff expref: 7 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.641483:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e6303153c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0ad5 expref: 9 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.641509:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e6303153c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0ad5 expref: 9 pid: 9128 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.641519:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548b9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604753280:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298640.641563:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.642270:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548b9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604753280:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298640.642284:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005c47b04a/0xaf35132e63031574 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [4407296->4415487] (req 4407296->4415487) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0aff expref: 7 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.642297:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3400, rc 0) 00000100:00100000:3.0:1780298640.642338:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255491880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609179904:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 933us (1002us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298640.642345:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8031 00000100:00100000:3.0:1780298640.647348:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.650242:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.651538:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609180800 00000100:00100000:3.0:1780298640.651542:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609180800 00002000:00100000:3.0:1780298640.651560:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (3149824->3153919) req@ffff9cf3481fd180 x1866777609180800/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:346/0 lens 488/0 e 0 to 0 dl 1780298651 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298640.651566:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fd180 with x1866777609180800 ext(3149824->3153919) 00002000:00010000:3.0:1780298640.651568:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481fd180 00000100:00100000:3.0:1780298640.651585:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5666 00000100:00100000:3.0:1780298640.651590:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609180800:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298640.651675:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609180800:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 83us (204us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1780298640.651682:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5666 00002000:00100000:3.0:1780298640.651688:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (3149824->3153919) req@ffff9cf3481fd180 x1866777609180800/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:346/0 lens 488/4536 e 0 to 0 dl 1780298651 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298640.651696:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fd180 with x1866777609180800 ext(3149824->3153919) 00002000:00010000:3.0:1780298640.651699:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481fd180 00000100:00100000:3.0:1780298640.652940:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609180928 00000100:00100000:3.0:1780298640.652945:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609180928 00000100:00100000:3.0:1780298640.652968:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8035 00000100:00100000:3.0:1780298640.652973:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609180928:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.652981:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.652992:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e63031590 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0b1b expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.653031:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548b9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604753792:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.653076:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.653886:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548b9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604753792:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.653905:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e63031590 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0b1b expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.653919:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e63031590 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0b1b expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.653935:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3a00, rc 0) 00000100:00100000:3.0:1780298640.653959:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609180928:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 986us (1059us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.653964:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8035 00000100:00100000:3.0:1780298640.660143:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609181440 00000100:00100000:3.0:1780298640.660148:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609181440 00000100:00100000:3.0:1780298640.660176:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8039 00000100:00100000:3.0:1780298640.660184:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609181440:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.660192:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.660202:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e630315ac lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0b37 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.660218:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298640 v=(30 177) 00010000:00010000:3.0:1780298640.660223:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298640 v=(30 177) 00010000:00010000:3.0:1780298640.660232:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e630315ac lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3108864->3141631] (req 3108864->3121151) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0b37 expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.660246:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e630315ac lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3108864->3141631] (req 3108864->3121151) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0b37 expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.660255:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2a00, rc 0) 00000100:00100000:3.0:1780298640.660280:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:29360:x1866777609181440:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 97us (190us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298640.660287:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8039 00000100:00100000:3.0:1780298640.667232:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.668163:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.669345:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.670614:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609182720 00000100:00100000:3.0:1780298640.670620:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609182720 00000100:00100000:3.0:1780298640.670656:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8046 00000100:00100000:3.0:1780298640.670668:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242860e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609182720:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.670677:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.670687:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e630315dd lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0b68 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.670723:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548b8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604755200:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.670756:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.672355:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.672372:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.674232:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.674252:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.676824:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.678042:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.680017:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8055 00000100:00100000:3.0:1780298640.680030:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242860700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609184000:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.680042:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298640.681899:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.681922:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.682773:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604756480:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.682792:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000021dd6ca7/0xaf35132e6303162a lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0bb5 expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.682816:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000021dd6ca7/0xaf35132e6303162a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0bb5 expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.682825:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36200, rc 0) 00000100:00100000:3.0:1780298640.682848:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609184256:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 839us (924us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.682854:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8057 00000100:00100000:3.0:1780298640.683705:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.683723:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.685447:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.685466:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.685527:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8061 00000100:00100000:3.0:1780298640.685540:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609184768:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.685548:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.685557:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000039c5f58c/0xaf35132e63031646 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0bd1 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.685585:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604756992:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.685625:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393c00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.688827:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.690466:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.694077:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609186048 00000100:00100000:3.0:1780298640.694081:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609186048 00000100:00100000:3.0:1780298640.694102:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8068 00000100:00100000:3.0:1780298640.694113:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:29360:x1866777609186048:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.694119:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.694139:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e6303167e lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0c02 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.694153:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e63031589 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3141632->4407295] (req 3141632->3153919) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0b14 expref: 10 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.694159:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005c47b04a/0xaf35132e63031574 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [4407296->37961727] (req 4407296->4415487) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0aff expref: 9 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.694163:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e630315ac lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3108864->3141631] (req 3108864->3121151) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0b37 expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.694167:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 3 to [0x2c0000401:0x11:0x0].0x0 time=1780298640 v=(30 177) 00010000:00010000:3.0:1780298640.694171:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298640 v=(33 177) 00010000:00010000:3.0:1780298640.694173:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e6303167e lrc: 6/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0c02 expref: 9 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.694185:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e630315ac lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3108864->3141631] (req 3108864->3121151) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0b37 expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.694192:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e630315ac lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3108864->3141631] (req 3108864->3121151) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0b37 expref: 10 pid: 9128 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.694198:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604758016:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298640.694235:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005c47b04a/0xaf35132e63031574 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [4407296->37961727] (req 4407296->4415487) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0aff expref: 9 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.694242:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005c47b04a/0xaf35132e63031574 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [4407296->37961727] (req 4407296->4415487) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0aff expref: 9 pid: 9128 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.694252:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604758144:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298640.694273:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e63031589 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3141632->4407295] (req 3141632->3153919) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0b14 expref: 10 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.694280:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e63031589 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3141632->4407295] (req 3141632->3153919) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0b14 expref: 10 pid: 8359 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.694286:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604758272:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298640.694364:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393c00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.694846:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24310ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604758016:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298640.694882:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24310e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604758144:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298640.694889:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24310c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604758272:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298640.694900:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e6303167e lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0c02 expref: 9 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.694940:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3a00, rc 0) 00000100:00100000:3.0:1780298640.694959:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:29360:x1866777609186048:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 847us (906us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298640.694968:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8068 00000100:00100000:3.0:1780298640.695366:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609186432 00000100:00100000:3.0:1780298640.695370:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609186432 00002000:00100000:3.0:1780298640.695387:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (3141632->3153919) req@ffff9cf24519dc00 x1866777609186432/t0(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:346/0 lens 12776/0 e 0 to 0 dl 1780298651 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298640.695394:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24519dc00 with x1866777609186432 ext(3141632->3153919) 00010000:00010000:3.0:1780298640.695398:0:19943:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e63031589 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3141632->4407295] (req 3141632->3153919) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0b14 expref: 10 pid: 8359 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.695406:0:19943:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e63031589 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3141632->4407295] (req 3141632->3153919) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0b14 expref: 10 pid: 8359 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1780298640.695447:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf24519dc00 00000100:00100000:3.0:1780298640.695463:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 166 00000100:00100000:3.0:1780298640.695468:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609186432:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:3.0:1780298640.695909:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609186432:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 441us (646us total) trans 4294969515 rc 0/0 00000100:00100000:3.0:1780298640.695915:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 166 00002000:00100000:3.0:1780298640.695919:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (3141632->3153919) req@ffff9cf24519dc00 x1866777609186432/t4294969515(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:346/0 lens 12776/448 e 0 to 0 dl 1780298651 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298640.695925:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24519dc00 with x1866777609186432 ext(3141632->3153919) 00010000:00010000:3.0:1780298640.695929:0:19943:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e63031589 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3141632->4407295] (req 3141632->3153919) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0b14 expref: 12 pid: 8359 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.695945:0:19943:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e63031589 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3141632->4407295] (req 3141632->3153919) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0b14 expref: 12 pid: 8359 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1780298640.695950:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf24519dc00 00000100:00100000:3.0:1780298640.699271:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.700818:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.703695:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609187712 00000100:00100000:3.0:1780298640.703700:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609187712 00000100:00100000:3.0:1780298640.703727:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8072 00000100:00100000:3.0:1780298640.703735:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609187712:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.703743:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.703751:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000077813dbd/0xaf35132e630316a8 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0c25 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.703823:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604759168:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.703874:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393c00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.704533:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24310ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604759168:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.704544:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000077813dbd/0xaf35132e630316a8 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0c25 expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.704556:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000077813dbd/0xaf35132e630316a8 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0c25 expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.704565:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2200, rc 0) 00000100:00100000:3.0:1780298640.704587:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609187712:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 854us (950us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.704600:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8072 00000100:00100000:3.0:1780298640.705447:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.706915:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.707551:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604759680:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298640.707562:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e630316c4 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2011136->2015231] (req 2011136->2015231) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0c41 expref: 12 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.707570:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36c00, rc 0) 00000100:00100000:3.0:1780298640.707591:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:29360:x1866777609188224:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 620us (677us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298640.707596:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8076 00000100:00100000:3.0:1780298640.710729:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609188736 00000100:00100000:3.0:1780298640.710733:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609188736 00000100:00100000:3.0:1780298640.710754:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8077 00000100:00100000:3.0:1780298640.710759:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2397:x1866777609188736:12345-192.168.203.25@tcp:16:fsx.0 00000100:00100000:3.0:1780298640.711592:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609188736:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 832us (899us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298640.711598:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8077 00000100:00100000:3.0:1780298640.712257:0:17578:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609188864 00000100:00100000:3.0:1780298640.712262:0:17578:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609188864 00000100:00100000:3.0:1780298640.712287:0:17578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61826 00000100:00100000:3.0:1780298640.712294:0:17578:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+36:29360:x1866777609188864:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298640.713331:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609188992 00000100:00100000:3.0:1780298640.713341:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609188992 00000100:00100000:3.0:1780298640.713363:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8078 00000100:00100000:3.0:1780298640.713370:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609188992:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.713377:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.713383:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000077813dbd/0xaf35132e630316d9 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0c4f expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.713406:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604759936:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.713438:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00002000:00010000:3.0:1780298640.713996:0:12101:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks: 664 -> 672 00000100:00100000:3.0:1780298640.714006:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24310df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604759936:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.714020:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000077813dbd/0xaf35132e630316d9 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0c4f expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.714033:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000077813dbd/0xaf35132e630316d9 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0c4f expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.714042:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2200, rc 0) 00000100:00100000:3.0:1780298640.714068:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609188992:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 699us (777us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.714075:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8078 00000100:00100000:3.0:1780298640.716864:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609189248 00000100:00100000:3.0:1780298640.716869:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609189248 00000100:00100000:3.0:1780298640.716897:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8080 00000100:00100000:3.0:1780298640.716905:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:29360:x1866777609189248:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.716923:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.716930:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e630316e7 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0c5d expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.716948:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e630316c4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2011136->35565567] (req 2011136->2015231) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0c41 expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.716957:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298640 v=(34 177) 00010000:00010000:3.0:1780298640.716961:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298640 v=(35 177) 00010000:00010000:3.0:1780298640.716966:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e630316e7 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3739648->3747839] (req 3739648->3747839) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0c5d expref: 7 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.716986:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e630316c4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2011136->35565567] (req 2011136->2015231) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0c41 expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.717000:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e630316c4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2011136->35565567] (req 2011136->2015231) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0c41 expref: 8 pid: 8356 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.717010:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604760192:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298640.717048:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.717638:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24310df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604760192:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298640.717653:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e630316e7 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3739648->3747839] (req 3739648->3747839) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0c5d expref: 7 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.717660:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3c00, rc 0) 00000100:00100000:3.0:1780298640.717726:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609189248:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 823us (896us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298640.717733:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8080 00000100:00100000:3.0:1780298640.717881:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609189376 00000100:00100000:3.0:1780298640.717884:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609189376 00010000:00010000:3.0:1780298640.717896:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e630316c4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2011136->35565567] (req 2011136->2015231) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0c41 expref: 9 pid: 8356 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.717914:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14804 00000100:00100000:3.0:1780298640.717919:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609189376:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298640.717925:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298640.717932:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e630316c4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2011136->35565567] (req 2011136->2015231) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0c41 expref: 9 pid: 8356 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.717939:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e630316c4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2011136->35565567] (req 2011136->2015231) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0c41 expref: 9 pid: 8356 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.717949:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e630316c4 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2011136->35565567] (req 2011136->2015231) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0c41 expref: 9 pid: 8356 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.717956:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298640 v=(35 177) 00010000:00010000:3.0:1780298640.717958:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298640 v=(35 177) 00010000:00010000:3.0:1780298640.717964:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e630316e7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3739648->37294079] (req 3739648->3747839) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0c5d expref: 6 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.717973:0:6463:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e630316e7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3739648->37294079] (req 3739648->3747839) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0c5d expref: 6 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.718001:0:6463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e630316e7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3739648->37294079] (req 3739648->3747839) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0c5d expref: 6 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.718011:0:6463:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310df80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6463:1866777604760320:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298640.718046:0:6463:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.718664:0:6463:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24310df80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6463:1866777604760320:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298640.718671:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298640.718692:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2400:x1866777609189376:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 773us (870us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298640.718697:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14804 00000100:00100000:3.0:1780298640.719187:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609189632 00000100:00100000:3.0:1780298640.719193:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609189632 00000100:00100000:3.0:1780298640.719226:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8081 00000100:00100000:3.0:1780298640.719235:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255491500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609189632:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.719242:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.719251:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e630316ee lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0c6b expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.719289:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604760448:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.719349:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.719945:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24310df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604760448:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.719956:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e630316ee lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0c6b expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.719975:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e630316ee lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0c6b expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.719984:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2200, rc 0) 00000100:00100000:3.0:1780298640.720009:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255491500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609189632:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 778us (855us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.720014:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8081 00000100:00100000:3.0:1780298640.720996:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609189888 00000100:00100000:3.0:1780298640.721002:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609189888 00000100:00100000:3.0:1780298640.721837:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604760832:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.721898:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004d34ec35/0xaf35132e63031703 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0c79 expref: 14 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.721919:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004d34ec35/0xaf35132e63031703 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0c79 expref: 14 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.721931:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37e00, rc 0) 00000100:00100000:3.0:1780298640.723803:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609190144 00000100:00100000:3.0:1780298640.723807:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609190144 00002000:00100000:3.0:1780298640.723826:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (1310720->1314815) req@ffff9cf255490000 x1866777609190144/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:346/0 lens 488/0 e 0 to 0 dl 1780298651 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298640.723833:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255490000 with x1866777609190144 ext(1310720->1314815) 00002000:00010000:3.0:1780298640.723837:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255490000 00000100:00100000:3.0:1780298640.723858:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5674 00000100:00100000:3.0:1780298640.723865:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255490000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609190144:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298640.723966:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255490000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609190144:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 103us (188us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1780298640.723972:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5674 00002000:00100000:3.0:1780298640.723978:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (1310720->1314815) req@ffff9cf255490000 x1866777609190144/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:346/0 lens 488/4536 e 0 to 0 dl 1780298651 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298640.723984:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255490000 with x1866777609190144 ext(1310720->1314815) 00002000:00010000:3.0:1780298640.723987:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255490000 00000100:00100000:3.0:1780298640.724827:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609190400 00000100:00100000:3.0:1780298640.724833:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609190400 00000100:00100000:3.0:1780298640.724878:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8087 00000100:00100000:3.0:1780298640.724890:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609190400:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.724899:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.724909:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e63031718 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0c8e expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.724943:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604761088:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.725644:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24310df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604761088:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.725660:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e63031718 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0c8e expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.725674:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e63031718 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0c8e expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.725701:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3800, rc 0) 00000100:00100000:3.0:1780298640.725728:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609190400:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 841us (923us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.725734:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8087 00000100:00100000:3.0:1780298640.731834:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.733005:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8092 00000100:00100000:3.0:1780298640.733016:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2397:x1866777609191296:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.733024:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.733033:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000077813dbd/0xaf35132e6303173b lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0cb1 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.733061:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310e680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604761856:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.733099:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393c00 going to sleep for 11 seconds 00002000:00010000:3.0:1780298640.733697:0:8356:0:(ofd_lvb.c:259:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size: 3203665 -> 3745341 00000100:00100000:3.0:1780298640.733704:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24310e680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604761856:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.733756:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000077813dbd/0xaf35132e6303173b lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0cb1 expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.733767:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000077813dbd/0xaf35132e6303173b lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0cb1 expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.733775:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2200, rc 0) 00000100:00100000:3.0:1780298640.733799:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2397:x1866777609191296:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 788us (866us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.733809:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8092 00000100:00100000:3.0:1780298640.734742:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609191680 00000100:00100000:3.0:1780298640.734747:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609191680 00000100:00100000:3.0:1780298640.734782:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8095 00000100:00100000:3.0:1780298640.734795:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609191680:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.734801:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.734809:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e63031750 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0cc6 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.734836:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310e680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604762240:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.734862:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393c00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.735438:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24310e680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604762240:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.735455:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e63031750 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0cc6 expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.735468:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e63031750 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0cc6 expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.735479:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3800, rc 0) 00000100:00100000:3.0:1780298640.735504:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609191680:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 712us (784us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.735523:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8095 00000100:00100000:3.0:1780298640.739313:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604762496:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298640.739324:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604762624:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298640.739346:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bf17d487/0xaf35132e6303175e lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0ccd expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.739356:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37400, rc 0) 00000100:00100000:3.0:1780298640.739378:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548bb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609192192:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 972us (1035us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298640.739383:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8096 00000100:00100000:3.0:1780298640.739667:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609192448 00000100:00100000:3.0:1780298640.739672:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609192448 00010000:00010000:3.0:1780298640.739694:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000042649f54/0xaf35132e63031734 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3198976->3739647] (req 3198976->3207167) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0caa expref: 10 pid: 12101 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.739716:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14807 00000100:00100000:3.0:1780298640.739724:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609192448:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298640.739731:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298640.739746:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000042649f54/0xaf35132e63031734 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [3198976->3739647] (req 3198976->3207167) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0caa expref: 10 pid: 12101 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.739769:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000042649f54/0xaf35132e63031734 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3198976->3739647] (req 3198976->3207167) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0caa expref: 10 pid: 12101 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.739781:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000042649f54/0xaf35132e63031734 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3198976->3739647] (req 3198976->3207167) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0caa expref: 10 pid: 12101 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.739790:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298640.739813:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609192448:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 89us (209us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298640.739824:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14807 00000100:00100000:3.0:1780298640.742044:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609192704 00000100:00100000:3.0:1780298640.742051:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609192704 00000100:00100000:3.0:1780298640.742204:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8097 00000100:00100000:3.0:1780298640.742211:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:29360:x1866777609192704:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.742219:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.742230:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e63031765 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0cd4 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.742247:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000027b1d9e0/0xaf35132e63031685 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0c09 expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.742254:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298640 v=(18 130) 00010000:00010000:3.0:1780298640.742258:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298640 v=(19 130) 00010000:00010000:3.0:1780298640.742261:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e63031765 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0cd4 expref: 14 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.742280:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000027b1d9e0/0xaf35132e63031685 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0c09 expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.742290:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000027b1d9e0/0xaf35132e63031685 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0c09 expref: 10 pid: 9128 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.742299:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310e680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604762880:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298640.742378:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393c00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.742953:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24310e680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604762880:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298640.742969:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e63031765 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0cd4 expref: 14 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.742981:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2600, rc 0) 00000100:00100000:3.0:1780298640.743014:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:29360:x1866777609192704:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 804us (1004us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298640.743021:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8097 00000100:00100000:3.0:1780298640.744346:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.746037:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.747715:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609193344 00000100:00100000:3.0:1780298640.747721:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609193344 00000100:00100000:3.0:1780298640.747760:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5678 00000100:00100000:3.0:1780298640.747768:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609193344:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:3.0:1780298640.747905:0:19943:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 948412 -> 948412 00002000:00010000:3.0:1780298640.747908:0:19943:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298641 -> 1780298641 00002000:00010000:3.0:1780298640.747911:0:19943:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298641 -> 1780298641 00002000:00010000:3.0:1780298640.747913:0:19943:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298641 -> 1780298641 00002000:00010000:3.0:1780298640.747915:0:19943:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 384 -> 400 00000100:00100000:3.0:1780298640.747951:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609193344:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 185us (282us total) trans 4294969521 rc 0/0 00000100:00100000:3.0:1780298640.747959:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5678 00000100:00100000:3.0:1780298640.750350:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609193728 00000100:00100000:3.0:1780298640.750355:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609193728 00000100:00100000:3.0:1780298640.750398:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8099 00000100:00100000:3.0:1780298640.750408:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548bb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609193728:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.750429:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.750440:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e6303178f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0cf0 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.750471:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310d180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604763264:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.750505:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393c00 going to sleep for 11 seconds 00002000:00010000:3.0:1780298640.751454:0:8359:0:(ofd_lvb.c:271:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime: 1780298640 -> 1780298641 00000100:00100000:3.0:1780298640.751468:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24310d180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604763264:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.751483:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e6303178f lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0cf0 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.751498:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e6303178f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0cf0 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.751507:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3800, rc 0) 00000100:00100000:3.0:1780298640.751537:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548bb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609193728:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1132us (1250us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.751544:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8099 00000100:00100000:3.0:1780298640.752478:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609193856 00000100:00100000:3.0:1780298640.752483:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609193856 00000100:00100000:3.0:1780298640.752518:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8100 00000100:00100000:3.0:1780298640.752529:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609193856:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.752538:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.752548:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000077813dbd/0xaf35132e63031796 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0cf7 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.752604:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310d180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604763392:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.752656:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393c00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.753443:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24310d180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604763392:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.753471:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000077813dbd/0xaf35132e63031796 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0cf7 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.753487:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000077813dbd/0xaf35132e63031796 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0cf7 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.753499:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2200, rc 0) 00000100:00100000:3.0:1780298640.753523:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609193856:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1000us (1083us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.753529:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8100 00000100:00100000:3.0:1780298640.755998:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609194240 00000100:00100000:3.0:1780298640.756005:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609194240 00000100:00100000:3.0:1780298640.756045:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8102 00000100:00100000:3.0:1780298640.756056:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609194240:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.756066:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.756084:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e630317a4 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0d05 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.756156:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310d180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604763648:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.759669:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.762789:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.777044:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609195648 00000100:00100000:3.0:1780298640.777095:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609195648 00000100:00100000:3.0:1780298640.777151:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5681 00000100:00100000:3.0:1780298640.777158:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243847480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609195648:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:3.0:1780298640.777268:0:8364:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 948412 -> 948412 00002000:00010000:3.0:1780298640.777271:0:8364:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298641 -> 1780298641 00002000:00010000:3.0:1780298640.777273:0:8364:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298641 -> 1780298641 00002000:00010000:3.0:1780298640.777274:0:8364:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298641 -> 1780298641 00002000:00010000:3.0:1780298640.777276:0:8364:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 424 -> 424 00000100:00100000:3.0:1780298640.777298:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243847480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609195648:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 142us (320us total) trans 4294969523 rc 0/0 00000100:00100000:3.0:1780298640.777316:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5681 00000100:00100000:3.0:1780298640.778240:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609195904 00000100:00100000:3.0:1780298640.778244:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609195904 00000100:00100000:3.0:1780298640.778282:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8111 00000100:00100000:3.0:1780298640.778292:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243846300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609195904:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.778309:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.778320:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e630317e3 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0d44 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.778362:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604764928:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.778401:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.779180:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24310f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604764928:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.779202:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e630317e3 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0d44 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.779244:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e630317e3 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0d44 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.779265:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2200, rc 0) 00000100:00100000:3.0:1780298640.779293:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243846300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609195904:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1003us (1083us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.779312:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8111 00000100:00100000:3.0:1780298640.780265:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609196160 00000100:00100000:3.0:1780298640.780271:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609196160 00000100:00100000:3.0:1780298640.780363:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8113 00000100:00100000:3.0:1780298640.780372:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242860000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609196160:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.780380:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.780390:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e630317f1 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0d52 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.780424:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604765312:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.780452:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.781066:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24310f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604765312:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.781086:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e630317f1 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0d52 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.781104:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e630317f1 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0d52 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.781180:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3800, rc 0) 00000100:00100000:3.0:1780298640.781219:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242860000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609196160:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 846us (987us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.781233:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8113 00000100:00100000:3.0:1780298640.782474:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8115 00000100:00100000:3.0:1780298640.782487:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609196416:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.782502:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.782512:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e630317ff lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0d60 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.782547:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604765568:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.782578:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.783355:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24310f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604765568:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.783375:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e630317ff lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0d60 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.783390:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e630317ff lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0d60 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.783410:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2800, rc 0) 00000100:00100000:3.0:1780298640.783437:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609196416:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 955us (1057us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.783442:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8115 00000100:00100000:3.0:1780298640.784523:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609196544 00000100:00100000:3.0:1780298640.784528:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609196544 00000100:00100000:3.0:1780298640.788511:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.790022:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.791255:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.792529:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609197312 00000100:00100000:3.0:1780298640.792534:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609197312 00010000:00010000:3.0:1780298640.792592:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000007ba4a4a6/0xaf35132e630317d5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [339968->33894399] (req 339968->348159) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0d36 expref: 11 pid: 9128 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.792620:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14813 00000100:00100000:3.0:1780298640.792629:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243844000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609197312:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298640.792640:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298640.792657:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000007ba4a4a6/0xaf35132e630317d5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [339968->33894399] (req 339968->348159) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0d36 expref: 11 pid: 9128 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.792669:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000007ba4a4a6/0xaf35132e630317d5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [339968->33894399] (req 339968->348159) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0d36 expref: 11 pid: 9128 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.792687:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007ba4a4a6/0xaf35132e630317d5 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [339968->33894399] (req 339968->348159) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0d36 expref: 11 pid: 9128 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.792698:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298640 v=(40 177) 00010000:00010000:3.0:1780298640.792703:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298640 v=(40 177) 00010000:00010000:3.0:1780298640.792709:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000bf17d487/0xaf35132e6303181b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0d75 expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.792735:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000bf17d487/0xaf35132e6303181b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0d75 expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.792759:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000bf17d487/0xaf35132e6303181b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0d75 expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.792770:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310f100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777604766208:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298640.792810:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.793865:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5683 00000100:00100000:3.0:1780298640.793878:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243845c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2398:x1866777609197568:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:3.0:1780298640.793983:0:19943:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 0 -> 4194304 00002000:00010000:3.0:1780298640.793986:0:19943:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298641 -> 1780298641 00002000:00010000:3.0:1780298640.793988:0:19943:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298641 -> 1780298640 00002000:00010000:3.0:1780298640.793993:0:19943:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298641 -> 1780298641 00002000:00010000:3.0:1780298640.793999:0:19943:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:3.0:1780298640.794030:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243845c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2398:x1866777609197568:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 156us (263us total) trans 4294969108 rc 0/0 00000100:00100000:3.0:1780298640.794035:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5683 00002000:00010000:3.0:1780298640.794038:0:19943:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4194304->18446744073709551615) 00002000:00100000:3.0:1780298640.794041:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243845c00 with x1866777609197568 ext(4194304->18446744073709551615) 00002000:00010000:3.0:1780298640.794045:0:19943:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf243845c00. 00000100:00100000:3.0:1780298640.802494:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609198336 00000100:00100000:3.0:1780298640.802509:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609198336 00000100:00100000:3.0:1780298640.802583:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8121 00000100:00100000:3.0:1780298640.802601:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243845500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609198336:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.802618:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.802635:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63031837 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0d8a expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.802697:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604766464:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.802813:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.804569:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609198592 00000100:00100000:3.0:1780298640.804574:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609198592 00000100:00100000:3.0:1780298640.804622:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8123 00000100:00100000:3.0:1780298640.804632:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243846300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609198592:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.804641:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.804652:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e63031845 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0d98 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.804685:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604766720:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.804738:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393c00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.805444:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24310ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604766720:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.805467:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e63031845 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0d98 expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.805484:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e63031845 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0d98 expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.805518:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3800, rc 0) 00000100:00100000:3.0:1780298640.805544:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243846300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609198592:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 914us (1005us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.805550:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8123 00000100:00100000:3.0:1780298640.808227:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609198848 00000100:00100000:3.0:1780298640.808231:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609198848 00000100:00100000:3.0:1780298640.808256:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8124 00000100:00100000:3.0:1780298640.808261:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243846d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:29360:x1866777609198848:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.808267:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.808274:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e6303185a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0da6 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.808289:0:19992:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e63031765 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0cd4 expref: 15 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.808295:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298640 v=(19 130) 00010000:00010000:3.0:1780298640.808310:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298640 v=(20 130) 00010000:00010000:3.0:1780298640.808317:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e6303185a lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1474560->1486847] (req 1474560->1486847) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0da6 expref: 11 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.808339:0:19992:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e63031765 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0cd4 expref: 15 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.808350:0:19992:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e63031765 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0cd4 expref: 15 pid: 9128 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.808365:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604766848:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298640.808401:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393c00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.809015:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24310ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604766848:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298640.809030:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e6303185a lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1474560->1486847] (req 1474560->1486847) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0da6 expref: 11 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.809039:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2200, rc 0) 00000100:00100000:3.0:1780298640.809071:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243846d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:29360:x1866777609198848:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 801us (863us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298640.809182:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8124 00000100:00100000:3.0:1780298640.810357:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609199104 00000100:00100000:3.0:1780298640.810368:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609199104 00000100:00100000:3.0:1780298640.810398:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5686 00000100:00100000:3.0:1780298640.810403:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a82d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609199104:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:3.0:1780298640.810545:0:8364:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:3.0:1780298640.810548:0:8364:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298641 -> 1780298641 00002000:00010000:3.0:1780298640.810550:0:8364:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298641 -> 1780298640 00002000:00010000:3.0:1780298640.810551:0:8364:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298641 -> 1780298641 00002000:00010000:3.0:1780298640.810553:0:8364:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:3.0:1780298640.810576:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a82d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609199104:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 172us (253us total) trans 4294969109 rc 0/0 00000100:00100000:3.0:1780298640.810582:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5686 00000100:00100000:3.0:1780298640.811412:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609199232 00000100:00100000:3.0:1780298640.811421:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609199232 00000100:00100000:3.0:1780298640.811444:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8125 00000100:00100000:3.0:1780298640.811451:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2400:x1866777609199232:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.811459:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.811469:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e63031861 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0dad expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.811501:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604767104:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.811539:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393c00 going to sleep for 11 seconds 00002000:00010000:3.0:1780298640.812217:0:8356:0:(ofd_lvb.c:271:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime: 1780298640 -> 1780298641 00000100:00100000:3.0:1780298640.812226:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24310ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604767104:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.812245:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e63031861 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0dad expref: 16 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.812262:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e63031861 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0dad expref: 16 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.812271:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3000, rc 0) 00000100:00100000:3.0:1780298640.812294:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2400:x1866777609199232:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 844us (905us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.812298:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8125 00000100:00100000:3.0:1780298640.815169:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609199488 00000100:00100000:3.0:1780298640.815174:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609199488 00000100:00100000:3.0:1780298640.815200:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609199616 00000100:00100000:3.0:1780298640.815202:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609199616 00000100:00100000:3.0:1780298640.816008:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241a81500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604767488:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.816038:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000af0426c7/0xaf35132e63031876 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0dc2 expref: 13 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.816051:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000af0426c7/0xaf35132e63031876 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0dc2 expref: 13 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.816061:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34841d200, rc 0) 00000100:00100000:3.0:1780298640.816104:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a81880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609199616:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 873us (930us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.816114:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8128 00000100:00100000:3.0:1780298640.817151:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609199744 00000100:00100000:3.0:1780298640.817155:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609199744 00000100:00100000:3.0:1780298640.817184:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8129 00000100:00100000:3.0:1780298640.817191:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609199744:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.817198:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.817206:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b93eb96b/0xaf35132e6303187d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0dc9 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.817237:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604767616:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.817279:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393c00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.818011:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24310ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604767616:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.818024:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b93eb96b/0xaf35132e6303187d lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0dc9 expref: 7 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.818035:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b93eb96b/0xaf35132e6303187d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0dc9 expref: 7 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.818048:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3800, rc 0) 00000100:00100000:3.0:1780298640.818071:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609199744:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 882us (982us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.818077:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8129 00000100:00100000:3.0:1780298640.819003:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609200128 00000100:00100000:3.0:1780298640.819007:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609200128 00000100:00100000:3.0:1780298640.819041:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8131 00000100:00100000:3.0:1780298640.819048:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a82680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609200128:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.819055:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.819064:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e6303188b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0dde expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.819095:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604767872:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.819155:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393c00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.820843:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609200256 00000100:00100000:3.0:1780298640.820847:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609200256 00000100:00100000:3.0:1780298640.820871:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8133 00000100:00100000:3.0:1780298640.820875:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a81f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:29360:x1866777609200256:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.820882:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.820888:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e63031899 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0de5 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.820901:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298640 v=(20 130) 00010000:00010000:3.0:1780298640.820904:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298640 v=(20 130) 00010000:00010000:3.0:1780298640.820916:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e63031899 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [180224->1474559] (req 180224->184319) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0de5 expref: 13 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.820927:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e63031899 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [180224->1474559] (req 180224->184319) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0de5 expref: 13 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.820933:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2e00, rc 0) 00000100:00100000:3.0:1780298640.820954:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a81f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:29360:x1866777609200256:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 78us (146us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298640.820958:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8133 00000100:00100000:3.0:1780298640.824083:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609200640 00000100:00100000:3.0:1780298640.824088:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609200640 00000100:00100000:3.0:1780298640.824120:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8134 00000100:00100000:3.0:1780298640.824144:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a83b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609200640:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.824151:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.824161:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e630318a0 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0dec expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.826038:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609201024 00000100:00100000:3.0:1780298640.826043:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609201024 00000100:00100000:3.0:1780298640.826081:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8137 00000100:00100000:3.0:1780298640.826094:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a81500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2397:x1866777609201024:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.826102:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.826112:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e630318b5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0e01 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.826157:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604768512:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.832383:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.832500:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.835078:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.835833:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.836754:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.838273:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.838349:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.843340:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.845641:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609204096 00000100:00100000:3.0:1780298640.845645:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609204096 00002000:00010000:3.0:1780298640.845660:0:8362:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:3.0:1780298640.845663:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f7bd880 with x1866777609204096 ext(4194304->18446744073709551615) 00002000:00010000:3.0:1780298640.845674:0:8362:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f7bd880. 00000100:00100000:3.0:1780298640.845692:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5695 00000100:00100000:3.0:1780298640.845701:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2399:x1866777609204096:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:3.0:1780298640.845754:0:8362:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:3.0:1780298640.845758:0:8362:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298641 -> 1780298641 00002000:00010000:3.0:1780298640.845762:0:8362:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298641 -> 1780298641 00002000:00010000:3.0:1780298640.845765:0:8362:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298641 -> 1780298641 00002000:00010000:3.0:1780298640.845768:0:8362:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 424 -> 424 00000100:00100000:3.0:1780298640.845790:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609204096:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 92us (189us total) trans 4294969529 rc 0/0 00000100:00100000:3.0:1780298640.845796:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5695 00002000:00010000:3.0:1780298640.845798:0:8362:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:3.0:1780298640.845801:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f7bd880 with x1866777609204096 ext(4194304->18446744073709551615) 00002000:00010000:3.0:1780298640.845804:0:8362:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f7bd880. 00000100:00100000:3.0:1780298640.850867:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.850888:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.853335:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.854412:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.855329:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.856298:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.857342:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.858326:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.863221:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609206528 00000100:00100000:3.0:1780298640.863227:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609206528 00000100:00100000:3.0:1780298640.863275:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8149 00000100:00100000:3.0:1780298640.863285:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609206528:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.863295:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.863321:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b93eb96b/0xaf35132e63031987 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0e86 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.863366:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf348007100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604770944:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.863428:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.866456:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609206912 00000100:00100000:3.0:1780298640.866461:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609206912 00010000:00010000:3.0:1780298640.866486:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000021dd6ca7/0xaf35132e6303195d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0e6a expref: 8 pid: 19992 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.866513:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14822 00000100:00100000:3.0:1780298640.866520:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242862300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609206912:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298640.866529:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298640.866549:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000021dd6ca7/0xaf35132e6303195d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0e6a expref: 8 pid: 19992 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.866559:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000021dd6ca7/0xaf35132e6303195d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0e6a expref: 8 pid: 19992 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.866573:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000021dd6ca7/0xaf35132e6303195d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0e6a expref: 8 pid: 19992 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.866583:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298640 v=(42 177) 00010000:00010000:3.0:1780298640.866586:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298640 v=(42 177) 00010000:00010000:3.0:1780298640.866594:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000009cb6f854/0xaf35132e63031995 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1409024->34963455] (req 1409024->1417215) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0e94 expref: 12 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.866603:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009cb6f854/0xaf35132e63031995 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1409024->34963455] (req 1409024->1417215) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0e94 expref: 12 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.866625:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000009cb6f854/0xaf35132e63031995 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1409024->34963455] (req 1409024->1417215) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0e94 expref: 12 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.866634:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf348004e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777604771328:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298640.866669:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393c00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.867315:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348004e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777604771328:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298640.867325:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298640.867439:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242862300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609206912:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 919us (1020us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298640.867495:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14822 00000100:00100000:3.0:1780298640.872617:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.876331:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.876350:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.878588:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8159 00000100:00100000:3.0:1780298640.878600:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242863b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2398:x1866777609208192:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.878614:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.878624:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e630319cd lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0ecc expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.878652:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf348007800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604772352:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.878694:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393c00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.880829:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609208320 00000100:00100000:3.0:1780298640.880835:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609208320 00000100:00100000:3.0:1780298640.880874:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8160 00000100:00100000:3.0:1780298640.880882:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242860e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609208320:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.880894:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.880905:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e630319d4 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0ed3 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.880925:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298640 v=(42 177) 00010000:00010000:3.0:1780298640.880929:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298640 v=(42 177) 00010000:00010000:3.0:1780298640.880940:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e630319d4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [921600->1409023] (req 921600->929791) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0ed3 expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.880968:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e630319d4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [921600->1409023] (req 921600->929791) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0ed3 expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.880978:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2400, rc 0) 00000100:00100000:3.0:1780298640.881011:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242860e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:29360:x1866777609208320:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 131us (232us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298640.881017:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8160 00000100:00100000:3.0:1780298640.881847:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609208448 00000100:00100000:3.0:1780298640.881853:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609208448 00000100:00100000:3.0:1780298640.881890:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5700 00000100:00100000:3.0:1780298640.881897:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242863100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609208448:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:3.0:1780298640.882025:0:8362:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 5118140 -> 5118140 00002000:00010000:3.0:1780298640.882030:0:8362:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298641 -> 1780298641 00002000:00010000:3.0:1780298640.882032:0:8362:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298641 -> 1780298641 00002000:00010000:3.0:1780298640.882034:0:8362:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298641 -> 1780298641 00002000:00010000:3.0:1780298640.882036:0:8362:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 448 -> 448 00000100:00100000:3.0:1780298640.882070:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242863100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609208448:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 173us (271us total) trans 4294969533 rc 0/0 00000100:00100000:3.0:1780298640.882078:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5700 00000100:00100000:3.0:1780298640.886713:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.887932:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.893618:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.896289:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242860000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604772992:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.896327:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e630319f0 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0eef expref: 15 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.896354:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e630319f0 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0eef expref: 15 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.896364:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9a200, rc 0) 00000100:00100000:3.0:1780298640.896398:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37bb86050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609209344:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1113us (1271us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.896403:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8164 00000100:00100000:3.0:1780298640.898061:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.898111:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.899029:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255491880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604773376:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.899055:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000d106f86/0xaf35132e63031a05 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0f04 expref: 19 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.899075:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d106f86/0xaf35132e63031a05 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0f04 expref: 19 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.899086:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36400, rc 0) 00000100:00100000:3.0:1780298640.899118:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348006a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2397:x1866777609209728:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 877us (1005us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.899149:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8167 00000100:00100000:3.0:1780298640.902407:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.903441:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.907055:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.907083:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.907204:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8172 00000100:00100000:3.0:1780298640.907213:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348004380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609210496:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.907221:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.907230:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e63031a28 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0f20 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.907261:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf348005880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604774144:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.907312:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.908196:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348005880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604774144:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.908215:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e63031a28 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0f20 expref: 15 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.908238:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e63031a28 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0f20 expref: 15 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.908261:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2800, rc 0) 00000100:00100000:3.0:1780298640.908294:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348004380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609210496:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1080us (1235us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.908323:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8172 00000100:00100000:3.0:1780298640.911678:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.914194:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609211264 00000100:00100000:3.0:1780298640.914199:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609211264 00000100:00100000:3.0:1780298640.914236:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609211392 00000100:00100000:3.0:1780298640.914240:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609211392 00000100:00100000:3.0:1780298640.914278:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8177 00000100:00100000:3.0:1780298640.914293:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242860e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2399:x1866777609211392:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.914315:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.914325:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63031a4b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0f4a expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.914359:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf348007480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604774656:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.914396:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.916831:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609211648 00000100:00100000:3.0:1780298640.916836:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609211648 00000100:00100000:3.0:1780298640.919296:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.920898:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.922866:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609212160 00000100:00100000:3.0:1780298640.922872:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609212160 00000100:00100000:3.0:1780298640.922913:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8182 00000100:00100000:3.0:1780298640.922927:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2397:x1866777609212160:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.922942:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.922953:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005c47b04a/0xaf35132e63031a6e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0f6d expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.923008:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf348007480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604775424:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.923052:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.923880:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348007480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604775424:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.923910:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005c47b04a/0xaf35132e63031a6e lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0f6d expref: 15 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.923925:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005c47b04a/0xaf35132e63031a6e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0f6d expref: 15 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.923942:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3400, rc 0) 00000100:00100000:3.0:1780298640.923985:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2397:x1866777609212160:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1065us (1166us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.923990:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8182 00000100:00100000:3.0:1780298640.926596:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609212672 00000100:00100000:3.0:1780298640.926601:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609212672 00010000:00010000:3.0:1780298640.926619:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b000/0xaf35132e63031972 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 24 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0e78 expref: 40 pid: 17578 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.926641:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14825 00000100:00100000:3.0:1780298640.926648:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243845f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+40:2400:x1866777609212672:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298640.926658:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298640.926663:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b000/0xaf35132e63031972 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 25 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0e78 expref: 40 pid: 17578 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.926675:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b000/0xaf35132e63031972 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 25 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0e78 expref: 40 pid: 17578 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.926690:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b000/0xaf35132e63031972 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 25 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0e78 expref: 40 pid: 17578 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.926701:0:7400:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298640.926705:0:7400:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36600/0xaf35132e63031a7c lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 24 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.926729:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36600/0xaf35132e63031a7c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 24 type: IBT flags: 0x40290000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.926737:0:7400:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36600/0xaf35132e63031a7c lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 24 type: IBT flags: 0x40290000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.926755:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298640.926782:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243845f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+39:2400:x1866777609212672:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 133us (253us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298640.926797:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14825 00000100:00100000:3.0:1780298640.927905:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609212800 00000100:00100000:3.0:1780298640.927909:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609212800 00000100:00100000:3.0:1780298640.927938:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8184 00000100:00100000:3.0:1780298640.927945:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243845500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:29360:x1866777609212800:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.927956:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.927966:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e63031a83 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0f7b expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.927985:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e630319d4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [921600->1409023] (req 921600->929791) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0ed3 expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.927995:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d4b625d7/0xaf35132e630319e9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3497984->37052415] (req 3497984->3506175) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0ee8 expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.928003:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 2 to [0x2c0000401:0x11:0x0].0x0 time=1780298640 v=(43 177) 00010000:00010000:3.0:1780298640.928007:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298640 v=(45 177) 00010000:00010000:3.0:1780298640.928011:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e63031a83 lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0f7b expref: 15 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.928037:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d4b625d7/0xaf35132e630319e9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3497984->37052415] (req 3497984->3506175) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0ee8 expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.928048:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d4b625d7/0xaf35132e630319e9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3497984->37052415] (req 3497984->3506175) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0ee8 expref: 9 pid: 8358 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.928059:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf348007480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604775680:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298640.928106:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e630319d4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [921600->1409023] (req 921600->929791) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0ed3 expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.928116:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e630319d4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [921600->1409023] (req 921600->929791) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0ed3 expref: 9 pid: 8356 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.928158:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf348007800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604775808:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298640.929783:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.933509:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.933550:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.934893:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609213824 00000100:00100000:3.0:1780298640.934897:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609213824 00002000:00010000:3.0:1780298640.934915:0:8364:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (1062154->18446744073709551615) 00002000:00100000:3.0:1780298640.934920:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24519ea00 with x1866777609213824 ext(1062154->18446744073709551615) 00002000:00010000:3.0:1780298640.934926:0:8364:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24519ea00. 00000100:00100000:3.0:1780298640.934949:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609213696 00000100:00100000:3.0:1780298640.934953:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609213696 00002000:00010000:3.0:1780298640.934974:0:8364:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:3.0:1780298640.934977:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24519d180 with x1866777609213696 ext(4194304->18446744073709551615) 00002000:00010000:3.0:1780298640.934981:0:8364:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24519d180. 00000100:00100000:3.0:1780298640.934999:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5707 00000100:00100000:3.0:1780298640.935004:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2400:x1866777609213696:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:3.0:1780298640.935220:0:8364:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 5118140 -> 4194304 00002000:00010000:3.0:1780298640.935224:0:8364:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298641 -> 1780298641 00002000:00010000:3.0:1780298640.935225:0:8364:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298641 -> 1780298641 00002000:00010000:3.0:1780298640.935227:0:8364:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298641 -> 1780298641 00002000:00010000:3.0:1780298640.935228:0:8364:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 464 -> 464 00000100:00100000:3.0:1780298640.935258:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609213696:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 253us (358us total) trans 4294969535 rc 0/0 00000100:00100000:3.0:1780298640.935263:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5707 00002000:00010000:3.0:1780298640.935266:0:8364:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:3.0:1780298640.935268:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24519d180 with x1866777609213696 ext(4194304->18446744073709551615) 00002000:00010000:3.0:1780298640.935272:0:8364:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24519d180. 00000100:00100000:3.0:1780298640.938967:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.940311:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609214336 00000100:00100000:3.0:1780298640.940316:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609214336 00000100:00100000:3.0:1780298640.940360:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8187 00000100:00100000:3.0:1780298640.940373:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348318a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609214336:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.940382:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.940394:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c47b04a/0xaf35132e63031ab4 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0f9e expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.940430:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf348004a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604776576:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.940497:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393c00 going to sleep for 11 seconds 00002000:00010000:3.0:1780298640.941338:0:20252:0:(ofd_lvb.c:271:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime: 1780298640 -> 1780298641 00000100:00100000:3.0:1780298640.941350:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348004a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604776576:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.941382:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005c47b04a/0xaf35132e63031ab4 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0f9e expref: 14 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.941396:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c47b04a/0xaf35132e63031ab4 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0f9e expref: 14 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.941409:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3400, rc 0) 00000100:00100000:3.0:1780298640.941434:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348318a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609214336:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1065us (1157us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.941442:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8187 00000100:00100000:3.0:1780298640.947369:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.947425:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.949561:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.949613:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.951637:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609215360 00000100:00100000:3.0:1780298640.951642:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609215360 00000100:00100000:3.0:1780298640.951675:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8193 00000100:00100000:3.0:1780298640.951684:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243844e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2397:x1866777609215360:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.951693:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.951701:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e63031ade lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0fc8 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.951731:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604777472:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.951774:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393c00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.952452:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24310ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604777472:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.952468:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e63031ade lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0fc8 expref: 16 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.952478:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e63031ade lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0fc8 expref: 16 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.952485:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2a00, rc 0) 00000100:00100000:3.0:1780298640.952506:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243844e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2397:x1866777609215360:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 825us (895us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.952511:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8193 00000100:00100000:3.0:1780298640.954317:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243846d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604777600:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.954337:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e63031ae5 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0fcf expref: 16 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.954352:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e63031ae5 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0fcf expref: 16 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.954362:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9a200, rc 0) 00000100:00100000:3.0:1780298640.954395:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348319180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2398:x1866777609215488:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 811us (901us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.954402:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8194 00000100:00100000:3.0:1780298640.955232:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609215616 00000100:00100000:3.0:1780298640.955256:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609215616 00002000:00100000:3.0:1780298640.955287:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (3850240->3854335) req@ffff9cf34831a680 x1866777609215616/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:346/0 lens 488/0 e 0 to 0 dl 1780298651 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298640.955296:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf34831a680 with x1866777609215616 ext(3850240->3854335) 00002000:00010000:3.0:1780298640.955311:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf34831a680 00000100:00100000:3.0:1780298640.955333:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5709 00000100:00100000:3.0:1780298640.955342:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34831a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2398:x1866777609215616:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298640.955483:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34831a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2398:x1866777609215616:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 142us (283us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298640.955491:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5709 00002000:00100000:3.0:1780298640.955498:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (3850240->3854335) req@ffff9cf34831a680 x1866777609215616/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:346/0 lens 488/440 e 0 to 0 dl 1780298651 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298640.955506:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf34831a680 with x1866777609215616 ext(3850240->3854335) 00002000:00010000:3.0:1780298640.955510:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf34831a680 00000100:00100000:3.0:1780298640.960717:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609216128 00000100:00100000:3.0:1780298640.960722:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609216128 00000100:00100000:3.0:1780298640.960749:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8197 00000100:00100000:3.0:1780298640.960755:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2399:x1866777609216128:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.960762:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.960768:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e63031afa lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0fe4 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.960790:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604777984:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.960820:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393c00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.963890:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609216384 00000100:00100000:3.0:1780298640.963901:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609216384 00000100:00100000:3.0:1780298640.963935:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8198 00000100:00100000:3.0:1780298640.963943:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2399:x1866777609216384:12345-192.168.203.25@tcp:16:fsx.0 00000100:00100000:3.0:1780298640.964893:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609216384:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 951us (1034us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298640.964899:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8198 00000100:00100000:3.0:1780298640.965943:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.969414:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609216768 00000100:00100000:3.0:1780298640.969419:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609216768 00000100:00100000:3.0:1780298640.969452:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8201 00000100:00100000:3.0:1780298640.969458:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a82a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2397:x1866777609216768:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.969464:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.969470:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005c47b04a/0xaf35132e63031b0f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d0ff9 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.969489:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604778368:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.969520:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.970120:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24310ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604778368:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.970177:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005c47b04a/0xaf35132e63031b0f lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0ff9 expref: 16 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.970190:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005c47b04a/0xaf35132e63031b0f lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0ff9 expref: 16 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.970199:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3400, rc 0) 00000100:00100000:3.0:1780298640.970232:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a82a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2397:x1866777609216768:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 774us (859us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.970238:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8201 00000100:00100000:3.0:1780298640.970971:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609216896 00000100:00100000:3.0:1780298640.970974:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609216896 00000100:00100000:3.0:1780298640.970995:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8202 00000100:00100000:3.0:1780298640.970999:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:29360:x1866777609216896:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.971005:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.971011:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e63031b16 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1000 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.971023:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63031abb lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [4571136->38125567] (req 4571136->4579327) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0fa5 expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.971030:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298640 v=(46 177) 00010000:00010000:3.0:1780298640.971033:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298640 v=(47 177) 00010000:00010000:3.0:1780298640.971036:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e63031b16 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [4603904->4607999] (req 4603904->4607999) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1000 expref: 16 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.971049:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63031abb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [4571136->38125567] (req 4571136->4579327) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0fa5 expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.971057:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63031abb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [4571136->38125567] (req 4571136->4579327) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0fa5 expref: 11 pid: 9128 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.971063:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604778496:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298640.971096:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.971713:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24310ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604778496:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298640.971731:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e63031b16 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [4603904->4607999] (req 4603904->4607999) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1000 expref: 16 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.971740:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3600, rc 0) 00000100:00100000:3.0:1780298640.971762:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:29360:x1866777609216896:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 762us (818us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298640.971767:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8202 00000100:00100000:3.0:1780298640.973031:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609217152 00000100:00100000:3.0:1780298640.973036:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609217152 00000100:00100000:3.0:1780298640.973074:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5712 00000100:00100000:3.0:1780298640.973081:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a82300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2399:x1866777609217152:12345-192.168.203.25@tcp:22:fsx.0 00000100:00100000:3.0:1780298640.974474:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.976368:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.976485:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.979383:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.980500:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.981448:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604779264:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298640.981471:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e63031b40 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d102a expref: 18 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.981487:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e63031b40 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d102a expref: 18 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298640.981513:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36c00, rc 0) 00000100:00100000:3.0:1780298640.981553:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609218048:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 955us (1052us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298640.981560:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8208 00000100:00100000:3.0:1780298640.982373:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.985154:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.988622:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609218688 00000100:00100000:3.0:1780298640.988629:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609218688 00000100:00100000:3.0:1780298640.988665:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8212 00000100:00100000:3.0:1780298640.988675:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a82a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2399:x1866777609218688:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298640.988685:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298640.988694:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005c47b04a/0xaf35132e63031b5c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1046 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298640.988740:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604779648:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298640.988781:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298640.997215:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.998312:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298640.999412:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.000902:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609219584 00000100:00100000:3.0:1780298641.000907:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609219584 00000100:00100000:3.0:1780298641.000953:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609219712 00000100:00100000:3.0:1780298641.000957:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609219712 00000100:00100000:3.0:1780298641.001010:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8217 00000100:00100000:3.0:1780298641.001019:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609219712:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.001039:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.001050:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000039c5f58c/0xaf35132e63031b7f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1069 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.001085:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bf800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604780416:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.001156:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393c00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.001921:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bf800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604780416:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.001950:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000039c5f58c/0xaf35132e63031b7f lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1069 expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.001970:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000039c5f58c/0xaf35132e63031b7f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1069 expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.001987:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3200, rc 0) 00000100:00100000:3.0:1780298641.002024:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609219712:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1005us (1099us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.002038:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8217 00000100:00100000:3.0:1780298641.003143:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609219968 00000100:00100000:3.0:1780298641.003148:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609219968 00000100:00100000:3.0:1780298641.003194:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8219 00000100:00100000:3.0:1780298641.003206:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a80380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609219968:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.003215:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.003226:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e63031b8d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1077 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.003276:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bf800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604780672:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.003324:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393c00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.004083:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bf800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604780672:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.004108:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e63031b8d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1077 expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.004194:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e63031b8d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1077 expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.004209:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3e00, rc 0) 00000100:00100000:3.0:1780298641.004234:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a80380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609219968:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1033us (1142us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.004241:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8219 00000100:00100000:3.0:1780298641.005572:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609220224 00000100:00100000:3.0:1780298641.005576:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609220224 00000100:00100000:3.0:1780298641.005611:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8221 00000100:00100000:3.0:1780298641.005625:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a80a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609220224:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.005635:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.005646:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c47b04a/0xaf35132e63031b9b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1085 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.005676:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bf800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604780928:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.005722:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393c00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.006532:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bf800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604780928:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.006552:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005c47b04a/0xaf35132e63031b9b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1085 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.006566:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c47b04a/0xaf35132e63031b9b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1085 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.006585:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3400, rc 0) 00000100:00100000:3.0:1780298641.006610:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a80a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609220224:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 989us (1068us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.006616:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8221 00000100:00100000:3.0:1780298641.008022:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609220480 00000100:00100000:3.0:1780298641.008027:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609220480 00000100:00100000:3.0:1780298641.008961:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241a81500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604781184:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.008984:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e63031ba9 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1093 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.009006:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e63031ba9 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1093 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.009031:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9ac00, rc 0) 00000100:00100000:3.0:1780298641.009067:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609220480:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 989us (1068us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.009073:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8223 00000100:00100000:3.0:1780298641.009994:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.011192:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.012035:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241a80700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777604781440:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298641.012047:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298641.012076:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3487c5450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2400:x1866777609220736:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 814us (885us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298641.012083:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14833 00000100:00100000:3.0:1780298641.013882:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609221120 00000100:00100000:3.0:1780298641.013888:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609221120 00000100:00100000:3.0:1780298641.013925:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8226 00000100:00100000:3.0:1780298641.013971:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609221120:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.013980:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.013987:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c47b04a/0xaf35132e63031bbe lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d10a8 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.014013:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604781696:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.014049:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.014845:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604781696:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.014859:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005c47b04a/0xaf35132e63031bbe lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d10a8 expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.014869:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c47b04a/0xaf35132e63031bbe lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d10a8 expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.014875:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3400, rc 0) 00000100:00100000:3.0:1780298641.014896:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609221120:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 965us (1081us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.014906:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8226 00000100:00100000:3.0:1780298641.015984:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609221376 00000100:00100000:3.0:1780298641.015990:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609221376 00000100:00100000:3.0:1780298641.017930:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.019156:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609221632 00000100:00100000:3.0:1780298641.019161:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609221632 00000100:00100000:3.0:1780298641.020701:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609221760 00000100:00100000:3.0:1780298641.020706:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609221760 00000100:00100000:3.0:1780298641.020746:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8230 00000100:00100000:3.0:1780298641.020757:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2397:x1866777609221760:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.020766:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.020777:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005c47b04a/0xaf35132e63031bda lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d10c4 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.020817:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bf100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604782080:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.020876:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.024977:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.026042:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.028246:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.030371:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8237 00000100:00100000:3.0:1780298641.030386:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a80000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2399:x1866777609222912:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.030395:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.030404:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e63031c0b lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d10f5 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.030436:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604782848:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.030486:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.031080:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604782848:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.031098:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e63031c0b lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d10f5 expref: 20 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.031108:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e63031c0b lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d10f5 expref: 20 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.031116:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3200, rc 0) 00000100:00100000:3.0:1780298641.031166:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a80000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2399:x1866777609222912:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 786us (882us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.031177:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8237 00000100:00100000:3.0:1780298641.033726:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609223424 00000100:00100000:3.0:1780298641.033737:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609223424 00010000:00010000:3.0:1780298641.033774:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ba00/0xaf35132e63031aa6 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 25 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0f90 expref: 40 pid: 17578 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.033813:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14834 00000100:00100000:3.0:1780298641.033823:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255491880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+40:2399:x1866777609223424:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298641.033837:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298641.033843:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ba00/0xaf35132e63031aa6 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 26 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0f90 expref: 40 pid: 17578 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.033857:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ba00/0xaf35132e63031aa6 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 26 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0f90 expref: 40 pid: 17578 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.033898:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ba00/0xaf35132e63031aa6 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 26 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d0f90 expref: 40 pid: 17578 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.033914:0:6463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298641.033919:0:6463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244f3aa00/0xaf35132e63031c19 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 25 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.033932:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244f3aa00/0xaf35132e63031c19 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 25 type: IBT flags: 0x40290000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.033939:0:6463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244f3aa00/0xaf35132e63031c19 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 25 type: IBT flags: 0x40290000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.033966:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298641.034021:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255491880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+39:2399:x1866777609223424:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 200us (370us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298641.034030:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14834 00000100:00100000:3.0:1780298641.036326:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.036469:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.038869:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.040242:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.040455:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.040776:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609224448 00000100:00100000:3.0:1780298641.040781:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609224448 00000100:00100000:3.0:1780298641.042262:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.043444:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.043512:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.046099:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.046231:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.063223:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609226880 00000100:00100000:3.0:1780298641.063227:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609226880 00000100:00100000:3.0:1780298641.063260:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8250 00000100:00100000:3.0:1780298641.063265:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348006680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609226880:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.063272:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.063280:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e63031c82 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1157 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.063323:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f744000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604785280:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.063359:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.063966:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f744000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604785280:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.063985:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e63031c82 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1157 expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.063996:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e63031c82 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1157 expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.064003:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3e00, rc 0) 00000100:00100000:3.0:1780298641.064026:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348006680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609226880:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 761us (838us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.064030:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8250 00000100:00100000:3.0:1780298641.064885:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609227008 00000100:00100000:3.0:1780298641.064890:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609227008 00000100:00100000:3.0:1780298641.064920:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5726 00000100:00100000:3.0:1780298641.064925:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548ba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2398:x1866777609227008:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:3.0:1780298641.065074:0:8364:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4963801 -> 4963801 00002000:00010000:3.0:1780298641.065077:0:8364:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298641 -> 1780298641 00002000:00010000:3.0:1780298641.065079:0:8364:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298641 -> 1780298641 00002000:00010000:3.0:1780298641.065080:0:8364:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298641 -> 1780298641 00002000:00010000:3.0:1780298641.065082:0:8364:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 552 -> 576 00000100:00100000:3.0:1780298641.065106:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548ba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2398:x1866777609227008:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 181us (251us total) trans 4294969543 rc 0/0 00000100:00100000:3.0:1780298641.065110:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5726 00000100:00100000:3.0:1780298641.067064:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348004380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604785536:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.067147:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e0a0f245/0xaf35132e63031c90 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1165 expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.067167:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0a0f245/0xaf35132e63031c90 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1165 expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.067179:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b200, rc 0) 00000100:00100000:3.0:1780298641.068277:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8253 00000100:00100000:3.0:1780298641.068345:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348005880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609227392:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.068355:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.068364:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e63031c9e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d116c expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.068399:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f744000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604785792:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.068426:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.069103:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f744000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604785792:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.069169:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e63031c9e lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d116c expref: 15 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.069180:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e63031c9e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d116c expref: 15 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.069188:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3200, rc 0) 00000100:00100000:3.0:1780298641.076959:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548b9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604786560:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.077042:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000097580845/0xaf35132e63031cc1 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1196 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.077053:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000097580845/0xaf35132e63031cc1 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1196 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.077061:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37600, rc 0) 00000100:00100000:3.0:1780298641.077092:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348007480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609228416:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1087us (1178us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.077098:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8259 00000100:00100000:3.0:1780298641.080647:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609228928 00000100:00100000:3.0:1780298641.080653:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609228928 00000100:00100000:3.0:1780298641.080696:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8262 00000100:00100000:3.0:1780298641.080709:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548bb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609228928:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.080718:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.080729:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000039c5f58c/0xaf35132e63031cd6 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d11ab expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.080773:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f744000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604786816:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.080817:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.081589:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f744000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604786816:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.081624:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000039c5f58c/0xaf35132e63031cd6 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d11ab expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.081643:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000039c5f58c/0xaf35132e63031cd6 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d11ab expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.081662:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3200, rc 0) 00000100:00100000:3.0:1780298641.081706:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548bb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609228928:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 997us (1098us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.081718:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8262 00000100:00100000:3.0:1780298641.082718:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609229056 00000100:00100000:3.0:1780298641.082723:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609229056 00000100:00100000:3.0:1780298641.082757:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8263 00000100:00100000:3.0:1780298641.082764:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348006680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2398:x1866777609229056:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.082773:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.082787:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e63031cdd lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d11b2 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.082867:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f744000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604786944:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.082918:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.083738:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f744000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604786944:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.083762:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e63031cdd lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d11b2 expref: 20 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.083779:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e63031cdd lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d11b2 expref: 20 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.083789:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2a00, rc 0) 00000100:00100000:3.0:1780298641.083816:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348006680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2398:x1866777609229056:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1052us (1171us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.083821:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8263 00000100:00100000:3.0:1780298641.085042:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609229440 00000100:00100000:3.0:1780298641.085046:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609229440 00000100:00100000:3.0:1780298641.086878:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609229696 00000100:00100000:3.0:1780298641.086884:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609229696 00000100:00100000:3.0:1780298641.086934:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8268 00000100:00100000:3.0:1780298641.086949:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348004380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609229696:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.086959:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.086970:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e63031d00 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d11d5 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.087007:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f744000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604787584:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.087044:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.087761:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f744000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604787584:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.087784:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e63031d00 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d11d5 expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.087808:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e63031d00 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d11d5 expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.087817:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3e00, rc 0) 00000100:00100000:3.0:1780298641.087843:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348004380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609229696:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 899us (990us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.087848:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8268 00000100:00100000:3.0:1780298641.088816:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609229952 00000100:00100000:3.0:1780298641.088821:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609229952 00000100:00100000:3.0:1780298641.088856:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8269 00000100:00100000:3.0:1780298641.088865:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348005f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609229952:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.088873:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.088881:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000010b6573d/0xaf35132e63031d07 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d11e3 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.088915:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f744000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604787712:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.088966:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.089650:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f744000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604787712:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.089673:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000010b6573d/0xaf35132e63031d07 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d11e3 expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.089689:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000010b6573d/0xaf35132e63031d07 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d11e3 expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.089712:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3a00, rc 0) 00000100:00100000:3.0:1780298641.089744:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348005f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609229952:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 879us (957us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.089751:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8269 00000100:00100000:3.0:1780298641.092160:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609230336 00000100:00100000:3.0:1780298641.092166:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609230336 00000100:00100000:3.0:1780298641.092908:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34831ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604788096:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.092927:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e0a0f245/0xaf35132e63031d1c lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d11f1 expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.092943:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0a0f245/0xaf35132e63031d1c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d11f1 expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.092954:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b200, rc 0) 00000100:00100000:3.0:1780298641.092987:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348318a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609230336:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 743us (886us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.093000:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8272 00000100:00100000:3.0:1780298641.093988:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609230464 00000100:00100000:3.0:1780298641.093994:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609230464 00000100:00100000:3.0:1780298641.095912:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.097080:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609230848 00000100:00100000:3.0:1780298641.097086:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609230848 00010000:00010000:3.0:1780298641.097107:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e7fd0135/0xaf35132e63031c4a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [933888->34488319] (req 933888->937983) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1126 expref: 8 pid: 8359 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.097177:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14843 00000100:00100000:3.0:1780298641.097187:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348319180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609230848:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298641.097197:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298641.097210:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e7fd0135/0xaf35132e63031c4a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [933888->34488319] (req 933888->937983) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1126 expref: 8 pid: 8359 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.097222:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e7fd0135/0xaf35132e63031c4a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [933888->34488319] (req 933888->937983) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1126 expref: 8 pid: 8359 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.097239:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7fd0135/0xaf35132e63031c4a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [933888->34488319] (req 933888->937983) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1126 expref: 8 pid: 8359 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.097249:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(32 130) 00010000:00010000:3.0:1780298641.097252:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(32 130) 00010000:00010000:3.0:1780298641.097261:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000007c073306/0xaf35132e63031d31 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1122304->34676735] (req 1122304->1126399) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1206 expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.097271:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007c073306/0xaf35132e63031d31 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1122304->34676735] (req 1122304->1126399) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1206 expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.097294:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000007c073306/0xaf35132e63031d31 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1122304->34676735] (req 1122304->1126399) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1206 expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.097317:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f745180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6461:1866777604788608:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298641.097360:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.099583:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.099617:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.101422:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.101453:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.103865:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.104963:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.104990:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.105068:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8282 00000100:00100000:3.0:1780298641.105080:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f747480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609232000:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.105090:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.105098:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000010b6573d/0xaf35132e63031d62 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1237 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.105148:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f746a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604789376:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.107054:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8283 00000100:00100000:3.0:1780298641.107068:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348318700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2397:x1866777609232128:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.107080:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.107091:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e63031d69 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d123e expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.107149:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f746a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604789504:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.110231:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.111369:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.111409:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.113354:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.113389:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.118271:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.118481:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.120639:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.121763:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.122934:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609234560 00000100:00100000:3.0:1780298641.122938:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609234560 00010000:00010000:3.0:1780298641.122953:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000007c073306/0xaf35132e63031d31 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1122304->34676735] (req 1122304->1126399) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1206 expref: 12 pid: 19992 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.122970:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14849 00000100:00100000:3.0:1780298641.122975:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609234560:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298641.122981:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298641.122987:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000007c073306/0xaf35132e63031d31 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1122304->34676735] (req 1122304->1126399) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1206 expref: 12 pid: 19992 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.122994:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000007c073306/0xaf35132e63031d31 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1122304->34676735] (req 1122304->1126399) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1206 expref: 12 pid: 19992 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.123003:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007c073306/0xaf35132e63031d31 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1122304->34676735] (req 1122304->1126399) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1206 expref: 12 pid: 19992 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.123011:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(35 130) 00010000:00010000:3.0:1780298641.123013:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(35 130) 00010000:00010000:3.0:1780298641.123016:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000342a3832/0xaf35132e63031da1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d126f expref: 7 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.123032:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000342a3832/0xaf35132e63031da1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d126f expref: 7 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.123048:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000342a3832/0xaf35132e63031da1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d126f expref: 7 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.123055:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e56e300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6461:1866777604791168:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298641.123086:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.123812:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.123935:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.124979:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.126451:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.127552:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609235328 00000100:00100000:3.0:1780298641.127558:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609235328 00000100:00100000:3.0:1780298641.127595:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8291 00000100:00100000:3.0:1780298641.127605:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609235328:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.127614:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.127625:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e63031dc4 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d128b expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.127670:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e56ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604791296:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.127730:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.131350:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.132359:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609235840 00000100:00100000:3.0:1780298641.132372:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609235840 00010000:00010000:3.0:1780298641.132392:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c60b4886/0xaf35132e63031d9a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1268 expref: 19 pid: 12101 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.132421:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14850 00000100:00100000:3.0:1780298641.132430:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b9500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+19:2400:x1866777609235840:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298641.132440:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298641.132450:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c60b4886/0xaf35132e63031d9a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1268 expref: 19 pid: 12101 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.132462:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c60b4886/0xaf35132e63031d9a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1268 expref: 19 pid: 12101 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.132479:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c60b4886/0xaf35132e63031d9a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1268 expref: 19 pid: 12101 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.132490:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(55 177) 00010000:00010000:3.0:1780298641.132493:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(55 177) 00010000:00010000:3.0:1780298641.132501:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e63031de0 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1941504->1945599) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d12a0 expref: 19 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.132509:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e63031de0 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1941504->1945599) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d12a0 expref: 19 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.132532:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e63031de0 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1941504->1945599) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d12a0 expref: 19 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.132555:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e56df80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777604791936:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298641.132594:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.133151:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e56df80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777604791936:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298641.133173:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298641.135623:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609236224 00000100:00100000:3.0:1780298641.135629:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609236224 00000100:00100000:3.0:1780298641.135662:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8297 00000100:00100000:3.0:1780298641.135670:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:29360:x1866777609236224:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.135679:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.135689:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ced6368f/0xaf35132e63031dee lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d12ae expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.135708:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000342a3832/0xaf35132e63031da1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d126f expref: 8 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.135717:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(35 130) 00010000:00010000:3.0:1780298641.135720:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(36 130) 00010000:00010000:3.0:1780298641.135724:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ced6368f/0xaf35132e63031dee lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1028096->1036287] (req 1028096->1036287) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d12ae expref: 12 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.135748:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000342a3832/0xaf35132e63031da1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d126f expref: 8 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.135761:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000342a3832/0xaf35132e63031da1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d126f expref: 8 pid: 9128 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.135783:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e56df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604792064:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298641.135826:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.137588:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.140087:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.141785:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.143072:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.144061:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.145097:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.146286:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609237504 00000100:00100000:3.0:1780298641.146334:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609237504 00000100:00100000:3.0:1780298641.146368:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8301 00000100:00100000:3.0:1780298641.146379:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243845f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+19:2400:x1866777609237504:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.146390:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.146401:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e63031e0a lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d12ca expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.146441:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e56e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604792960:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.146490:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.147235:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e56e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604792960:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.147255:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e63031e0a lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d12ca expref: 20 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.147279:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e63031e0a lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d12ca expref: 20 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.147342:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2800, rc 0) 00000100:00100000:3.0:1780298641.147370:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243845f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+19:2400:x1866777609237504:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 994us (1150us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.147377:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8301 00000100:00100000:3.0:1780298641.151892:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609238272 00000100:00100000:3.0:1780298641.151898:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609238272 00000100:00100000:3.0:1780298641.151923:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8307 00000100:00100000:3.0:1780298641.151929:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f744000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:29360:x1866777609238272:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.151937:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.151945:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000039c5f58c/0xaf35132e63031e34 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d12f4 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.151960:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(36 130) 00010000:00010000:3.0:1780298641.151964:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(36 130) 00010000:00010000:3.0:1780298641.151972:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000039c5f58c/0xaf35132e63031e34 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [729088->1028095] (req 729088->733183) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d12f4 expref: 14 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.151985:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000039c5f58c/0xaf35132e63031e34 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [729088->1028095] (req 729088->733183) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d12f4 expref: 14 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.151994:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3200, rc 0) 00000100:00100000:3.0:1780298641.152018:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f744000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:29360:x1866777609238272:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 89us (150us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298641.152025:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8307 00000100:00100000:3.0:1780298641.155116:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609238784 00000100:00100000:3.0:1780298641.155140:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609238784 00000100:00100000:3.0:1780298641.155202:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8309 00000100:00100000:3.0:1780298641.155213:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2397:x1866777609238784:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.155221:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.155229:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63031e42 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1302 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.155250:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e56e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604793856:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.155280:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.156584:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.159845:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.159861:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.161427:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.161450:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.163193:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609239936 00000100:00100000:3.0:1780298641.163197:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609239936 00000100:00100000:3.0:1780298641.163224:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8317 00000100:00100000:3.0:1780298641.163231:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f747480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2398:x1866777609239936:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.163238:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.163245:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e63031e7a lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d133a expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.163272:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b6a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604794752:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.163325:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.165002:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609240192 00000100:00100000:3.0:1780298641.165007:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609240192 00000100:00100000:3.0:1780298641.165036:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8319 00000100:00100000:3.0:1780298641.165041:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:29360:x1866777609240192:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.165048:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.165055:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b93eb96b/0xaf35132e63031e88 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1348 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.165069:0:19992:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63031e03 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1490944->35045375] (req 1490944->1499135) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d12c3 expref: 20 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.165075:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(55 177) 00010000:00010000:3.0:1780298641.165078:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(56 177) 00010000:00010000:3.0:1780298641.165081:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b93eb96b/0xaf35132e63031e88 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2457600->2461695] (req 2457600->2461695) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1348 expref: 20 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.165095:0:19992:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63031e03 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1490944->35045375] (req 1490944->1499135) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d12c3 expref: 20 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.165105:0:19992:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63031e03 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1490944->35045375] (req 1490944->1499135) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d12c3 expref: 20 pid: 20252 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.165112:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b68000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604795008:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298641.165161:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.165641:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b68000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604795008:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298641.165654:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b93eb96b/0xaf35132e63031e88 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2457600->2461695] (req 2457600->2461695) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1348 expref: 20 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.165670:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3800, rc 0) 00000100:00100000:3.0:1780298641.165691:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:29360:x1866777609240192:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 650us (722us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298641.165696:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8319 00000100:00100000:3.0:1780298641.169354:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609240832 00000100:00100000:3.0:1780298641.169363:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609240832 00000100:00100000:3.0:1780298641.171005:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609240960 00000100:00100000:3.0:1780298641.171009:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609240960 00000100:00100000:3.0:1780298641.171040:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8322 00000100:00100000:3.0:1780298641.171053:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2398:x1866777609240960:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.171062:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.171071:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63031e9d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d135d expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.171088:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63031e9d lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d135d expref: 23 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.171162:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63031e9d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d135d expref: 23 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.171182:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2600, rc 0) 00000100:00100000:3.0:1780298641.171216:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2398:x1866777609240960:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 168us (261us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.171222:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8322 00000100:00100000:3.0:1780298641.171755:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604795392:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.171766:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e7fd0135/0xaf35132e63031ea4 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1364 expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.171775:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7fd0135/0xaf35132e63031ea4 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1364 expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.171782:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9be00, rc 0) 00000100:00100000:3.0:1780298641.171804:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609241088:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 644us (755us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.171808:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8323 00000100:00100000:3.0:1780298641.174593:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.176378:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609241728 00000100:00100000:3.0:1780298641.176382:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609241728 00000100:00100000:3.0:1780298641.176406:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8327 00000100:00100000:3.0:1780298641.176413:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:2397:x1866777609241728:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.176420:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.176426:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63031ec0 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1380 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.176454:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63031ec0 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1380 expref: 23 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.176509:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63031ec0 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1380 expref: 23 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.176535:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3000, rc 0) 00000100:00100000:3.0:1780298641.176561:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:2397:x1866777609241728:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 149us (247us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.176566:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8327 00000100:00100000:3.0:1780298641.178242:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8329 00000100:00100000:3.0:1780298641.178289:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2398:x1866777609242112:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.178308:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.178318:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e63031ece lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1395 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.178341:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b69500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604795776:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.178378:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.178907:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b69500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604795776:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.178920:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e63031ece lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1395 expref: 15 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.178929:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e63031ece lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1395 expref: 15 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.178937:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2e00, rc 0) 00000100:00100000:3.0:1780298641.178958:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2398:x1866777609242112:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 675us (797us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.178962:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8329 00000100:00100000:3.0:1780298641.181985:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609242624 00000100:00100000:3.0:1780298641.181990:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609242624 00000100:00100000:3.0:1780298641.182018:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8333 00000100:00100000:3.0:1780298641.182028:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609242624:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.182036:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.182044:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e63031eea lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d13aa expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.182071:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b69500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604795904:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.182107:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.182711:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b69500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604795904:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.182725:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e63031eea lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d13aa expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.182734:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e63031eea lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d13aa expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.182740:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2600, rc 0) 00000100:00100000:3.0:1780298641.182761:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609242624:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 737us (802us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.182765:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8333 00000100:00100000:3.0:1780298641.183568:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609242752 00000100:00100000:3.0:1780298641.183572:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609242752 00000100:00100000:3.0:1780298641.183592:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609242880 00000100:00100000:3.0:1780298641.183594:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609242880 00000100:00100000:3.0:1780298641.183618:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8335 00000100:00100000:3.0:1780298641.183630:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609242880:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.183643:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.183651:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e63031ef8 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d13b8 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.183685:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b69500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604796032:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.183719:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.184323:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b69500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604796032:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.184336:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e63031ef8 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d13b8 expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.184344:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e63031ef8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d13b8 expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.184350:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3000, rc 0) 00000100:00100000:3.0:1780298641.184372:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609242880:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 748us (818us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.184376:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8335 00000100:00100000:3.0:1780298641.187327:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609243264 00000100:00100000:3.0:1780298641.187332:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609243264 00010000:00010000:3.0:1780298641.187468:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e0a0f245/0xaf35132e63031edc lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2326528->18446744073709551615] (req 2326528->2338815) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d139c expref: 23 pid: 20252 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.188314:0:10277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1780298641.188908:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609243392 00000100:00100000:3.0:1780298641.188911:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609243392 00000100:00100000:3.0:1780298641.188933:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609243520 00000100:00100000:3.0:1780298641.188943:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609243520 00000100:00100000:3.0:1780298641.190666:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.190767:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.192602:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.195460:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.196685:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.196704:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.196787:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8344 00000100:00100000:3.0:1780298641.196797:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b68e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2398:x1866777609244544:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.196804:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.196813:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e63031f37 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d13f7 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.196848:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b68700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604797440:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.196896:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.197613:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b68700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604797440:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.197691:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e63031f37 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d13f7 expref: 15 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.197721:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e63031f37 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d13f7 expref: 15 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.197734:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3000, rc 0) 00000100:00100000:3.0:1780298641.197759:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b68e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2398:x1866777609244544:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 965us (1052us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.197768:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8344 00000100:00100000:3.0:1780298641.198806:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.203297:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609245184 00000100:00100000:3.0:1780298641.203312:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609245184 00000100:00100000:3.0:1780298641.205248:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.207163:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.208459:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.209599:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.211000:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.211239:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.212377:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.212730:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.213589:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.214643:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.214667:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.215754:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609247360 00000100:00100000:3.0:1780298641.215758:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609247360 00002000:00010000:3.0:1780298641.215772:0:19928:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (2350915->18446744073709551615) 00002000:00100000:3.0:1780298641.215776:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf347cd4380 with x1866777609247360 ext(2350915->18446744073709551615) 00002000:00010000:3.0:1780298641.215781:0:19928:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf347cd4380. 00000100:00100000:3.0:1780298641.215801:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5750 00000100:00100000:3.0:1780298641.215814:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+24:2399:x1866777609247360:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:3.0:1780298641.216212:0:19928:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 5122200 -> 2350915 00002000:00010000:3.0:1780298641.216216:0:19928:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298641 -> 1780298641 00002000:00010000:3.0:1780298641.216220:0:19928:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298641 -> 1780298641 00002000:00010000:3.0:1780298641.216222:0:19928:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298641 -> 1780298641 00002000:00010000:3.0:1780298641.216226:0:19928:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 624 -> 528 00000100:00100000:3.0:1780298641.216260:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+25:2399:x1866777609247360:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 448us (530us total) trans 4294969552 rc 0/0 00000100:00100000:3.0:1780298641.216281:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5750 00002000:00010000:3.0:1780298641.216286:0:19928:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (2350915->18446744073709551615) 00002000:00100000:3.0:1780298641.216289:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf347cd4380 with x1866777609247360 ext(2350915->18446744073709551615) 00002000:00010000:3.0:1780298641.216294:0:19928:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf347cd4380. 00000100:00100000:3.0:1780298641.216993:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.218077:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609247744 00000100:00100000:3.0:1780298641.218081:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609247744 00000100:00100000:3.0:1780298641.218104:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609247872 00000100:00100000:3.0:1780298641.218106:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609247872 00000100:00100000:3.0:1780298641.218146:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8355 00000100:00100000:3.0:1780298641.218159:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609247872:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.218166:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298641.220105:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8357 00000100:00100000:3.0:1780298641.220117:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348004700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:2400:x1866777609248000:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298641.220972:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348004a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604799872:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.220992:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63031fa7 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1452 expref: 23 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.221019:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63031fa7 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1452 expref: 23 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.221028:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9ac00, rc 0) 00000100:00100000:3.0:1780298641.221052:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348004700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:2400:x1866777609248000:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 938us (1012us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.221070:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8357 00000100:00100000:3.0:1780298641.225071:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609248512 00000100:00100000:3.0:1780298641.225076:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609248512 00002000:00100000:3.0:1780298641.225099:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (577536->643071) req@ffff9cf348005f80 x1866777609248512/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 504/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298641.225109:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf348005f80 with x1866777609248512 ext(577536->643071) 00002000:00010000:3.0:1780298641.225113:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf348005f80 00000100:00100000:3.0:1780298641.225155:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5752 00000100:00100000:3.0:1780298641.225163:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348005f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+25:2400:x1866777609248512:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298641.225994:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348005f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+25:2400:x1866777609248512:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 832us (949us total) trans 0 rc 61440/61440 00000100:00100000:3.0:1780298641.226002:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5752 00002000:00100000:3.0:1780298641.226009:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (577536->643071) req@ffff9cf348005f80 x1866777609248512/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 504/440 e 0 to 0 dl 1780298652 ref 1 fl Complete:/600/0 rc 61440/61440 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298641.226018:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf348005f80 with x1866777609248512 ext(577536->643071) 00002000:00010000:3.0:1780298641.226022:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf348005f80 00000100:00100000:3.0:1780298641.227446:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609248768 00000100:00100000:3.0:1780298641.227451:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609248768 00000100:00100000:3.0:1780298641.227490:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8358 00000100:00100000:3.0:1780298641.227502:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348007b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609248768:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.227515:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.227532:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e63031fbc lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d146e expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.227604:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b68000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604800128:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.227670:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.228394:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b68000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604800128:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.228418:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e63031fbc lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d146e expref: 13 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.228436:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e63031fbc lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d146e expref: 13 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.228445:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2e00, rc 0) 00000100:00100000:3.0:1780298641.228470:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348007b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609248768:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 972us (1046us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.228477:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8358 00000100:00100000:3.0:1780298641.232518:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609249280 00000100:00100000:3.0:1780298641.232522:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609249280 00000100:00100000:3.0:1780298641.232560:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609249408 00000100:00100000:3.0:1780298641.232564:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609249408 00000100:00100000:3.0:1780298641.234764:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609249664 00000100:00100000:3.0:1780298641.234770:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609249664 00000100:00100000:3.0:1780298641.234862:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8364 00000100:00100000:3.0:1780298641.234872:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609249664:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.234880:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.234888:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e63031fe6 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1491 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.234931:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b68000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604800896:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.234968:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.235680:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b68000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604800896:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.235691:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e63031fe6 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1491 expref: 15 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.235701:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e63031fe6 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1491 expref: 15 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.235707:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3000, rc 0) 00000100:00100000:3.0:1780298641.235727:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609249664:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 860us (1003us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.235732:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8364 00000100:00100000:3.0:1780298641.238531:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609250048 00000100:00100000:3.0:1780298641.238537:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609250048 00010000:00010000:3.0:1780298641.238559:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d78a00/0xaf35132e63031fb5 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 28 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1460 expref: 46 pid: 6477 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.238585:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14865 00000100:00100000:3.0:1780298641.238630:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+46:2399:x1866777609250048:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298641.238641:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298641.238646:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d78a00/0xaf35132e63031fb5 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 29 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1460 expref: 46 pid: 6477 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.238672:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d78a00/0xaf35132e63031fb5 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 29 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1460 expref: 46 pid: 6477 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.238687:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d78a00/0xaf35132e63031fb5 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 29 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1460 expref: 46 pid: 6477 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.238697:0:20253:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298641.238701:0:20253:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b200/0xaf35132e63031fed lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 28 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.238710:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b200/0xaf35132e63031fed lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 28 type: IBT flags: 0x40290000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.238716:0:20253:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b200/0xaf35132e63031fed lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 28 type: IBT flags: 0x40290000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.238737:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298641.238768:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+45:2399:x1866777609250048:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 175us (261us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298641.238776:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14865 00000100:00100000:3.0:1780298641.239604:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609250176 00000100:00100000:3.0:1780298641.239609:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609250176 00000100:00100000:3.0:1780298641.239642:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8365 00000100:00100000:3.0:1780298641.239650:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:29360:x1866777609250176:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.239660:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.239670:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e63031ff4 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1498 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.239702:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000b77c5ae2/0xaf35132e63031f76 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d142f expref: 24 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.239711:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(62 177) 00010000:00010000:3.0:1780298641.239716:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(63 177) 00010000:00010000:3.0:1780298641.239721:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e63031ff4 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1498 expref: 23 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.239741:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000b77c5ae2/0xaf35132e63031f76 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d142f expref: 24 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.239753:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000b77c5ae2/0xaf35132e63031f76 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d142f expref: 24 pid: 12101 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.239765:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b68000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604801152:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298641.239801:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.240391:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b68000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604801152:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298641.240406:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e63031ff4 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1498 expref: 23 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.240418:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2e00, rc 0) 00000100:00100000:3.0:1780298641.240447:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:29360:x1866777609250176:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 798us (869us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298641.240463:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8365 00000100:00100000:3.0:1780298641.243963:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609250816 00000100:00100000:3.0:1780298641.243968:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609250816 00002000:00010000:3.0:1780298641.243986:0:8363:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (2527006->18446744073709551615) 00002000:00100000:3.0:1780298641.243990:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24339c000 with x1866777609250816 ext(2527006->18446744073709551615) 00002000:00010000:3.0:1780298641.243995:0:8363:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24339c000. 00000100:00100000:3.0:1780298641.244026:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5755 00000100:00100000:3.0:1780298641.244039:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609250816:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:3.0:1780298641.246168:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609251072 00000100:00100000:3.0:1780298641.246173:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609251072 00000100:00100000:3.0:1780298641.246282:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.248056:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609251456 00000100:00100000:3.0:1780298641.248061:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609251456 00000100:00100000:3.0:1780298641.248097:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8369 00000100:00100000:3.0:1780298641.248107:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348318a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609251456:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.248119:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.248161:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e6303201e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d14c2 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.248253:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b6aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604802048:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.248288:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.248894:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b6aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604802048:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.248915:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e6303201e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d14c2 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.248959:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e6303201e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d14c2 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.248975:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3000, rc 0) 00000100:00100000:3.0:1780298641.249004:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348318a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609251456:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 899us (1013us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.249010:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8369 00000100:00100000:3.0:1780298641.251321:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609251712 00000100:00100000:3.0:1780298641.251325:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609251712 00000100:00100000:3.0:1780298641.251353:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8371 00000100:00100000:3.0:1780298641.251360:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34831ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+24:2398:x1866777609251712:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.251367:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.251374:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e6303203a lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d14d0 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.251401:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b6aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604802176:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.257440:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609252352 00000100:00100000:3.0:1780298641.257445:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609252352 00000100:00100000:3.0:1780298641.257468:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8374 00000100:00100000:3.0:1780298641.257474:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609252352:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.257481:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.257490:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e6303204f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d14e5 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.257529:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b69500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604802688:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.257563:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.258181:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b69500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604802688:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.258193:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e6303204f lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d14e5 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.258204:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e6303204f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d14e5 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.258211:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3000, rc 0) 00000100:00100000:3.0:1780298641.258232:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609252352:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 759us (820us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.258236:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8374 00000100:00100000:3.0:1780298641.263600:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609252864 00000100:00100000:3.0:1780298641.263604:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609252864 00000100:00100000:3.0:1780298641.263629:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8378 00000100:00100000:3.0:1780298641.263636:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609252864:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.263642:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.263650:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e6303206b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1501 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.263671:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b69500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604803072:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.263703:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.264271:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b69500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604803072:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.264282:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e6303206b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1501 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.264291:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e6303206b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1501 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.264297:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2200, rc 0) 00000100:00100000:3.0:1780298641.264333:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609252864:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 697us (763us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.264339:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8378 00000100:00100000:3.0:1780298641.268791:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609253376 00000100:00100000:3.0:1780298641.268796:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609253376 00002000:00100000:3.0:1780298641.268821:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (1347584->1351679) req@ffff9cf2548bb800 x1866777609253376/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 488/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298641.268830:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2548bb800 with x1866777609253376 ext(1347584->1351679) 00002000:00010000:3.0:1780298641.268833:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2548bb800 00000100:00100000:3.0:1780298641.268848:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5757 00000100:00100000:3.0:1780298641.268854:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548bb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609253376:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298641.268983:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548bb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609253376:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 129us (214us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1780298641.268988:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5757 00002000:00100000:3.0:1780298641.268993:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (1347584->1351679) req@ffff9cf2548bb800 x1866777609253376/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 488/4536 e 0 to 0 dl 1780298652 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298641.269013:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2548bb800 with x1866777609253376 ext(1347584->1351679) 00002000:00010000:3.0:1780298641.269017:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2548bb800 00000100:00100000:3.0:1780298641.274317:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609254144 00000100:00100000:3.0:1780298641.274322:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609254144 00000100:00100000:3.0:1780298641.274348:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8382 00000100:00100000:3.0:1780298641.274355:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+24:29360:x1866777609254144:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.274363:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.274373:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e6303208e lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d151d expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.274390:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000004a398cf8/0xaf35132e63032048 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 995328->999423) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d14de expref: 24 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.274397:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000002656d227/0xaf35132e6303205d lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d14f3 expref: 25 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.274401:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(64 177) 00010000:00010000:3.0:1780298641.274403:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(64 177) 00010000:00010000:3.0:1780298641.274406:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e6303208e lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d151d expref: 25 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.274419:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000002656d227/0xaf35132e6303205d lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d14f3 expref: 25 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.274426:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000002656d227/0xaf35132e6303205d lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d14f3 expref: 25 pid: 8358 timeout: 930 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.274439:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b69500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604803712:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298641.274476:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000004a398cf8/0xaf35132e63032048 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 995328->999423) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d14de expref: 24 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.274483:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000004a398cf8/0xaf35132e63032048 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 995328->999423) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d14de expref: 24 pid: 20252 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.274489:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b68380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604803840:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298641.274504:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.275020:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b69500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604803712:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298641.275026:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b68380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604803840:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298641.275036:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e6303208e lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d151d expref: 25 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.275043:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3000, rc 0) 00000100:00100000:3.0:1780298641.275064:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+25:29360:x1866777609254144:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 710us (777us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298641.275071:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8382 00000100:00100000:3.0:1780298641.275250:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609254272 00000100:00100000:3.0:1780298641.275255:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609254272 00010000:00010000:3.0:1780298641.275272:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000002656d227/0xaf35132e6303205d lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d14f3 expref: 25 pid: 8358 timeout: 930 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.275319:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14871 00000100:00100000:3.0:1780298641.275327:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+25:2400:x1866777609254272:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298641.275335:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298641.275349:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000002656d227/0xaf35132e6303205d lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d14f3 expref: 25 pid: 8358 timeout: 930 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.275359:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000002656d227/0xaf35132e6303205d lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d14f3 expref: 25 pid: 8358 timeout: 930 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.275369:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002656d227/0xaf35132e6303205d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d14f3 expref: 25 pid: 8358 timeout: 930 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.275377:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298641.275397:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+24:2400:x1866777609254272:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 73us (172us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298641.275439:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14871 00000100:00100000:3.0:1780298641.278678:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.278702:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.278906:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b69500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+25:2400:x1866777609254912:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 103us (204us total) trans 4294969554 rc 0/0 00000100:00100000:0.0:1780298641.278912:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5760 00002000:00010000:0.0:1780298641.278915:0:8363:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:0.0:1780298641.278919:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b69500 with x1866777609254912 ext(4194304->18446744073709551615) 00002000:00010000:0.0:1780298641.278924:0:8363:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf242b69500. 00000100:00100000:3.0:1780298641.280014:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.280043:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609255168 00000100:00100000:0.0:1780298641.280046:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609255168 00002000:00100000:0.0:1780298641.280063:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x12:0x0] (2641920->2646015) req@ffff9cf242b68a80 x1866777609255168/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:347/0 lens 488/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.280073:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b68a80 with x1866777609255168 ext(2641920->2646015) 00002000:00010000:0.0:1780298641.280077:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf242b68a80 00000100:00100000:0.0:1780298641.280098:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5761 00000100:00100000:0.0:1780298641.280107:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b68a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2399:x1866777609255168:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298641.280236:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b68a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2399:x1866777609255168:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 131us (220us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298641.280241:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5761 00002000:00100000:0.0:1780298641.280246:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x12:0x0] (2641920->2646015) req@ffff9cf242b68a80 x1866777609255168/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:347/0 lens 488/4536 e 0 to 0 dl 1780298652 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.280252:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b68a80 with x1866777609255168 ext(2641920->2646015) 00002000:00010000:0.0:1780298641.280254:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf242b68a80 00000100:00100000:3.0:1780298641.281429:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609255296 00000100:00100000:3.0:1780298641.281433:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609255296 00002000:00100000:3.0:1780298641.281447:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (2637824->2641919) req@ffff9cf37f744e00 x1866777609255296/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:347/0 lens 488/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298641.281453:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f744e00 with x1866777609255296 ext(2637824->2641919) 00002000:00010000:3.0:1780298641.281459:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f744e00 00000100:00100000:3.0:1780298641.281474:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5762 00000100:00100000:3.0:1780298641.281479:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f744e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609255296:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298641.281548:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f744e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609255296:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 68us (153us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1780298641.281555:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5762 00002000:00100000:3.0:1780298641.281560:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (2637824->2641919) req@ffff9cf37f744e00 x1866777609255296/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:347/0 lens 488/4536 e 0 to 0 dl 1780298652 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298641.281568:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f744e00 with x1866777609255296 ext(2637824->2641919) 00002000:00010000:3.0:1780298641.281571:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f744e00 00000100:00100000:0.0:1780298641.282548:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609255424 00000100:00100000:0.0:1780298641.282553:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609255424 00002000:00100000:0.0:1780298641.282571:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (2637824->2642848) req@ffff9cf37f745c00 x1866777609255424/t0(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:347/0 lens 5520/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.282577:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f745c00 with x1866777609255424 ext(2637824->2642848) 00002000:00010000:0.0:1780298641.282580:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f745c00 00000100:00100000:0.0:1780298641.282595:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5763 00000100:00100000:0.0:1780298641.282601:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609255424:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:0.0:1780298641.282966:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609255424:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 364us (449us total) trans 4294969130 rc 0/0 00000100:00100000:0.0:1780298641.282973:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5763 00002000:00100000:0.0:1780298641.282980:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (2637824->2642848) req@ffff9cf37f745c00 x1866777609255424/t4294969130(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:347/0 lens 5520/448 e 0 to 0 dl 1780298652 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.282989:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f745c00 with x1866777609255424 ext(2637824->2642848) 00002000:00010000:0.0:1780298641.282993:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f745c00 00000100:00100000:0.0:1780298641.283702:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609255552 00000100:00100000:0.0:1780298641.283707:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609255552 00000100:00100000:0.0:1780298641.283736:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8384 00000100:00100000:0.0:1780298641.283744:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243845500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2399:x1866777609255552:12345-192.168.203.25@tcp:16:fsx.0 00000100:00100000:0.0:1780298641.284559:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243845500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609255552:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 816us (884us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298641.284566:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8384 00000100:00100000:0.0:1780298641.286365:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609255808 00000100:00100000:0.0:1780298641.286369:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609255808 00000100:00100000:0.0:1780298641.286396:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8385 00000100:00100000:0.0:1780298641.286403:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+24:2400:x1866777609255808:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.286409:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.286417:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004a398cf8/0xaf35132e630320aa lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1532 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.286458:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24339df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604804480:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.286514:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.287184:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24339df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604804480:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.287207:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004a398cf8/0xaf35132e630320aa lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1532 expref: 25 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1780298641.287214:0:12101:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks: 16 -> 32 00010000:00010000:0.0:1780298641.287219:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004a398cf8/0xaf35132e630320aa lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1532 expref: 25 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.287223:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243845f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604804608:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.287235:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375d78600, rc 0) 00010000:00010000:3.0:1780298641.287245:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003b0cbb9d/0xaf35132e630320b1 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1539 expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.287255:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b0cbb9d/0xaf35132e630320b1 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1539 expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.287263:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b800, rc 0) 00000100:00100000:0.0:1780298641.287266:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+24:2400:x1866777609255808:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 861us (931us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.287275:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8385 00000100:00100000:3.0:1780298641.287284:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f744380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609255936:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 854us (932us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.287291:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8386 00000100:00100000:0.0:1780298641.288159:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609256064 00000100:00100000:0.0:1780298641.288164:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609256064 00000100:00100000:0.0:1780298641.288192:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8387 00000100:00100000:0.0:1780298641.288199:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+24:2398:x1866777609256064:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.288205:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.288211:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002848d803/0xaf35132e630320b8 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1540 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.288231:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24339df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604804736:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.288269:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.288283:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609256192 00000100:00100000:3.0:1780298641.288287:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609256192 00000100:00100000:3.0:1780298641.288332:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8388 00000100:00100000:3.0:1780298641.288341:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243845f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609256192:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.288349:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.288356:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e630320bf lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1547 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.288422:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b68e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604804864:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.288446:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.288803:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24339df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604804736:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.288817:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002848d803/0xaf35132e630320b8 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1540 expref: 25 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.288830:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002848d803/0xaf35132e630320b8 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1540 expref: 25 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.288838:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375d79c00, rc 0) 00000100:00100000:0.0:1780298641.288864:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+24:2398:x1866777609256064:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 666us (754us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.288870:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8387 00000100:00100000:3.0:1780298641.288972:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b68e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604804864:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.288983:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e630320bf lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1547 expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.288991:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e630320bf lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1547 expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.288998:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2200, rc 0) 00000100:00100000:3.0:1780298641.289020:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243845f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609256192:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 681us (799us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.289026:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8388 00000100:00100000:3.0:1780298641.289674:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.290419:0:9976:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433df480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:9976:1866777604804992:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298641.290432:0:9976:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a600/0xaf35132e630320c6 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 30 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.290551:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:0.0:1780298641.290744:0:9976:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a600/0xaf35132e630320c6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 29 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.290754:0:9976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a600/0xaf35132e630320c6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 29 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1780298641.290800:0:9976:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294990958 00000004:00010000:0.0:1780298641.290817:0:9976:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000e2846b97 reply state 00000000f4a04129 transno 4294990959 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a600/0xaf35132e630320c6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 29 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.290825:0:9976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a600/0xaf35132e630320c6 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 29 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.290830:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298641.290832:0:9976:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1780298641.290834:0:9976:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1780298641.290876:0:9976:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b68e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+72:29360:x1866777609256448:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1136us (1201us total) trans 4294990959 rc 0/0 00000100:00100000:0.0:1780298641.290881:0:9976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61861 00000020:00080000:0.0:1780298641.290884:0:9976:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1780298641.291412:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.291571:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609256832 00000100:00100000:3.0:1780298641.291575:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609256832 00002000:00010000:3.0:1780298641.291587:0:19949:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00002000:00100000:3.0:1780298641.291590:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f747100 with x1866777609256832 ext(0->18446744073709551615) 00002000:00010000:3.0:1780298641.291600:0:19949:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f747100. 00000100:00100000:3.0:1780298641.291620:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5765 00000100:00100000:3.0:1780298641.291627:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f747100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609256832:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:3.0:1780298641.291812:0:19949:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 2642849 -> 0 00002000:00010000:3.0:1780298641.291815:0:19949:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298641 -> 1780298641 00002000:00010000:3.0:1780298641.291816:0:19949:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298641 -> 1780298641 00002000:00010000:3.0:1780298641.291818:0:19949:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298641 -> 1780298641 00002000:00010000:3.0:1780298641.291819:0:19949:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 32 -> 0 00000100:00100000:3.0:1780298641.291841:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f747100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609256832:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 215us (297us total) trans 4294969131 rc 0/0 00000100:00100000:3.0:1780298641.291845:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5765 00002000:00010000:3.0:1780298641.291847:0:19949:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00002000:00100000:3.0:1780298641.291848:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f747100 with x1866777609256832 ext(0->18446744073709551615) 00002000:00010000:3.0:1780298641.291851:0:19949:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f747100. 00000100:00100000:0.0:1780298641.292497:0:17578:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609256960 00000100:00100000:0.0:1780298641.292501:0:17578:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609256960 00000100:00100000:3.0:1780298641.292534:0:6476:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61862 00000100:00100000:3.0:1780298641.292542:0:6476:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+72:29360:x1866777609256960:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.292550:0:6476:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.292557:0:6476:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2000/0xaf35132e630320cd lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 30 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d154e expref: -99 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.292591:0:6476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2e00/0xaf35132e630320d4 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 31 type: IBT flags: 0x40000000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.292597:0:6476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2e00/0xaf35132e630320d4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 31 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.292602:0:6476:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2e00/0xaf35132e630320d4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 31 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1780298641.292618:0:6476:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 3821755 00000004:00200000:3.0:1780298641.292621:0:6476:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1780298641.292628:0:6476:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2e00/0xaf35132e630320d4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 31 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.292641:0:6476:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2000/0xaf35132e630320cd lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 31 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d154e expref: 74 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.292649:0:6476:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2e00/0xaf35132e630320d4 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 30 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d154e expref: 73 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.292654:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298641.292656:0:6476:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1780298641.292658:0:6476:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2e00, rc 0) 00000100:00100000:3.0:1780298641.292679:0:6476:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+73:29360:x1866777609256960:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 140us (210us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298641.292682:0:6476:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61862 00000100:00100000:0.0:1780298641.294484:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.294504:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.294524:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609257216 00000100:00100000:3.0:1780298641.294530:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609257216 00000100:00100000:3.0:1780298641.294570:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8389 00000100:00100000:0.0:1780298641.294579:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8390 00000100:00100000:3.0:1780298641.294583:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+24:2398:x1866777609257216:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298641.294592:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609257344:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.294593:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.294601:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.294602:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e630320e9 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d155c expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.294609:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046a75400/0xaf35132e630320f0 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1563 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.294634:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b69c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604805120:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.294639:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24339ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604805248:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.294682:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1300 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.294683:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.295374:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24339ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604805248:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.295385:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b69c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604805120:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.295390:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000046a75400/0xaf35132e630320f0 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1563 expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.295403:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e630320e9 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d155c expref: 25 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.295403:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046a75400/0xaf35132e630320f0 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1563 expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.295425:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375d79800, rc 0) 00010000:00010000:3.0:1780298641.295429:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e630320e9 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d155c expref: 25 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.295437:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2600, rc 0) 00000100:00100000:0.0:1780298641.295450:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609257344:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 862us (945us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.295455:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8390 00000100:00100000:3.0:1780298641.295465:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+24:2398:x1866777609257216:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 885us (978us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.295473:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8389 00000100:00100000:0.0:1780298641.296382:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609257600 00000100:00100000:0.0:1780298641.296386:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609257600 00000100:00100000:3.0:1780298641.296423:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8392 00000100:00100000:3.0:1780298641.296435:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609257600:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.296444:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.296454:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e630320fe lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1571 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.296490:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b69c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604805504:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.296516:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.297105:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b69c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604805504:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.297120:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e630320fe lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1571 expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.297162:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e630320fe lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1571 expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.297171:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2200, rc 0) 00000100:00100000:3.0:1780298641.297200:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609257600:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 767us (843us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.297214:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8392 00000100:00100000:0.0:1780298641.298078:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609257856 00000100:00100000:0.0:1780298641.298082:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609257856 00000100:00100000:0.0:1780298641.298113:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8394 00000100:00100000:0.0:1780298641.298138:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609257856:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.298145:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.298153:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004a398cf8/0xaf35132e6303210c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d157f expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.298184:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24339ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604805760:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.298210:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1300 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.298794:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24339ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604805760:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.298812:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004a398cf8/0xaf35132e6303210c lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d157f expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.298837:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004a398cf8/0xaf35132e6303210c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d157f expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.298845:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375d78600, rc 0) 00000100:00100000:3.0:1780298641.298867:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609257856:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 745us (814us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.298879:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8394 00000100:00100000:3.0:1780298641.299632:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.299672:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609257984 00000100:00100000:0.0:1780298641.299677:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609257984 00000100:00100000:3.0:1780298641.299702:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.299717:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609258112 00000100:00100000:0.0:1780298641.299721:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609258112 00000100:00100000:3.0:1780298641.300450:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604805888:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.300473:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e63032113 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1586 expref: 25 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.300525:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e63032113 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1586 expref: 25 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.300538:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37000, rc 0) 00000100:00100000:3.0:1780298641.300569:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b69c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+24:2399:x1866777609257984:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 829us (936us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.300575:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8395 00000100:00100000:0.0:1780298641.301729:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.302743:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.302784:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609258368 00000100:00100000:3.0:1780298641.302788:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609258368 00002000:00100000:3.0:1780298641.302804:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (2662400->2666495) req@ffff9cf24339e300 x1866777609258368/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:347/0 lens 488/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298641.302811:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24339e300 with x1866777609258368 ext(2662400->2666495) 00002000:00010000:3.0:1780298641.302814:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24339e300 00000100:00100000:3.0:1780298641.302828:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5767 00000100:00100000:3.0:1780298641.302832:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+26:2397:x1866777609258368:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298641.302902:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+26:2397:x1866777609258368:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 69us (159us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1780298641.302907:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5767 00002000:00100000:3.0:1780298641.302912:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (2662400->2666495) req@ffff9cf24339e300 x1866777609258368/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:347/0 lens 488/4536 e 0 to 0 dl 1780298652 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298641.302918:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24339e300 with x1866777609258368 ext(2662400->2666495) 00002000:00010000:3.0:1780298641.302921:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24339e300 00000100:00100000:0.0:1780298641.303806:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609258496 00000100:00100000:0.0:1780298641.303810:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609258496 00002000:00100000:0.0:1780298641.303826:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (2637824->2703359) req@ffff9cf34372e300 x1866777609258496/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:347/0 lens 504/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.303833:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf34372e300 with x1866777609258496 ext(2637824->2703359) 00002000:00010000:0.0:1780298641.303835:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf34372e300 00000100:00100000:0.0:1780298641.303850:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5768 00000100:00100000:0.0:1780298641.303855:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+26:2397:x1866777609258496:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298641.305393:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609258624 00000100:00100000:3.0:1780298641.305397:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609258624 00000100:00100000:3.0:1780298641.305423:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609258752 00000100:00100000:3.0:1780298641.305426:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609258752 00000100:00100000:0.0:1780298641.305460:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8398 00000100:00100000:0.0:1780298641.305473:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609258752:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.305481:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.305489:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046a75400/0xaf35132e63032128 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d159b expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.305517:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24339d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604806272:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.305578:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1300 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.306256:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24339d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604806272:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.306279:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000046a75400/0xaf35132e63032128 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d159b expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.306293:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046a75400/0xaf35132e63032128 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d159b expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.306318:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375d79800, rc 0) 00000100:00100000:0.0:1780298641.306339:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609258752:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 870us (939us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.306344:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8398 00000100:00100000:3.0:1780298641.307145:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609258880 00000100:00100000:3.0:1780298641.307149:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609258880 00000100:00100000:0.0:1780298641.307207:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.308942:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609259136 00000100:00100000:3.0:1780298641.308946:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609259136 00000100:00100000:3.0:1780298641.308967:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8401 00000100:00100000:3.0:1780298641.308977:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+24:29360:x1866777609259136:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.308984:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298641.309877:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.310699:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372ce00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777604806784:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298641.310714:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298641.310735:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+25:2399:x1866777609259264:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 779us (858us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298641.310740:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14876 00000100:00100000:0.0:1780298641.315798:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.327199:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.327239:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609259520 00000100:00100000:0.0:1780298641.327250:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609259520 00000100:00100000:0.0:1780298641.327295:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8402 00000100:00100000:0.0:1780298641.327315:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b69180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609259520:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.327330:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.327347:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b3c3e92c/0xaf35132e63032144 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d15b7 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.327396:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24339dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604806912:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.327469:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1300 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.328268:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24339dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604806912:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.328295:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b3c3e92c/0xaf35132e63032144 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d15b7 expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.328330:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b3c3e92c/0xaf35132e63032144 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d15b7 expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.328343:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375d78400, rc 0) 00000100:00100000:3.0:1780298641.328376:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b69180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609259520:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1064us (1184us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.328384:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8402 00000100:00100000:0.0:1780298641.329345:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609259648 00000100:00100000:0.0:1780298641.329351:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609259648 00000100:00100000:0.0:1780298641.329389:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8403 00000100:00100000:0.0:1780298641.329397:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609259648:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.329407:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.329417:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000091ca8d05/0xaf35132e6303214b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d15be expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.329453:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24339fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604807040:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.329497:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1300 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.330122:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24339fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604807040:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.330151:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000091ca8d05/0xaf35132e6303214b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d15be expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.330161:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000091ca8d05/0xaf35132e6303214b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d15be expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.330168:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375d79a00, rc 0) 00000100:00100000:0.0:1780298641.330190:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609259648:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 795us (896us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.330194:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8403 00000100:00100000:0.0:1780298641.332999:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609259904 00000100:00100000:0.0:1780298641.333005:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609259904 00000100:00100000:0.0:1780298641.333036:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8405 00000100:00100000:0.0:1780298641.333044:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609259904:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.333053:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.333063:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001308b102/0xaf35132e63032159 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d15cc expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.333094:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24339fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604807296:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.333138:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1300 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.337386:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609260288 00000100:00100000:3.0:1780298641.337391:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609260288 00010000:00010000:3.0:1780298641.337413:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a200/0xaf35132e630320e2 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 31 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1555 expref: 46 pid: 6477 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.337461:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14877 00000100:00100000:3.0:1780298641.337470:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433de300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+46:2399:x1866777609260288:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298641.337481:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298641.337486:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a200/0xaf35132e630320e2 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 32 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1555 expref: 46 pid: 6477 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.337498:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a200/0xaf35132e630320e2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 32 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1555 expref: 46 pid: 6477 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.337512:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a200/0xaf35132e630320e2 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 32 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1555 expref: 46 pid: 6477 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.337523:0:20253:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298641.337527:0:20253:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37a00/0xaf35132e63032160 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 31 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.337537:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37a00/0xaf35132e63032160 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 31 type: IBT flags: 0x40290000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.337544:0:20253:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37a00/0xaf35132e63032160 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 31 type: IBT flags: 0x40290000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.337564:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298641.337601:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433de300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+45:2399:x1866777609260288:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 130us (302us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298641.337614:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14877 00000100:00100000:3.0:1780298641.338506:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609260416 00000100:00100000:3.0:1780298641.338516:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609260416 00000100:00100000:3.0:1780298641.338546:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8406 00000100:00100000:3.0:1780298641.338553:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+25:29360:x1866777609260416:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.338563:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.338574:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e63032167 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d15d3 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.338594:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d71a6aba/0xaf35132e6303213d lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 925696->933887) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d15b0 expref: 24 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.338602:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(65 177) 00010000:00010000:3.0:1780298641.338606:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(65 177) 00010000:00010000:3.0:1780298641.338610:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e63032167 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d15d3 expref: 26 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.338629:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d71a6aba/0xaf35132e6303213d lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 925696->933887) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d15b0 expref: 24 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.338640:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d71a6aba/0xaf35132e6303213d lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 925696->933887) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d15b0 expref: 24 pid: 8356 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.338648:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b68000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604807552:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298641.338687:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.339312:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b68000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604807552:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298641.339324:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e63032167 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d15d3 expref: 26 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.339341:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2a00, rc 0) 00000100:00100000:3.0:1780298641.339363:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+26:29360:x1866777609260416:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 812us (883us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298641.339367:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8406 00000100:00100000:0.0:1780298641.342398:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.343473:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433dc700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777604807680:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298641.343485:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298641.343513:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+24:2400:x1866777609260544:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1009us (1119us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298641.343522:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14878 00000100:00100000:0.0:1780298641.343749:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.343778:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609260672 00000100:00100000:3.0:1780298641.343784:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609260672 00002000:00010000:3.0:1780298641.343803:0:19949:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:3.0:1780298641.343806:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24339df80 with x1866777609260672 ext(4194304->18446744073709551615) 00002000:00010000:3.0:1780298641.343813:0:19949:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24339df80. 00000100:00100000:0.0:1780298641.343859:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5770 00000100:00100000:0.0:1780298641.343870:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+26:2398:x1866777609260672:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:3.0:1780298641.343967:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609260800 00000100:00100000:3.0:1780298641.343970:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609260800 00002000:00010000:3.0:1780298641.343982:0:19943:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (1433734->18446744073709551615) 00002000:00100000:3.0:1780298641.343985:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2433dce00 with x1866777609260800 ext(1433734->18446744073709551615) 00002000:00010000:3.0:1780298641.343990:0:19943:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2433dce00. 00000100:00100000:3.0:1780298641.344008:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5771 00000100:00100000:3.0:1780298641.344014:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609260800:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:0.0:1780298641.344022:0:19928:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 3821755 -> 4194304 00002000:00010000:0.0:1780298641.344027:0:19928:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298641 -> 1780298641 00002000:00010000:0.0:1780298641.344030:0:19928:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298641 -> 1780298641 00002000:00010000:0.0:1780298641.344033:0:19928:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298641 -> 1780298641 00002000:00010000:0.0:1780298641.344035:0:19928:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 528 -> 528 00000100:00100000:0.0:1780298641.344064:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+27:2398:x1866777609260672:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 198us (317us total) trans 4294969556 rc 0/0 00000100:00100000:0.0:1780298641.344070:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5770 00002000:00010000:0.0:1780298641.344073:0:19928:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:0.0:1780298641.344075:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24339df80 with x1866777609260672 ext(4194304->18446744073709551615) 00002000:00010000:0.0:1780298641.344079:0:19928:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24339df80. 00002000:00010000:3.0:1780298641.344163:0:19943:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 0 -> 1433734 00002000:00010000:3.0:1780298641.344167:0:19943:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298641 -> 1780298641 00002000:00010000:3.0:1780298641.344170:0:19943:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298641 -> 1780298641 00002000:00010000:3.0:1780298641.344173:0:19943:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298641 -> 1780298641 00002000:00010000:3.0:1780298641.344215:0:19943:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:3.0:1780298641.345523:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609260928 00000100:00100000:3.0:1780298641.345528:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609260928 00002000:00100000:3.0:1780298641.345556:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (1433600->1437695) req@ffff9cf2433df800 x1866777609260928/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:347/0 lens 488/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298641.345580:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2433df800 with x1866777609260928 ext(1433600->1437695) 00002000:00010000:3.0:1780298641.345584:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2433df800 00000100:00100000:3.0:1780298641.345607:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5772 00000100:00100000:3.0:1780298641.345616:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609260928:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298641.347092:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.348673:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609261184 00000100:00100000:0.0:1780298641.348679:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609261184 00002000:00100000:0.0:1780298641.348718:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (1429504->1433733) req@ffff9cf34372ce00 x1866777609261184/t0(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:347/0 lens 4720/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.348729:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf34372ce00 with x1866777609261184 ext(1429504->1433733) 00002000:00010000:0.0:1780298641.348733:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf34372ce00 00000100:00100000:0.0:1780298641.348765:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5774 00000100:00100000:0.0:1780298641.348774:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609261184:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:0.0:1780298641.349271:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609261184:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 498us (638us total) trans 4294969133 rc 0/0 00000100:00100000:0.0:1780298641.349283:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5774 00002000:00100000:0.0:1780298641.349290:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (1429504->1433733) req@ffff9cf34372ce00 x1866777609261184/t4294969133(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:347/0 lens 4720/448 e 0 to 0 dl 1780298652 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.349311:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf34372ce00 with x1866777609261184 ext(1429504->1433733) 00002000:00010000:0.0:1780298641.349315:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf34372ce00 00000100:00100000:0.0:1780298641.350200:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609261312 00000100:00100000:0.0:1780298641.350205:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609261312 00000100:00100000:0.0:1780298641.350236:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8407 00000100:00100000:0.0:1780298641.350243:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2398:x1866777609261312:12345-192.168.203.25@tcp:16:fsx.0 00000100:00100000:0.0:1780298641.351000:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609261312:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 755us (845us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298641.351009:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8407 00000100:00100000:0.0:1780298641.351927:0:9976:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609261440 00000100:00100000:0.0:1780298641.351932:0:9976:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609261440 00000100:00100000:0.0:1780298641.351960:0:9976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61865 00000100:00100000:0.0:1780298641.351965:0:9976:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+75:29360:x1866777609261440:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.351979:0:9976:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.351997:0:9976:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d78800/0xaf35132e6303216e lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 31 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d15da expref: -99 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.352065:0:9976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d79800/0xaf35132e63032175 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 32 type: IBT flags: 0x40000000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.352078:0:9976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d79800/0xaf35132e63032175 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 32 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.352090:0:9976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d79800/0xaf35132e63032175 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 32 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1780298641.352108:0:9976:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 5628038 00000004:00200000:0.0:1780298641.352111:0:9976:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1780298641.352149:0:9976:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d79800/0xaf35132e63032175 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 32 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.352169:0:9976:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d78800/0xaf35132e6303216e lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 32 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d15da expref: 77 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.352183:0:9976:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d79800/0xaf35132e63032175 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 31 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d15da expref: 76 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.352189:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298641.352192:0:9976:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1780298641.352195:0:9976:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375d79800, rc 0) 00000100:00100000:0.0:1780298641.352217:0:9976:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433dd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+76:29360:x1866777609261440:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 252us (327us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298641.352221:0:9976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61865 00000100:00100000:0.0:1780298641.355222:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.356402:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609261824 00000100:00100000:0.0:1780298641.356408:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609261824 00002000:00100000:0.0:1780298641.356436:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (450560->454655) req@ffff9cf34372dc00 x1866777609261824/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 488/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.356447:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf34372dc00 with x1866777609261824 ext(450560->454655) 00002000:00010000:0.0:1780298641.356452:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf34372dc00 00000100:00100000:0.0:1780298641.356471:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5775 00000100:00100000:0.0:1780298641.356477:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609261824:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298641.356601:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609261824:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 125us (233us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298641.356606:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5775 00002000:00100000:0.0:1780298641.356610:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (450560->454655) req@ffff9cf34372dc00 x1866777609261824/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 488/4536 e 0 to 0 dl 1780298652 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.356615:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf34372dc00 with x1866777609261824 ext(450560->454655) 00002000:00010000:0.0:1780298641.356617:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf34372dc00 00000100:00100000:0.0:1780298641.357335:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.358323:0:6476:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609262080 00000100:00100000:3.0:1780298641.358328:0:6476:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609262080 00000100:00100000:3.0:1780298641.358355:0:6476:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61866 00000100:00100000:3.0:1780298641.358361:0:6476:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+45:29360:x1866777609262080:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.358369:0:6476:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.358384:0:6476:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2000/0xaf35132e63032183 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 32 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d15e8 expref: -99 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.358451:0:6476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2600/0xaf35132e6303218a lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 33 type: IBT flags: 0x40000000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.358466:0:6476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2600/0xaf35132e6303218a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 33 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.358477:0:6476:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2600/0xaf35132e6303218a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 33 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1780298641.358501:0:6476:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 5628038 00000004:00200000:3.0:1780298641.358504:0:6476:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1780298641.358518:0:6476:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2600/0xaf35132e6303218a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 33 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.358536:0:6476:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2000/0xaf35132e63032183 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 33 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d15e8 expref: 47 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.358554:0:6476:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2600/0xaf35132e6303218a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 32 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d15e8 expref: 46 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.358560:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298641.358562:0:6476:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1780298641.358565:0:6476:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2600, rc 0) 00000100:00100000:3.0:1780298641.358591:0:6476:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+46:29360:x1866777609262080:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 231us (312us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298641.358595:0:6476:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61866 00000100:00100000:3.0:1780298641.361226:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609262336 00000100:00100000:3.0:1780298641.361230:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609262336 00000100:00100000:3.0:1780298641.361253:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8410 00000100:00100000:3.0:1780298641.361258:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+24:2397:x1866777609262336:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.361265:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.361272:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000077813dbd/0xaf35132e63032198 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d15f6 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.361295:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b68000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604808192:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.361340:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.362104:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b68000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604808192:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.362116:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000077813dbd/0xaf35132e63032198 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d15f6 expref: 25 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.362151:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000077813dbd/0xaf35132e63032198 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d15f6 expref: 25 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.362170:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2200, rc 0) 00000100:00100000:3.0:1780298641.362193:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+24:2397:x1866777609262336:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 935us (1008us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.362198:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8410 00000100:00100000:0.0:1780298641.363146:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.367426:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.369958:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000001:00080000:0.0:1780298641.372572:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969557 is committed 00000001:00080000:0.0:1780298641.372592:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969552 is committed 00000100:00100000:3.0:1780298641.374831:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.376215:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.376251:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609263488 00000100:00100000:0.0:1780298641.376256:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609263488 00010000:00010000:0.0:1780298641.376274:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d79800/0xaf35132e63032175 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 32 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d15da expref: 76 pid: 9976 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.376295:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14880 00000100:00100000:0.0:1780298641.376311:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b6a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+76:2400:x1866777609263488:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298641.376320:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298641.376324:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d79800/0xaf35132e63032175 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 33 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d15da expref: 76 pid: 9976 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.376337:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d79800/0xaf35132e63032175 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 33 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d15da expref: 76 pid: 9976 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.376348:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d79800/0xaf35132e63032175 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 33 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d15da expref: 76 pid: 9976 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.376360:0:7400:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298641.376364:0:7400:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be200/0xaf35132e630321a6 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 32 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.376374:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be200/0xaf35132e630321a6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 32 type: IBT flags: 0x40290000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.376383:0:7400:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be200/0xaf35132e630321a6 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 32 type: IBT flags: 0x40290000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.376403:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298641.376429:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b6a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+75:2400:x1866777609263488:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 127us (214us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298641.376436:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14880 00000100:00100000:0.0:1780298641.377393:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609263616 00000100:00100000:0.0:1780298641.377397:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609263616 00000100:00100000:0.0:1780298641.377420:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8413 00000100:00100000:0.0:1780298641.377426:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:29360:x1866777609263616:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.377434:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.377444:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000091ca8d05/0xaf35132e630321ad lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1604 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.377465:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e63032167 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d15d3 expref: 8 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.377471:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(65 177) 00010000:00010000:0.0:1780298641.377474:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(66 177) 00010000:00010000:0.0:1780298641.377477:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000091ca8d05/0xaf35132e630321ad lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1604 expref: 7 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.377490:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e63032167 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d15d3 expref: 8 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.377497:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e63032167 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d15d3 expref: 8 pid: 12101 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.377503:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e56fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604808448:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298641.377533:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1300 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.378482:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609263744 00000100:00100000:0.0:1780298641.378485:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609263744 00010000:00010000:0.0:1780298641.378498:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e63032167 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d15d3 expref: 9 pid: 12101 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.378520:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14881 00000100:00100000:0.0:1780298641.378524:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f3b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609263744:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298641.378530:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:0.0:1780298641.378542:0:8374:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 528 -> 552 00010000:00010000:0.0:1780298641.378546:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e63032167 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d15d3 expref: 9 pid: 12101 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.378553:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e63032167 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d15d3 expref: 9 pid: 12101 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.378565:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e63032167 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d15d3 expref: 9 pid: 12101 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.378572:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(66 177) 00010000:00010000:0.0:1780298641.378575:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(66 177) 00010000:00010000:0.0:1780298641.378579:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000091ca8d05/0xaf35132e630321ad lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1604 expref: 6 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.378585:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000091ca8d05/0xaf35132e630321ad lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1604 expref: 6 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.378599:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000091ca8d05/0xaf35132e630321ad lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1604 expref: 6 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.378606:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e56ea00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777604808576:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298641.378634:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1300 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.379517:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.379550:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609263872 00000100:00100000:0.0:1780298641.379554:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609263872 00000100:00100000:0.0:1780298641.379586:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8414 00000100:00100000:0.0:1780298641.379596:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b69c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609263872:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.379606:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.379617:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001308b102/0xaf35132e630321b4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d160b expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.379638:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000ad6f98a8/0xaf35132e6303217c lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 450560->458751) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d15e1 expref: 8 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.379645:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003c8dab73/0xaf35132e6303219f lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d15fd expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.379649:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(42 130) 00010000:00010000:0.0:1780298641.379651:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(42 130) 00010000:00010000:0.0:1780298641.379653:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001308b102/0xaf35132e630321b4 lrc: 5/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d160b expref: 8 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.379668:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003c8dab73/0xaf35132e6303219f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d15fd expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.379676:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003c8dab73/0xaf35132e6303219f lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d15fd expref: 9 pid: 8359 timeout: 930 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.379683:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e56f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604808704:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298641.379717:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000ad6f98a8/0xaf35132e6303217c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 450560->458751) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d15e1 expref: 8 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.379728:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000ad6f98a8/0xaf35132e6303217c lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 450560->458751) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d15e1 expref: 8 pid: 19992 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.379735:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e56ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604808832:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298641.379748:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.380348:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e56f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604808704:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298641.380580:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.380604:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e56ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604808832:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298641.380625:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001308b102/0xaf35132e630321b4 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d160b expref: 8 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.380639:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375d79600, rc 0) 00000100:00100000:0.0:1780298641.380669:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b69c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:29360:x1866777609263872:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1076us (1154us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298641.380676:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8414 00000100:00100000:0.0:1780298641.381794:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.381929:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.390337:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.390367:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.390443:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609264640 00000100:00100000:3.0:1780298641.390448:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609264640 00002000:00100000:3.0:1780298641.390472:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (4190208->4194303) req@ffff9cf36e56d500 x1866777609264640/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 488/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298641.390487:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e56d500 with x1866777609264640 ext(4190208->4194303) 00002000:00010000:3.0:1780298641.390491:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf36e56d500 00000100:00100000:3.0:1780298641.390511:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5784 00000100:00100000:3.0:1780298641.390519:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609264640:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298641.390677:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609264640:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 158us (309us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1780298641.390685:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5784 00002000:00100000:3.0:1780298641.390691:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (4190208->4194303) req@ffff9cf36e56d500 x1866777609264640/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 488/4536 e 0 to 0 dl 1780298652 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298641.390700:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e56d500 with x1866777609264640 ext(4190208->4194303) 00002000:00010000:3.0:1780298641.390703:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf36e56d500 00000100:00100000:0.0:1780298641.404926:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.405214:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.405255:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609264896 00000100:00100000:3.0:1780298641.405261:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609264896 00002000:00100000:3.0:1780298641.405287:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x12:0x0] (0->4194303) req@ffff9cf36e56f800 x1866777609264896/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 488/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298641.405313:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e56f800 with x1866777609264896 ext(0->4194303) 00002000:00010000:3.0:1780298641.405320:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf36e56f800 00000100:00100000:3.0:1780298641.405347:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5786 00000100:00100000:3.0:1780298641.405360:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609264896:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298641.406283:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.407538:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000001:00080000:0.0:1780298641.408286:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969559 is committed 00000100:00100000:0.0:1780298641.409109:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.411373:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.412710:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.412747:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.412789:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609265792 00000100:00100000:3.0:1780298641.412795:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609265792 00002000:00010000:3.0:1780298641.412814:0:19928:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (3701709->18446744073709551615) 00002000:00100000:3.0:1780298641.412821:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf241a82d80 with x1866777609265792 ext(3701709->18446744073709551615) 00002000:00010000:3.0:1780298641.412828:0:19928:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf241a82d80. 00000100:00100000:3.0:1780298641.412855:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5789 00000100:00100000:3.0:1780298641.413080:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a82d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609265792:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:3.0:1780298641.413344:0:19928:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4194304 -> 3701709 00002000:00010000:3.0:1780298641.413347:0:19928:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298641 -> 1780298641 00002000:00010000:3.0:1780298641.413349:0:19928:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298641 -> 1780298641 00002000:00010000:3.0:1780298641.413350:0:19928:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298641 -> 1780298641 00002000:00010000:3.0:1780298641.413352:0:19928:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 16 -> 16 00000100:00100000:3.0:1780298641.413375:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a82d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609265792:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 514us (629us total) trans 4294969135 rc 0/0 00000100:00100000:3.0:1780298641.413379:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5789 00002000:00010000:3.0:1780298641.413381:0:19928:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (3701709->18446744073709551615) 00002000:00100000:3.0:1780298641.413383:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf241a82d80 with x1866777609265792 ext(3701709->18446744073709551615) 00002000:00010000:3.0:1780298641.413394:0:19928:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf241a82d80. 00000100:00100000:3.0:1780298641.418046:0:6476:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609265920 00000100:00100000:3.0:1780298641.418053:0:6476:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609265920 00000100:00100000:3.0:1780298641.418087:0:6476:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61870 00000100:00100000:3.0:1780298641.418092:0:6476:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433df800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+75:29360:x1866777609265920:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.418105:0:6476:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.418120:0:6476:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2200/0xaf35132e630321d0 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 33 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1619 expref: -99 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.419249:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.419352:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.420997:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.421020:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.421041:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609266304 00000100:00100000:0.0:1780298641.421047:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609266304 00000100:00100000:0.0:1780298641.421085:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8418 00000100:00100000:0.0:1780298641.421098:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b6aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609266304:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.421109:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.421120:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e630321ec lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d162e expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.421170:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241a80e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604809472:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.421201:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.421813:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241a80e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604809472:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.421834:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e630321ec lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d162e expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.421847:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e630321ec lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d162e expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.421854:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9ac00, rc 0) 00000100:00100000:3.0:1780298641.421876:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b6aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609266304:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 782us (879us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.421881:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8418 00000100:00100000:3.0:1780298641.422764:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.422781:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.422862:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8421 00000100:00100000:0.0:1780298641.422873:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b69f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609266688:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.422880:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.422888:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7fd0135/0xaf35132e63032201 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1643 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.422911:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241a81180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604809728:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.422934:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1300 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.423559:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241a81180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604809728:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.423578:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e7fd0135/0xaf35132e63032201 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1643 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.423600:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7fd0135/0xaf35132e63032201 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1643 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.423611:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9be00, rc 0) 00000100:00100000:0.0:1780298641.423640:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b69f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609266688:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 771us (857us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.423648:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8421 00000100:00100000:3.0:1780298641.424214:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.427685:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.428690:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.429631:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.430694:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.430770:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609267328 00000100:00100000:0.0:1780298641.430776:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609267328 00000100:00100000:0.0:1780298641.430808:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8423 00000100:00100000:0.0:1780298641.430815:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b68a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609267328:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.430821:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.430828:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000019780e8d/0xaf35132e6303221d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1658 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.430850:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241a82300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604810112:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.430883:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1300 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.431451:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241a82300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604810112:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.431463:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000019780e8d/0xaf35132e6303221d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1658 expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.431473:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000019780e8d/0xaf35132e6303221d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1658 expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.431480:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9a200, rc 0) 00000100:00100000:0.0:1780298641.431501:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b68a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609267328:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 688us (808us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.431506:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8423 00000100:00100000:3.0:1780298641.432181:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.433734:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.433771:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609267584 00000100:00100000:0.0:1780298641.433777:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609267584 00000100:00100000:0.0:1780298641.433809:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8425 00000100:00100000:0.0:1780298641.433817:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f755180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:29360:x1866777609267584:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.433825:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.433835:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d71a6aba/0xaf35132e6303222b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1666 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.433851:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(43 130) 00010000:00010000:0.0:1780298641.433856:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(43 130) 00010000:00010000:0.0:1780298641.433862:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d71a6aba/0xaf35132e6303222b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1765376->2293759] (req 1765376->1769471) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1666 expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.433874:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d71a6aba/0xaf35132e6303222b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1765376->2293759] (req 1765376->1769471) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1666 expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.433883:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9bc00, rc 0) 00000100:00100000:0.0:1780298641.433912:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f755180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:29360:x1866777609267584:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 96us (178us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298641.433920:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8425 00000100:00100000:3.0:1780298641.434521:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.434556:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609267712 00000100:00100000:0.0:1780298641.434561:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609267712 00000100:00100000:0.0:1780298641.434597:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5791 00000100:00100000:0.0:1780298641.434605:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f755500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609267712:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298641.434698:0:19950:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 3701709 -> 3701709 00002000:00010000:0.0:1780298641.434702:0:19950:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298641 -> 1780298641 00002000:00010000:0.0:1780298641.434704:0:19950:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298641 -> 1780298641 00002000:00010000:0.0:1780298641.434707:0:19950:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298641 -> 1780298641 00002000:00010000:0.0:1780298641.434709:0:19950:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 40 -> 40 00000100:00100000:0.0:1780298641.434738:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f755500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609267712:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 134us (216us total) trans 4294969137 rc 0/0 00000100:00100000:0.0:1780298641.434746:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5791 00000100:00100000:3.0:1780298641.435474:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.437014:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.437054:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609267968 00000100:00100000:0.0:1780298641.437059:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609267968 00000100:00100000:0.0:1780298641.437090:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8427 00000100:00100000:0.0:1780298641.437099:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f755f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609267968:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.437108:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.437117:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b0cbb9d/0xaf35132e63032239 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1674 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.437174:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241a82300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604810496:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.437214:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1300 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.437931:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241a82300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604810496:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.437948:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003b0cbb9d/0xaf35132e63032239 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1674 expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.437969:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b0cbb9d/0xaf35132e63032239 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1674 expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.437979:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b800, rc 0) 00000100:00100000:0.0:1780298641.438010:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f755f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609267968:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 913us (995us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.438019:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8427 00000100:00100000:3.0:1780298641.439154:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.440246:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.441354:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.446480:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.446534:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609268608 00000100:00100000:0.0:1780298641.446544:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609268608 00010000:00010000:0.0:1780298641.446587:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000091ca8d05/0xaf35132e630321ad lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1604 expref: 8 pid: 8356 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.446629:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14886 00000100:00100000:0.0:1780298641.446640:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f757800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609268608:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298641.446656:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298641.446675:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000091ca8d05/0xaf35132e630321ad lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1604 expref: 8 pid: 8356 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.446689:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000091ca8d05/0xaf35132e630321ad lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1604 expref: 8 pid: 8356 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.446712:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000091ca8d05/0xaf35132e630321ad lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1604 expref: 8 pid: 8356 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.446735:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(67 177) 00010000:00010000:0.0:1780298641.446739:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(67 177) 00010000:00010000:0.0:1780298641.446746:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000408198ca/0xaf35132e63032247 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d167b expref: 8 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.446757:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000408198ca/0xaf35132e63032247 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d167b expref: 8 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.446790:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000408198ca/0xaf35132e63032247 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d167b expref: 8 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.446805:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241a82300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777604810880:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298641.446874:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1300 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.447518:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241a82300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777604810880:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298641.447531:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298641.447557:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f757800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609268608:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 921us (1083us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298641.447564:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14886 00000100:00100000:0.0:1780298641.447672:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.448969:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.448987:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.449017:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609268864 00000100:00100000:3.0:1780298641.449022:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609268864 00010000:00010000:3.0:1780298641.449040:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000c189d300/0xaf35132e63032208 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [2293760->35848191] (req 2293760->2306047) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d164a expref: 12 pid: 20252 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.449082:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14888 00000100:00100000:3.0:1780298641.449094:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a82680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609268864:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298641.449103:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298641.449114:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000c189d300/0xaf35132e63032208 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [2293760->35848191] (req 2293760->2306047) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d164a expref: 12 pid: 20252 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.449151:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000c189d300/0xaf35132e63032208 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [2293760->35848191] (req 2293760->2306047) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d164a expref: 12 pid: 20252 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.449196:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c189d300/0xaf35132e63032208 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [2293760->35848191] (req 2293760->2306047) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d164a expref: 12 pid: 20252 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.449207:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(45 130) 00010000:00010000:3.0:1780298641.449213:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(45 130) 00010000:00010000:3.0:1780298641.449221:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000710bbd58/0xaf35132e6303224e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1682 expref: 11 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.449227:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000710bbd58/0xaf35132e6303224e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1682 expref: 11 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.449244:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000710bbd58/0xaf35132e6303224e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1682 expref: 11 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.449252:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f755c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777604811264:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298641.449283:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.450170:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.450195:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.450218:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609269248 00000100:00100000:3.0:1780298641.450223:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609269248 00002000:00010000:3.0:1780298641.450244:0:16972:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4194304->18446744073709551615) 00002000:00100000:3.0:1780298641.450249:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf241a81f80 with x1866777609269248 ext(4194304->18446744073709551615) 00002000:00010000:3.0:1780298641.450256:0:16972:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf241a81f80. 00000100:00100000:3.0:1780298641.450276:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5792 00000100:00100000:3.0:1780298641.450284:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a81f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609269248:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:3.0:1780298641.450416:0:16972:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 3701709 -> 4194304 00002000:00010000:3.0:1780298641.450423:0:16972:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298641 -> 1780298641 00002000:00010000:3.0:1780298641.450425:0:16972:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298641 -> 1780298641 00002000:00010000:3.0:1780298641.450428:0:16972:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298641 -> 1780298641 00002000:00010000:3.0:1780298641.450430:0:16972:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 40 -> 40 00000100:00100000:3.0:1780298641.450457:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a81f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2399:x1866777609269248:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 173us (286us total) trans 4294969138 rc 0/0 00000100:00100000:3.0:1780298641.450465:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5792 00002000:00010000:3.0:1780298641.450468:0:16972:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4194304->18446744073709551615) 00002000:00100000:3.0:1780298641.450471:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf241a81f80 with x1866777609269248 ext(4194304->18446744073709551615) 00002000:00010000:3.0:1780298641.450477:0:16972:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf241a81f80. 00000100:00100000:0.0:1780298641.451719:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.453240:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.454407:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000001:00080000:0.0:1780298641.455097:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969562 is committed 00000001:00080000:0.0:1780298641.455107:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969560 is committed 00000100:00100000:0.0:1780298641.455910:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.457613:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.458834:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.458884:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609270016 00000100:00100000:3.0:1780298641.458888:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609270016 00002000:00100000:3.0:1780298641.458905:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (5373952->5439487) req@ffff9cf241a80e00 x1866777609270016/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:347/0 lens 504/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298641.458912:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf241a80e00 with x1866777609270016 ext(5373952->5439487) 00002000:00010000:3.0:1780298641.458915:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf241a80e00 00000100:00100000:3.0:1780298641.458930:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5797 00000100:00100000:3.0:1780298641.458935:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a80e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609270016:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298641.459747:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a80e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609270016:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 811us (912us total) trans 0 rc 61440/61440 00000100:00100000:3.0:1780298641.459752:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5797 00002000:00100000:3.0:1780298641.459758:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (5373952->5439487) req@ffff9cf241a80e00 x1866777609270016/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:347/0 lens 504/440 e 0 to 0 dl 1780298652 ref 1 fl Complete:/600/0 rc 61440/61440 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298641.459764:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf241a80e00 with x1866777609270016 ext(5373952->5439487) 00002000:00010000:3.0:1780298641.459771:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf241a80e00 00000100:00100000:3.0:1780298641.460989:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.462175:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.463738:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.466068:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.467333:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.467376:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609270656 00000100:00100000:3.0:1780298641.467382:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609270656 00000100:00100000:3.0:1780298641.467413:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8431 00000100:00100000:3.0:1780298641.467421:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609270656:12345-192.168.203.25@tcp:16:fsx.0 00000001:00080000:0.0:1780298641.468153:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969563 is committed 00000100:00100000:3.0:1780298641.468218:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609270656:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 797us (885us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298641.468224:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8431 00000100:00100000:0.0:1780298641.468994:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.470267:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.470290:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.470326:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609270912 00000100:00100000:3.0:1780298641.470386:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609270912 00000100:00100000:3.0:1780298641.470417:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8433 00000100:00100000:3.0:1780298641.470425:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609270912:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.470432:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.470444:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b93eb96b/0xaf35132e63032278 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1697 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.470480:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f754a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604811520:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.470552:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00002000:00010000:3.0:1780298641.471386:0:8359:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks: 552 -> 584 00000100:00100000:3.0:1780298641.471398:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f754a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604811520:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.471419:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b93eb96b/0xaf35132e63032278 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1697 expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.471436:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b93eb96b/0xaf35132e63032278 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1697 expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.471462:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3800, rc 0) 00000100:00100000:3.0:1780298641.471499:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609270912:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1074us (1232us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.471509:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8433 00000100:00100000:0.0:1780298641.472504:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.472609:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.472675:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609271296 00000100:00100000:3.0:1780298641.472680:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609271296 00000100:00100000:3.0:1780298641.472717:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8435 00000100:00100000:3.0:1780298641.472726:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609271296:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.472734:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.472743:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e63032286 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d16ac expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.472770:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f754a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604811776:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.472803:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.474503:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.475501:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.475601:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14889 00000100:00100000:0.0:1780298641.475610:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b1f050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2399:x1866777609271552:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298641.475621:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298641.475634:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000710bbd58/0xaf35132e6303224e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1682 expref: 12 pid: 8356 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.475646:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000710bbd58/0xaf35132e6303224e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1682 expref: 12 pid: 8356 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.475663:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000710bbd58/0xaf35132e6303224e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1682 expref: 12 pid: 8356 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.475680:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(46 130) 00010000:00010000:0.0:1780298641.475683:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(46 130) 00010000:00010000:0.0:1780298641.475690:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000a10929f0/0xaf35132e6303228d lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1728512->1736703) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d16b3 expref: 9 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.475700:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a10929f0/0xaf35132e6303228d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1728512->1736703) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d16b3 expref: 9 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.475722:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000a10929f0/0xaf35132e6303228d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1728512->1736703) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d16b3 expref: 9 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.475733:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519ce00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:7400:1866777604812032:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298641.475771:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1300 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.476723:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.476760:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609271680 00000100:00100000:0.0:1780298641.476764:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609271680 00002000:00100000:0.0:1780298641.476786:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x12:0x0] (1728512->1736703) req@ffff9cf37f756680 x1866777609271680/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 488/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.476792:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f756680 with x1866777609271680 ext(1728512->1736703) 00002000:00010000:0.0:1780298641.476796:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f756680 00000100:00100000:0.0:1780298641.476816:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5802 00000100:00100000:0.0:1780298641.476821:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f756680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609271680:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298641.476940:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f756680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609271680:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 118us (218us total) trans 0 rc 8192/8192 00000100:00100000:0.0:1780298641.476947:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5802 00002000:00100000:0.0:1780298641.476952:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x12:0x0] (1728512->1736703) req@ffff9cf37f756680 x1866777609271680/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 488/8632 e 0 to 0 dl 1780298652 ref 1 fl Complete:/600/0 rc 8192/8192 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.476958:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f756680 with x1866777609271680 ext(1728512->1736703) 00002000:00010000:0.0:1780298641.476960:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f756680 00000100:00100000:3.0:1780298641.477821:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.479657:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.481609:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.483388:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.485246:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.486440:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.486500:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609272448 00000100:00100000:0.0:1780298641.486505:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609272448 00010000:00010000:0.0:1780298641.486528:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000a10929f0/0xaf35132e6303228d lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1728512->1736703) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d16b3 expref: 10 pid: 9128 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.486558:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14890 00000100:00100000:0.0:1780298641.486565:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f754a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609272448:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298641.486575:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298641.486601:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000a10929f0/0xaf35132e6303228d lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1728512->1736703) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d16b3 expref: 10 pid: 9128 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.486614:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000a10929f0/0xaf35132e6303228d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1728512->1736703) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d16b3 expref: 10 pid: 9128 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.486630:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a10929f0/0xaf35132e6303228d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1728512->1736703) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d16b3 expref: 10 pid: 9128 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.486640:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(46 130) 00010000:00010000:0.0:1780298641.486644:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(46 130) 00010000:00010000:0.0:1780298641.486652:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e630322b0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [843776->34398207] (req 843776->851967) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d16d6 expref: 11 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.486661:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e630322b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [843776->34398207] (req 843776->851967) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d16d6 expref: 11 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.486686:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e630322b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [843776->34398207] (req 843776->851967) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d16d6 expref: 11 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.486696:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519d880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:20253:1866777604812800:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298641.486734:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.487739:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519d880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:20253:1866777604812800:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298641.487754:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298641.487762:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.487789:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f754a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609272448:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1223us (1347us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298641.487797:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14890 00000100:00100000:3.0:1780298641.489164:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.493208:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.493247:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609272960 00000100:00100000:0.0:1780298641.493253:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609272960 00000100:00100000:0.0:1780298641.493294:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5804 00000100:00100000:0.0:1780298641.493311:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609272960:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298641.493538:0:19949:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 5562137 -> 5562137 00002000:00010000:0.0:1780298641.493542:0:19949:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298641 -> 1780298641 00002000:00010000:0.0:1780298641.493544:0:19949:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298641 -> 1780298641 00002000:00010000:0.0:1780298641.493545:0:19949:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298641 -> 1780298641 00002000:00010000:0.0:1780298641.493547:0:19949:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 584 -> 592 00000100:00100000:0.0:1780298641.493576:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609272960:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 267us (371us total) trans 4294969564 rc 0/0 00000100:00100000:0.0:1780298641.493584:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5804 00000100:00100000:0.0:1780298641.494689:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.495732:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433dc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604813312:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.495749:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000a323d4b/0xaf35132e630322cc lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d16f2 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.495760:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000a323d4b/0xaf35132e630322cc lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d16f2 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.495767:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf2445bf000, rc 0) 00000100:00100000:3.0:1780298641.495788:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e60c050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609273216:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 987us (1103us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.495793:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8445 00000100:00100000:0.0:1780298641.497105:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.497177:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.497234:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8446 00000100:00100000:3.0:1780298641.497246:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609273344:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298641.499489:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609273600 00000100:00100000:3.0:1780298641.499495:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609273600 00000100:00100000:3.0:1780298641.499528:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8448 00000100:00100000:3.0:1780298641.499535:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:29360:x1866777609273600:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.499545:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.499555:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e630322e1 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1707 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.499581:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000408198ca/0xaf35132e63032247 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d167b expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.499607:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(67 177) 00010000:00010000:3.0:1780298641.499611:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(68 177) 00010000:00010000:3.0:1780298641.499617:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e630322e1 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2920448->2932735] (req 2920448->2932735) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1707 expref: 7 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.499641:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000408198ca/0xaf35132e63032247 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d167b expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.499654:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000408198ca/0xaf35132e63032247 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d167b expref: 9 pid: 19992 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.499666:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604813696:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298641.499710:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.500377:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8b1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604813696:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298641.500394:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e630322e1 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2920448->2932735] (req 2920448->2932735) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1707 expref: 7 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.500406:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3200, rc 0) 00000100:00100000:3.0:1780298641.500429:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609273600:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 896us (978us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298641.500433:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8448 00000100:00100000:0.0:1780298641.500828:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.501908:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433df480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777604813824:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298641.501920:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298641.501952:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609273728:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1025us (1124us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298641.501970:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14891 00000100:00100000:0.0:1780298641.502180:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.503534:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.504981:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609274112 00000100:00100000:3.0:1780298641.504993:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609274112 00000100:00100000:3.0:1780298641.505031:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8449 00000100:00100000:3.0:1780298641.505042:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609274112:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.505054:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.505071:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b93eb96b/0xaf35132e630322e8 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d170e expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.505154:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604814080:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.507219:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.507347:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.507388:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609274496 00000100:00100000:3.0:1780298641.507391:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609274496 00000100:00100000:0.0:1780298641.507445:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8452 00000100:00100000:0.0:1780298641.507457:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609274496:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.507466:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.507478:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d71a6aba/0xaf35132e630322fd lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1723 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.507506:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604814336:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.507541:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00002000:00010000:0.0:1780298641.508246:0:20252:0:(ofd_lvb.c:265:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime: 1780298641 -> 1780298642 00002000:00010000:0.0:1780298641.508250:0:20252:0:(ofd_lvb.c:271:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime: 1780298641 -> 1780298642 00000100:00100000:0.0:1780298641.508262:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604814336:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.508284:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d71a6aba/0xaf35132e630322fd lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1723 expref: 14 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.508309:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d71a6aba/0xaf35132e630322fd lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1723 expref: 14 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.508319:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9bc00, rc 0) 00000100:00100000:0.0:1780298641.508351:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609274496:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 898us (1005us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.508358:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8452 00000100:00100000:0.0:1780298641.509367:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.509400:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609274752 00000100:00100000:3.0:1780298641.509404:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609274752 00000100:00100000:3.0:1780298641.509481:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8454 00000100:00100000:3.0:1780298641.509489:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609274752:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.509532:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.509539:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e6303230b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1731 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.509560:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b3100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604814592:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.509585:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.510342:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8b3100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604814592:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.510352:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e6303230b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1731 expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.510361:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e6303230b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1731 expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.510384:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3c00, rc 0) 00000100:00100000:3.0:1780298641.510413:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609274752:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 926us (1047us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.510420:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8454 00000100:00100000:0.0:1780298641.511260:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.511278:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.511291:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609274880 00000100:00100000:3.0:1780298641.511306:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609274880 00000100:00100000:3.0:1780298641.511334:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8455 00000100:00100000:3.0:1780298641.511342:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609274880:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.511348:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298641.511357:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8456 00010000:00010000:3.0:1780298641.511358:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e63032312 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1738 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.511368:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609275008:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.511375:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.511385:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e63032319 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d173f expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.511419:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604814848:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.511475:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.512236:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604814848:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.512261:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e63032319 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d173f expref: 14 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.512286:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e63032319 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d173f expref: 14 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.512307:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b000, rc 0) 00000100:00100000:3.0:1780298641.512334:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609275008:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 966us (1052us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.512342:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8456 00000100:00100000:0.0:1780298641.514598:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609275392 00000100:00100000:0.0:1780298641.514603:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609275392 00010000:00010000:0.0:1780298641.514624:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf400/0xaf35132e6303225c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 35 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1689 expref: 79 pid: 9976 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.514658:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14892 00000100:00100000:0.0:1780298641.514669:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f3800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+79:2400:x1866777609275392:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298641.514679:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298641.514685:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf400/0xaf35132e6303225c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 36 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1689 expref: 79 pid: 9976 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.514696:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf400/0xaf35132e6303225c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 36 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1689 expref: 79 pid: 9976 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.514711:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf400/0xaf35132e6303225c lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 36 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1689 expref: 79 pid: 9976 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.514722:0:6461:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298641.514725:0:6461:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf600/0xaf35132e63032320 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 35 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.514735:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf600/0xaf35132e63032320 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 35 type: IBT flags: 0x40290000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.514742:0:6461:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf600/0xaf35132e63032320 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 35 type: IBT flags: 0x40290000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.514762:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298641.514792:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f3800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+78:2400:x1866777609275392:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 126us (233us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298641.514799:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14892 00000100:00100000:3.0:1780298641.515776:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609275520 00000100:00100000:3.0:1780298641.515780:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609275520 00000100:00100000:3.0:1780298641.515812:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8457 00000100:00100000:3.0:1780298641.515821:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609275520:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.515831:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.515843:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e63032327 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1746 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.515862:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e630322e1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2920448->36474879] (req 2920448->2932735) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1707 expref: 8 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.515871:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(68 177) 00010000:00010000:3.0:1780298641.515875:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(69 177) 00010000:00010000:3.0:1780298641.515879:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e63032327 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1746 expref: 8 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.515899:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e630322e1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2920448->36474879] (req 2920448->2932735) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1707 expref: 8 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.515911:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e630322e1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2920448->36474879] (req 2920448->2932735) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1707 expref: 8 pid: 8359 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.515921:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604815104:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298641.515968:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.517647:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8b2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604815104:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298641.517687:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609275648 00000100:00100000:0.0:1780298641.517694:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609275648 00010000:00010000:3.0:1780298641.517706:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e63032327 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1746 expref: 9 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00100000:0.0:1780298641.517727:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (2920448->2932735) req@ffff9cf2558f0e00 x1866777609275648/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 12776/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00010000:00010000:3.0:1780298641.517728:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2400, rc 0) 00002000:00100000:0.0:1780298641.517739:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2558f0e00 with x1866777609275648 ext(2920448->2932735) 00010000:00010000:0.0:1780298641.517747:0:19928:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e630322e1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2920448->36474879] (req 2920448->2932735) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1707 expref: 9 pid: 8359 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.517764:0:19928:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e630322e1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2920448->36474879] (req 2920448->2932735) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1707 expref: 9 pid: 8359 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.517765:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:29360:x1866777609275520:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1943us (2027us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298641.517780:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8457 00002000:00010000:0.0:1780298641.517782:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf2558f0e00 00000100:00100000:0.0:1780298641.517814:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 175 00000100:00100000:0.0:1780298641.517824:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609275648:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:0.0:1780298641.518383:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609275648:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 560us (738us total) trans 4294969565 rc 0/0 00000100:00100000:0.0:1780298641.518391:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 175 00002000:00100000:0.0:1780298641.518399:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (2920448->2932735) req@ffff9cf2558f0e00 x1866777609275648/t4294969565(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 12776/448 e 0 to 0 dl 1780298652 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.518408:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2558f0e00 with x1866777609275648 ext(2920448->2932735) 00010000:00010000:0.0:1780298641.518414:0:19928:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e630322e1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2920448->36474879] (req 2920448->2932735) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1707 expref: 10 pid: 8359 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.518429:0:19928:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e630322e1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2920448->36474879] (req 2920448->2932735) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1707 expref: 10 pid: 8359 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1780298641.518443:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf2558f0e00 00000100:00100000:3.0:1780298641.520595:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.521892:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.521922:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609276032 00000100:00100000:3.0:1780298641.521927:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609276032 00010000:00010000:3.0:1780298641.521945:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e630322b0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [843776->34398207] (req 843776->851967) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d16d6 expref: 13 pid: 19992 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.521966:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14894 00000100:00100000:3.0:1780298641.521971:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609276032:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298641.521980:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298641.521991:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e630322b0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [843776->34398207] (req 843776->851967) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d16d6 expref: 13 pid: 19992 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.521999:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e630322b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [843776->34398207] (req 843776->851967) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d16d6 expref: 13 pid: 19992 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.522011:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e630322b0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [843776->34398207] (req 843776->851967) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d16d6 expref: 13 pid: 19992 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.522020:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(47 130) 00010000:00010000:3.0:1780298641.522023:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(47 130) 00010000:00010000:3.0:1780298641.522035:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000408198ca/0xaf35132e6303232e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d174d expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.522041:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000408198ca/0xaf35132e6303232e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d174d expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.522059:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000408198ca/0xaf35132e6303232e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d174d expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.522066:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b2680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777604815488:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298641.522101:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.522912:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8b2680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777604815488:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298641.522974:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:3.0:1780298641.523015:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298641.523051:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609276032:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1077us (1157us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298641.523066:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14894 00000100:00100000:3.0:1780298641.526179:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.528170:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.530002:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.532092:0:9976:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609276928 00000100:00100000:0.0:1780298641.532097:0:9976:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609276928 00000100:00100000:0.0:1780298641.532150:0:9976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61877 00000100:00100000:0.0:1780298641.532160:0:9976:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+55:29360:x1866777609276928:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:0.0:1780298641.532198:0:9976:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298641.532219:0:9976:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36a00/0xaf35132e6303233c lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 36 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1754 expref: 55 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.532231:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298641.532235:0:9976:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1780298641.532367:0:9976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ba00/0xaf35132e63032343 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 35 type: IBT flags: 0x40000000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.532383:0:9976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ba00/0xaf35132e63032343 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 35 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.532391:0:9976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ba00/0xaf35132e63032343 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 35 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1780298641.532421:0:9976:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294990970 00000004:00010000:0.0:1780298641.532435:0:9976:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000a2bd8815 reply state 000000002f276bab transno 4294990971 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ba00/0xaf35132e63032343 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 35 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.532447:0:9976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ba00/0xaf35132e63032343 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 35 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.532452:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298641.532453:0:9976:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1780298641.532455:0:9976:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1780298641.532507:0:9976:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433dd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+57:29360:x1866777609276928:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 349us (464us total) trans 4294990971 rc 0/0 00000100:00100000:0.0:1780298641.532514:0:9976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61877 00000020:00080000:0.0:1780298641.532518:0:9976:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1780298641.533352:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.533373:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.533399:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609277056 00000100:00100000:3.0:1780298641.533405:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609277056 00002000:00010000:3.0:1780298641.533431:0:19928:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:3.0:1780298641.533438:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24519c380 with x1866777609277056 ext(4194304->18446744073709551615) 00002000:00010000:3.0:1780298641.533447:0:19928:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24519c380. 00000100:00100000:3.0:1780298641.533485:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5812 00000100:00100000:3.0:1780298641.533498:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609277056:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:3.0:1780298641.533585:0:19928:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:3.0:1780298641.533595:0:19928:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:3.0:1780298641.533598:0:19928:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298641 -> 1780298641 00002000:00010000:3.0:1780298641.533600:0:19928:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:3.0:1780298641.533603:0:19928:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 624 -> 624 00000100:00100000:3.0:1780298641.533638:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2399:x1866777609277056:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 142us (284us total) trans 4294969569 rc 0/0 00000100:00100000:3.0:1780298641.533653:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5812 00002000:00010000:3.0:1780298641.533657:0:19928:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:3.0:1780298641.533661:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24519c380 with x1866777609277056 ext(4194304->18446744073709551615) 00002000:00010000:3.0:1780298641.533666:0:19928:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24519c380. 00000100:00100000:0.0:1780298641.534905:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609277312 00000100:00100000:0.0:1780298641.534908:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609277312 00002000:00100000:0.0:1780298641.534927:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x12:0x0] (2191360->2195455) req@ffff9cf2433dd880 x1866777609277312/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 488/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.534933:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2433dd880 with x1866777609277312 ext(2191360->2195455) 00002000:00010000:0.0:1780298641.534936:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2433dd880 00000100:00100000:0.0:1780298641.534954:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5813 00000100:00100000:0.0:1780298641.534966:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609277312:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298641.535094:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609277312:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 130us (228us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298641.535099:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5813 00002000:00100000:0.0:1780298641.535103:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x12:0x0] (2191360->2195455) req@ffff9cf2433dd880 x1866777609277312/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 488/4536 e 0 to 0 dl 1780298652 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.535113:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2433dd880 with x1866777609277312 ext(2191360->2195455) 00002000:00010000:0.0:1780298641.535116:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2433dd880 00000100:00100000:0.0:1780298641.537844:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609277568 00000100:00100000:0.0:1780298641.537849:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609277568 00002000:00100000:0.0:1780298641.537870:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (2170880->2203647) req@ffff9cf2558f1c00 x1866777609277568/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 504/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.537879:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2558f1c00 with x1866777609277568 ext(2170880->2203647) 00002000:00010000:0.0:1780298641.537883:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2558f1c00 00000100:00100000:0.0:1780298641.537901:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5815 00000100:00100000:0.0:1780298641.537908:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609277568:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298641.538730:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609277568:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 821us (921us total) trans 0 rc 24576/24576 00000100:00100000:0.0:1780298641.538738:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5815 00002000:00100000:0.0:1780298641.538745:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (2170880->2203647) req@ffff9cf2558f1c00 x1866777609277568/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 504/440 e 0 to 0 dl 1780298652 ref 1 fl Complete:/600/0 rc 24576/24576 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.538752:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2558f1c00 with x1866777609277568 ext(2170880->2203647) 00002000:00010000:0.0:1780298641.538755:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2558f1c00 00000100:00100000:0.0:1780298641.539946:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609277696 00000100:00100000:0.0:1780298641.539951:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609277696 00002000:00100000:0.0:1780298641.539968:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x12:0x0] (2191360->2203282) req@ffff9cf2558f3100 x1866777609277696/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 12416/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.539974:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2558f3100 with x1866777609277696 ext(2191360->2203282) 00002000:00010000:0.0:1780298641.539977:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2558f3100 00000100:00100000:0.0:1780298641.539991:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5816 00000100:00100000:0.0:1780298641.539997:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609277696:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:0.0:1780298641.540464:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2399:x1866777609277696:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 465us (549us total) trans 4294969142 rc 0/0 00000100:00100000:0.0:1780298641.540471:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5816 00002000:00100000:0.0:1780298641.540479:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x12:0x0] (2191360->2203282) req@ffff9cf2558f3100 x1866777609277696/t4294969142(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 12416/448 e 0 to 0 dl 1780298652 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.540487:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2558f3100 with x1866777609277696 ext(2191360->2203282) 00002000:00010000:0.0:1780298641.540490:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2558f3100 00000100:00100000:0.0:1780298641.544402:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.544466:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609278080 00000100:00100000:3.0:1780298641.544470:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609278080 00000100:00100000:3.0:1780298641.544530:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609278208 00000100:00100000:3.0:1780298641.544533:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609278208 00000100:00100000:3.0:1780298641.544564:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8461 00000100:00100000:3.0:1780298641.544574:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609278208:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.544583:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.544597:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e6303235f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1769 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.544638:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604815744:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.544714:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00002000:00010000:3.0:1780298641.545545:0:20252:0:(ofd_lvb.c:271:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime: 1780298641 -> 1780298642 00002000:00010000:3.0:1780298641.545551:0:20252:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks: 0 -> 24 00000100:00100000:3.0:1780298641.545563:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8b1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604815744:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.545589:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e6303235f lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1769 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.545632:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e6303235f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1769 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.545645:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3c00, rc 0) 00000100:00100000:3.0:1780298641.545677:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609278208:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1104us (1165us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.545684:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8461 00000100:00100000:3.0:1780298641.546860:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609278336 00000100:00100000:3.0:1780298641.546867:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609278336 00000100:00100000:0.0:1780298641.546875:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609278464 00000100:00100000:0.0:1780298641.546881:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609278464 00000100:00100000:3.0:1780298641.546903:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8462 00000100:00100000:3.0:1780298641.546912:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609278336:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.546922:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.546933:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e63032366 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1770 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.546975:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604815872:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.547025:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.547811:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8b1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604815872:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.547832:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e63032366 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1770 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.547847:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e63032366 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1770 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.547869:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3a00, rc 0) 00000100:00100000:3.0:1780298641.547943:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609278336:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1031us (1152us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.547958:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8462 00000100:00100000:0.0:1780298641.550347:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.553370:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609278976 00000100:00100000:0.0:1780298641.553375:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609278976 00000100:00100000:0.0:1780298641.553415:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61879 00000100:00100000:0.0:1780298641.553424:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348004000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+57:29360:x1866777609278976:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.553440:0:6477:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.553457:0:6477:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b800/0xaf35132e63032374 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 37 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d177e expref: -99 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.553539:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a200/0xaf35132e6303237b lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 38 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.553556:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a200/0xaf35132e6303237b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 38 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.553572:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a200/0xaf35132e6303237b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 38 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1780298641.553597:0:6477:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 6397587 00000004:00200000:0.0:1780298641.553602:0:6477:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1780298641.553619:0:6477:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a200/0xaf35132e6303237b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 38 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.553642:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b800/0xaf35132e63032374 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 38 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d177e expref: 59 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.553658:0:6477:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a200/0xaf35132e6303237b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 37 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d177e expref: 58 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.553666:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298641.553669:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1780298641.553673:0:6477:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9a200, rc 0) 00000100:00100000:0.0:1780298641.553706:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348004000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+58:29360:x1866777609278976:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 283us (384us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298641.553714:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61879 00000100:00100000:0.0:1780298641.555151:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.555204:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609279104 00000100:00100000:3.0:1780298641.555210:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609279104 00000100:00100000:0.0:1780298641.557570:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609279360 00000100:00100000:0.0:1780298641.557576:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609279360 00002000:00100000:0.0:1780298641.557599:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (1863680->1867775) req@ffff9cf348005c00 x1866777609279360/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:347/0 lens 488/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.557609:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf348005c00 with x1866777609279360 ext(1863680->1867775) 00002000:00010000:0.0:1780298641.557612:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf348005c00 00000100:00100000:0.0:1780298641.557637:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5820 00000100:00100000:0.0:1780298641.557645:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348005c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609279360:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298641.557782:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348005c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609279360:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 138us (249us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298641.557790:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5820 00002000:00100000:0.0:1780298641.557797:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (1863680->1867775) req@ffff9cf348005c00 x1866777609279360/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:347/0 lens 488/4536 e 0 to 0 dl 1780298652 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.557806:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf348005c00 with x1866777609279360 ext(1863680->1867775) 00002000:00010000:0.0:1780298641.557809:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf348005c00 00000100:00100000:3.0:1780298641.559001:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609279488 00000100:00100000:3.0:1780298641.559007:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609279488 00000100:00100000:3.0:1780298641.559040:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8465 00000100:00100000:3.0:1780298641.559048:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348004e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609279488:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.559057:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.559069:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63032389 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d178c expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.559107:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604816384:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.560802:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.563901:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609280000 00000100:00100000:0.0:1780298641.563906:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609280000 00010000:00010000:0.0:1780298641.563926:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36e00/0xaf35132e63032351 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 37 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d175b expref: 79 pid: 6476 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.563949:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14896 00000100:00100000:0.0:1780298641.563955:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b1d850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+79:2399:x1866777609280000:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298641.563962:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298641.563966:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36e00/0xaf35132e63032351 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 38 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d175b expref: 79 pid: 6476 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.563973:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36e00/0xaf35132e63032351 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 38 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d175b expref: 79 pid: 6476 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.563983:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36e00/0xaf35132e63032351 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 38 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d175b expref: 79 pid: 6476 timeout: 930 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.563994:0:20253:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298641.563996:0:20253:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bee00/0xaf35132e63032397 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 37 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.564004:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bee00/0xaf35132e63032397 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 37 type: IBT flags: 0x40290000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.564009:0:20253:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bee00/0xaf35132e63032397 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 37 type: IBT flags: 0x40290000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.564024:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298641.564046:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b1d850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+78:2399:x1866777609280000:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 90us (181us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298641.564061:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14896 00000100:00100000:3.0:1780298641.566055:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.566083:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609280256 00000100:00100000:0.0:1780298641.566088:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609280256 00002000:00100000:0.0:1780298641.566108:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x12:0x0] (1863680->1867775) req@ffff9cf37f8b0700 x1866777609280256/t0(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:347/0 lens 4584/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.566116:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f8b0700 with x1866777609280256 ext(1863680->1867775) 00010000:00010000:0.0:1780298641.566121:0:16972:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000bf17d487/0xaf35132e63032382 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1863680->35418111] (req 1863680->1867775) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1785 expref: 10 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.566142:0:16972:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000bf17d487/0xaf35132e63032382 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1863680->35418111] (req 1863680->1867775) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1785 expref: 10 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1780298641.566148:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9cf37f8b0700 00000100:00100000:0.0:1780298641.566162:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 176 00000100:00100000:0.0:1780298641.566168:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609280256:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:0.0:1780298641.566625:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609280256:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 455us (570us total) trans 4294969143 rc 0/0 00000100:00100000:0.0:1780298641.566634:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 176 00002000:00100000:0.0:1780298641.566640:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x12:0x0] (1863680->1867775) req@ffff9cf37f8b0700 x1866777609280256/t4294969143(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:347/0 lens 4584/448 e 0 to 0 dl 1780298652 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.566649:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f8b0700 with x1866777609280256 ext(1863680->1867775) 00010000:00010000:0.0:1780298641.566655:0:16972:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000bf17d487/0xaf35132e63032382 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1863680->35418111] (req 1863680->1867775) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1785 expref: 12 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.566666:0:16972:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000bf17d487/0xaf35132e63032382 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1863680->35418111] (req 1863680->1867775) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1785 expref: 12 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1780298641.566673:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9cf37f8b0700 00000100:00100000:0.0:1780298641.567399:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609280384 00000100:00100000:0.0:1780298641.567403:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609280384 00010000:00010000:0.0:1780298641.567416:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000bf17d487/0xaf35132e63032382 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1863680->35418111] (req 1863680->1867775) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1785 expref: 12 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.567435:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14897 00000100:00100000:0.0:1780298641.567440:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f1880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609280384:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298641.567447:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:0.0:1780298641.567454:0:8374:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 24 -> 32 00010000:00010000:0.0:1780298641.567459:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000bf17d487/0xaf35132e63032382 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1863680->35418111] (req 1863680->1867775) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1785 expref: 12 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.567470:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000bf17d487/0xaf35132e63032382 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1863680->35418111] (req 1863680->1867775) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1785 expref: 12 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.567483:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bf17d487/0xaf35132e63032382 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1863680->35418111] (req 1863680->1867775) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1785 expref: 12 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.567491:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(49 130) 00010000:00010000:0.0:1780298641.567494:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(49 130) 00010000:00010000:0.0:1780298641.567497:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000000a323d4b/0xaf35132e6303239e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d179a expref: 6 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.567503:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000a323d4b/0xaf35132e6303239e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d179a expref: 6 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.567519:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000000a323d4b/0xaf35132e6303239e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d179a expref: 6 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.567526:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242863100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:8374:1866777604816896:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298641.567558:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733fbd80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.568460:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.568488:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.569910:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609280768 00000100:00100000:0.0:1780298641.569914:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609280768 00002000:00100000:0.0:1780298641.569933:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (3403776->3407871) req@ffff9cf2558f2680 x1866777609280768/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 488/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.569939:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2558f2680 with x1866777609280768 ext(3403776->3407871) 00002000:00010000:0.0:1780298641.569942:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2558f2680 00000100:00100000:0.0:1780298641.569957:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5823 00000100:00100000:0.0:1780298641.569965:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609280768:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298641.570055:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609280768:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 92us (174us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298641.570060:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5823 00002000:00100000:0.0:1780298641.570065:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (3403776->3407871) req@ffff9cf2558f2680 x1866777609280768/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 488/4536 e 0 to 0 dl 1780298652 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.570071:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2558f2680 with x1866777609280768 ext(3403776->3407871) 00002000:00010000:0.0:1780298641.570073:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2558f2680 00000100:00100000:3.0:1780298641.571182:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.571211:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609280896 00000100:00100000:0.0:1780298641.571215:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609280896 00002000:00100000:0.0:1780298641.571233:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (3407872->3411967) req@ffff9cf37f8b2300 x1866777609280896/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 488/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.571240:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f8b2300 with x1866777609280896 ext(3407872->3411967) 00002000:00010000:0.0:1780298641.571242:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f8b2300 00000100:00100000:0.0:1780298641.571258:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5824 00000100:00100000:0.0:1780298641.571263:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609280896:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298641.571345:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609280896:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 82us (165us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298641.571351:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5824 00002000:00100000:0.0:1780298641.571356:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (3407872->3411967) req@ffff9cf37f8b2300 x1866777609280896/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 488/4536 e 0 to 0 dl 1780298652 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.571367:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f8b2300 with x1866777609280896 ext(3407872->3411967) 00002000:00010000:0.0:1780298641.571369:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f8b2300 00000100:00100000:3.0:1780298641.572502:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.572533:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609281024 00000100:00100000:0.0:1780298641.572537:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609281024 00002000:00100000:0.0:1780298641.572553:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x12:0x0] (3383296->3416063) req@ffff9cf37f8b3480 x1866777609281024/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 504/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.572560:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f8b3480 with x1866777609281024 ext(3383296->3416063) 00002000:00010000:0.0:1780298641.572562:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f8b3480 00000100:00100000:0.0:1780298641.572577:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5825 00000100:00100000:0.0:1780298641.572582:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609281024:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298641.573351:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609281024:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 765us (848us total) trans 0 rc 24576/24576 00000100:00100000:0.0:1780298641.573357:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5825 00002000:00100000:0.0:1780298641.573362:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x12:0x0] (3383296->3416063) req@ffff9cf37f8b3480 x1866777609281024/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 504/440 e 0 to 0 dl 1780298652 ref 1 fl Complete:/600/0 rc 24576/24576 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.573369:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f8b3480 with x1866777609281024 ext(3383296->3416063) 00002000:00010000:0.0:1780298641.573372:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f8b3480 00000100:00100000:3.0:1780298641.574725:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609281152 00000100:00100000:3.0:1780298641.574729:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609281152 00002000:00100000:3.0:1780298641.574747:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (3403776->3413912) req@ffff9cf2558f3480 x1866777609281152/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 10632/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298641.574762:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2558f3480 with x1866777609281152 ext(3403776->3413912) 00002000:00010000:3.0:1780298641.574765:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2558f3480 00000100:00100000:3.0:1780298641.574781:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5826 00000100:00100000:3.0:1780298641.574786:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609281152:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:3.0:1780298641.575240:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609281152:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 454us (572us total) trans 4294969145 rc 0/0 00000100:00100000:3.0:1780298641.575245:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5826 00002000:00100000:3.0:1780298641.575250:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (3403776->3413912) req@ffff9cf2558f3480 x1866777609281152/t4294969145(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 10632/448 e 0 to 0 dl 1780298652 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298641.575256:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2558f3480 with x1866777609281152 ext(3403776->3413912) 00002000:00010000:3.0:1780298641.575258:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2558f3480 00000100:00100000:0.0:1780298641.577575:0:9976:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609281408 00000100:00100000:0.0:1780298641.577579:0:9976:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609281408 00000100:00100000:0.0:1780298641.577605:0:9976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61881 00000100:00100000:0.0:1780298641.577611:0:9976:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348007800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+60:29360:x1866777609281408:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.577625:0:9976:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.577642:0:9976:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ac00/0xaf35132e630323a5 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 37 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d17a1 expref: -99 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.577718:0:9976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b600/0xaf35132e630323ac lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 38 type: IBT flags: 0x40000000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.577730:0:9976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b600/0xaf35132e630323ac lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 38 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.577740:0:9976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b600/0xaf35132e630323ac lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 38 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1780298641.577758:0:9976:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 7608217 00000004:00200000:0.0:1780298641.577760:0:9976:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1780298641.577774:0:9976:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b600/0xaf35132e630323ac lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 38 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.577802:0:9976:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ac00/0xaf35132e630323a5 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 38 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d17a1 expref: 62 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.577814:0:9976:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b600/0xaf35132e630323ac lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 37 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d17a1 expref: 61 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.577820:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298641.577822:0:9976:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1780298641.577824:0:9976:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b600, rc 0) 00000100:00100000:0.0:1780298641.577846:0:9976:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348007800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+61:29360:x1866777609281408:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 235us (311us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298641.577850:0:9976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61881 00000100:00100000:0.0:1780298641.579100:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.580241:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609281664 00000100:00100000:0.0:1780298641.580245:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609281664 00000100:00100000:0.0:1780298641.580270:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8469 00000100:00100000:0.0:1780298641.580276:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348004380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609281664:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.580283:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.580293:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e630323c1 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d17af expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.580347:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242863b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604817024:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.580419:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733fbd80 going to sleep for 11 seconds 00002000:00010000:0.0:1780298641.581099:0:20252:0:(ofd_lvb.c:271:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime: 1780298641 -> 1780298642 00000100:00100000:0.0:1780298641.581117:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242863b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604817024:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.581162:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e630323c1 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d17af expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.581182:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e630323c1 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d17af expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.581194:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9aa00, rc 0) 00000100:00100000:0.0:1780298641.581230:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348004380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609281664:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 953us (1026us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.581239:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8469 00000100:00100000:3.0:1780298641.583186:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.583221:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609282048 00000100:00100000:0.0:1780298641.583225:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609282048 00010000:00010000:0.0:1780298641.583240:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000000a323d4b/0xaf35132e6303239e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d179a expref: 7 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.583266:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14898 00000100:00100000:0.0:1780298641.583271:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609282048:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298641.583278:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298641.583286:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000000a323d4b/0xaf35132e6303239e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d179a expref: 7 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.583294:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000000a323d4b/0xaf35132e6303239e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d179a expref: 7 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.583320:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000a323d4b/0xaf35132e6303239e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d179a expref: 7 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.583328:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(50 130) 00010000:00010000:0.0:1780298641.583331:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(50 130) 00010000:00010000:0.0:1780298641.583336:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000408198ca/0xaf35132e630323cf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2424832->35979263] (req 2424832->2428927) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d17bd expref: 9 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.583343:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000408198ca/0xaf35132e630323cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2424832->35979263] (req 2424832->2428927) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d17bd expref: 9 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.583358:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000408198ca/0xaf35132e630323cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2424832->35979263] (req 2424832->2428927) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d17bd expref: 9 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.583369:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242863b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6461:1866777604817408:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298641.583397:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733fbd80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.584058:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242863b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6461:1866777604817408:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298641.584067:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298641.584088:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609282048:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 818us (903us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298641.584095:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14898 00000100:00100000:0.0:1780298641.584255:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.585432:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8472 00000100:00100000:0.0:1780298641.585443:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348005f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609282304:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.585450:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.585461:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e630323d6 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d17c4 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.585490:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242862d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604817536:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.585528:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733fbd80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.586273:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242862d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604817536:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.586310:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e630323d6 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d17c4 expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.586326:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e630323d6 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d17c4 expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.586353:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9ac00, rc 0) 00000100:00100000:0.0:1780298641.586382:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348005f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609282304:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 941us (1035us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.586390:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8472 00000100:00100000:3.0:1780298641.587378:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.589225:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.590389:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.592163:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.593857:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.593923:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.595435:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.596345:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.597751:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.597780:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609283712 00000100:00100000:0.0:1780298641.597784:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609283712 00002000:00100000:0.0:1780298641.597802:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x12:0x0] (2416640->2420735) req@ffff9cf37f8b1f80 x1866777609283712/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:347/0 lens 488/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.597809:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f8b1f80 with x1866777609283712 ext(2416640->2420735) 00002000:00010000:0.0:1780298641.597811:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f8b1f80 00000100:00100000:0.0:1780298641.597825:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5830 00000100:00100000:0.0:1780298641.597830:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609283712:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298641.597899:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609283712:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 68us (149us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298641.597904:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5830 00002000:00100000:0.0:1780298641.597908:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x12:0x0] (2416640->2420735) req@ffff9cf37f8b1f80 x1866777609283712/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:347/0 lens 488/4536 e 0 to 0 dl 1780298652 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.597918:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f8b1f80 with x1866777609283712 ext(2416640->2420735) 00002000:00010000:0.0:1780298641.597920:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f8b1f80 00000100:00100000:0.0:1780298641.598909:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.598940:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609283840 00000100:00100000:3.0:1780298641.598944:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609283840 00002000:00100000:3.0:1780298641.598961:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (2420736->2424831) req@ffff9cf242861c00 x1866777609283840/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:347/0 lens 488/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298641.598968:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242861c00 with x1866777609283840 ext(2420736->2424831) 00002000:00010000:3.0:1780298641.598971:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf242861c00 00000100:00100000:3.0:1780298641.598993:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5831 00000100:00100000:3.0:1780298641.599000:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242861c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609283840:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298641.599092:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242861c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609283840:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 92us (183us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1780298641.599099:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5831 00002000:00100000:3.0:1780298641.599106:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (2420736->2424831) req@ffff9cf242861c00 x1866777609283840/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:347/0 lens 488/4536 e 0 to 0 dl 1780298652 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298641.599114:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242861c00 with x1866777609283840 ext(2420736->2424831) 00002000:00010000:3.0:1780298641.599118:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf242861c00 00000100:00100000:3.0:1780298641.600260:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609283968 00000100:00100000:3.0:1780298641.600264:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609283968 00002000:00100000:3.0:1780298641.600287:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (2412544->2424831) req@ffff9cf348007b80 x1866777609283968/t0(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:347/0 lens 12776/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298641.600310:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf348007b80 with x1866777609283968 ext(2412544->2424831) 00002000:00010000:3.0:1780298641.600314:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf348007b80 00000100:00100000:3.0:1780298641.600332:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5832 00000100:00100000:3.0:1780298641.600340:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348007b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2399:x1866777609283968:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:0.0:1780298641.601576:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.603148:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.603213:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.604012:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348004a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604818560:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.604035:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002de18a35/0xaf35132e63032415 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1803 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.604052:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002de18a35/0xaf35132e63032415 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1803 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.604070:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf2445bfa00, rc 0) 00000100:00100000:0.0:1780298641.604097:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609284224:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 879us (971us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.604102:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8482 00000100:00100000:0.0:1780298641.605171:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.605192:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.605225:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609284608 00000100:00100000:3.0:1780298641.605230:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609284608 00000100:00100000:3.0:1780298641.606119:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348007480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604818816:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.606164:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000fedd83c9/0xaf35132e63032423 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1811 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.606177:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fedd83c9/0xaf35132e63032423 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1811 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.606205:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf2445be000, rc 0) 00000100:00100000:0.0:1780298641.607151:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609284736 00000100:00100000:0.0:1780298641.607156:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609284736 00000100:00100000:0.0:1780298641.607190:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8486 00000100:00100000:0.0:1780298641.607198:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348006680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609284736:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.607206:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.607214:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7fd0135/0xaf35132e63032431 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d181f expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.607241:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255490380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604819072:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.607279:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733fbd80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.608774:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.609908:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.610994:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000001:00080000:0.0:1780298641.611822:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969571 is committed 00000001:00080000:0.0:1780298641.611838:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969564 is committed 00000100:00100000:3.0:1780298641.612690:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.612739:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609285248 00000100:00100000:0.0:1780298641.612743:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609285248 00000100:00100000:0.0:1780298641.612771:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8488 00000100:00100000:0.0:1780298641.612777:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2398:x1866777609285248:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.612786:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298641.612794:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:0.0:1780298641.612801:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009fd8d343/0xaf35132e63032438 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1826 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.612851:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255493480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604819200:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.612907:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733fb600 going to sleep for 11 seconds 00002000:00010000:0.0:1780298641.613569:0:12101:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks: 624 -> 632 00000100:00100000:0.0:1780298641.613579:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255493480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604819200:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.613603:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009fd8d343/0xaf35132e63032438 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1826 expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.613627:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009fd8d343/0xaf35132e63032438 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1826 expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.613656:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b000, rc 0) 00000100:00100000:0.0:1780298641.613685:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2398:x1866777609285248:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 905us (993us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.613692:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8488 00000100:00100000:0.0:1780298641.614782:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.615633:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604819584:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.615649:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e6303244d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d183b expref: 12 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.615663:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e6303244d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d183b expref: 12 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.615673:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36a00, rc 0) 00000100:00100000:3.0:1780298641.615703:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e60c850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609285632:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 863us (921us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.615710:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8491 00000100:00100000:0.0:1780298641.616427:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.617156:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604819712:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298641.617168:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032454 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2977792->2990079] (req 2977792->2990079) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1842 expref: 7 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.617177:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37000, rc 0) 00000100:00100000:0.0:1780298641.617199:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255493480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609285760:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 686us (772us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298641.617203:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8492 00000100:00100000:0.0:1780298641.617390:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.618541:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.618575:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609286016 00000100:00100000:0.0:1780298641.618579:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609286016 00000100:00100000:0.0:1780298641.618610:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5835 00000100:00100000:0.0:1780298641.618615:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609286016:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298641.618759:0:16972:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 3413913 -> 3413913 00002000:00010000:0.0:1780298641.618762:0:16972:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298641.618764:0:16972:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298641.618766:0:16972:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298641.618767:0:16972:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 96 -> 96 00000100:00100000:0.0:1780298641.618788:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609286016:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 174us (250us total) trans 4294969148 rc 0/0 00000100:00100000:0.0:1780298641.618793:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5835 00000100:00100000:0.0:1780298641.619616:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.619642:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609286144 00000100:00100000:3.0:1780298641.619648:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609286144 00000100:00100000:3.0:1780298641.619677:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8493 00000100:00100000:3.0:1780298641.619686:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255490e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609286144:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.619694:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.619705:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000010b6573d/0xaf35132e6303245b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1849 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.619749:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604819968:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.619793:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.620370:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8b3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604819968:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.620384:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000010b6573d/0xaf35132e6303245b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1849 expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.620394:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000010b6573d/0xaf35132e6303245b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1849 expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.620401:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3a00, rc 0) 00000100:00100000:0.0:1780298641.620422:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255490e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609286144:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 740us (808us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.620427:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8493 00000100:00100000:0.0:1780298641.621224:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609286272 00000100:00100000:0.0:1780298641.621228:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609286272 00000100:00100000:0.0:1780298641.621249:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8494 00000100:00100000:0.0:1780298641.621253:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609286272:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.621259:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.621265:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e63032462 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1850 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.621285:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255490000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604820096:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.621324:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733fb600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.621888:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255490000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604820096:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.621900:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e63032462 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1850 expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.621911:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e63032462 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1850 expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.621917:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b000, rc 0) 00000100:00100000:0.0:1780298641.621940:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609286272:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 686us (747us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.621945:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8494 00000100:00100000:3.0:1780298641.622793:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.622897:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.622926:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609286528 00000100:00100000:0.0:1780298641.622931:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609286528 00000100:00100000:0.0:1780298641.622959:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8496 00000100:00100000:0.0:1780298641.622966:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609286528:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.622973:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.622980:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000019780e8d/0xaf35132e63032470 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d185e expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.623002:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255490000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604820352:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.623024:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733fb600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.623759:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255490000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604820352:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.623776:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000019780e8d/0xaf35132e63032470 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d185e expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.623789:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000019780e8d/0xaf35132e63032470 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d185e expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.623795:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9a200, rc 0) 00000100:00100000:0.0:1780298641.623818:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609286528:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 854us (922us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.623823:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8496 00000100:00100000:3.0:1780298641.624516:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.624544:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609286656 00000100:00100000:0.0:1780298641.624549:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609286656 00000100:00100000:3.0:1780298641.624567:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.624578:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609286784 00000100:00100000:0.0:1780298641.624580:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609286784 00000100:00100000:0.0:1780298641.624606:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8497 00000100:00100000:0.0:1780298641.624616:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609286656:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.624625:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.624632:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63032477 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1865 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.624658:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255490000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604820480:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.624696:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733fb600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.625284:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255490000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604820480:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.625310:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63032477 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1865 expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.625329:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63032477 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1865 expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.625337:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9ac00, rc 0) 00000100:00100000:0.0:1780298641.625363:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609286656:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 750us (884us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.625368:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8497 00000100:00100000:3.0:1780298641.626317:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609286912 00000100:00100000:3.0:1780298641.626321:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609286912 00000100:00100000:3.0:1780298641.626347:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8499 00000100:00100000:3.0:1780298641.626355:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:29360:x1866777609286912:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.626363:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.626370:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e63032485 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1873 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.626387:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e63032327 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1746 expref: 6 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.626396:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(69 177) 00010000:00010000:3.0:1780298641.626400:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(70 177) 00010000:00010000:3.0:1780298641.626405:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e63032485 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [659456->671743] (req 659456->671743) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1873 expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.626423:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e63032327 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1746 expref: 6 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.626448:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e63032327 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1746 expref: 6 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.626458:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604820736:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298641.626497:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.627493:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.627523:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609287040 00000100:00100000:3.0:1780298641.627527:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609287040 00010000:00010000:3.0:1780298641.627546:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e63032327 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1746 expref: 7 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.627569:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14900 00000100:00100000:3.0:1780298641.627575:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255490000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609287040:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298641.627585:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298641.627594:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e63032327 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1746 expref: 7 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.627605:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e63032327 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1746 expref: 7 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.627622:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e63032327 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1746 expref: 7 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.627633:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(70 177) 00010000:00010000:3.0:1780298641.627642:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(70 177) 00010000:00010000:3.0:1780298641.627648:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e63032485 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 659456->671743) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1873 expref: 8 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.627657:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e63032485 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 659456->671743) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1873 expref: 8 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.627678:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e63032485 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 659456->671743) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1873 expref: 8 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.627687:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b3800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777604820864:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298641.627719:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.628337:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8b3800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777604820864:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298641.628347:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298641.628371:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255490000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609287040:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 795us (879us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298641.628379:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14900 00000100:00100000:0.0:1780298641.628778:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.628801:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609287168 00000100:00100000:3.0:1780298641.628806:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609287168 00002000:00100000:3.0:1780298641.628828:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (659456->671743) req@ffff9cf255491500 x1866777609287168/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:347/0 lens 488/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298641.628838:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255491500 with x1866777609287168 ext(659456->671743) 00002000:00010000:3.0:1780298641.628843:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255491500 00000100:00100000:3.0:1780298641.628859:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5836 00000100:00100000:3.0:1780298641.628866:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255491500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609287168:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298641.628999:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255491500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609287168:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 133us (220us total) trans 0 rc 12288/12288 00000100:00100000:3.0:1780298641.629006:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5836 00002000:00100000:3.0:1780298641.629012:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (659456->671743) req@ffff9cf255491500 x1866777609287168/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:347/0 lens 488/12728 e 0 to 0 dl 1780298652 ref 1 fl Complete:/600/0 rc 12288/12288 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298641.629021:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255491500 with x1866777609287168 ext(659456->671743) 00002000:00010000:3.0:1780298641.629025:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255491500 00000100:00100000:0.0:1780298641.629983:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.631727:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.631748:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609287424 00000100:00100000:3.0:1780298641.631752:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609287424 00000100:00100000:3.0:1780298641.631780:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8501 00000100:00100000:3.0:1780298641.631788:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255491f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609287424:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.631796:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.631804:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e63032493 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1881 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.631830:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604821120:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.631860:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.632517:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8b3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604821120:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.632532:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e63032493 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1881 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.632545:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e63032493 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1881 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.632568:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3e00, rc 0) 00000100:00100000:3.0:1780298641.632592:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255491f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609287424:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 805us (867us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.632599:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8501 00000100:00100000:0.0:1780298641.633363:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609287552 00000100:00100000:0.0:1780298641.633368:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609287552 00000100:00100000:0.0:1780298641.633397:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8502 00000100:00100000:0.0:1780298641.633404:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348007480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:29360:x1866777609287552:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.633410:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.633416:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000019780e8d/0xaf35132e6303249a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1888 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.633428:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(51 130) 00010000:00010000:0.0:1780298641.633431:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(51 130) 00010000:00010000:0.0:1780298641.633437:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000019780e8d/0xaf35132e6303249a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [733184->2412543] (req 733184->741375) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1888 expref: 9 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.633445:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000019780e8d/0xaf35132e6303249a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [733184->2412543] (req 733184->741375) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1888 expref: 9 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.633449:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9a200, rc 0) 00000100:00100000:0.0:1780298641.633472:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348007480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:29360:x1866777609287552:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 68us (147us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298641.633478:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8502 00000100:00100000:0.0:1780298641.634030:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.635024:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.635792:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f1180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604821248:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.635805:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003c8dab73/0xaf35132e630324a1 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d188f expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.635815:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003c8dab73/0xaf35132e630324a1 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d188f expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.635821:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36800, rc 0) 00000100:00100000:3.0:1780298641.635842:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609287808:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 743us (820us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.635847:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8503 00000100:00100000:0.0:1780298641.636632:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.638192:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609288064 00000100:00100000:3.0:1780298641.638195:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609288064 00000100:00100000:3.0:1780298641.638218:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8505 00000100:00100000:3.0:1780298641.638225:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348004a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609288064:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.638232:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.638239:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e630324af lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d189d expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.638274:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433dea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604821504:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.638313:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.638913:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433dea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604821504:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.638924:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e630324af lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d189d expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.638932:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e630324af lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d189d expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.638938:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2400, rc 0) 00000100:00100000:3.0:1780298641.638959:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348004a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609288064:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 735us (794us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.638963:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8505 00000100:00100000:0.0:1780298641.639817:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.642523:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.642553:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609288448 00000100:00100000:3.0:1780298641.642556:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609288448 00002000:00100000:3.0:1780298641.642570:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (892928->1454079) req@ffff9cf37f7bf480 x1866777609288448/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:347/0 lens 504/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298641.642576:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f7bf480 with x1866777609288448 ext(892928->1454079) 00002000:00010000:3.0:1780298641.642578:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f7bf480 00000100:00100000:3.0:1780298641.642591:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5840 00000100:00100000:3.0:1780298641.642595:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609288448:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298641.644079:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609288448:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 1480us (1554us total) trans 0 rc 552960/552960 00000100:00100000:0.0:1780298641.644088:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5840 00002000:00100000:0.0:1780298641.644095:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (892928->1454079) req@ffff9cf37f7bf480 x1866777609288448/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:347/0 lens 504/440 e 0 to 0 dl 1780298652 ref 1 fl Complete:/600/0 rc 552960/552960 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.644105:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f7bf480 with x1866777609288448 ext(892928->1454079) 00002000:00010000:0.0:1780298641.644109:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f7bf480 00000100:00100000:0.0:1780298641.645016:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.645816:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348004e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604821632:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.645835:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a860e235/0xaf35132e630324b6 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d18a4 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.645849:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a860e235/0xaf35132e630324b6 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d18a4 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.645857:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244f3b400, rc 0) 00000100:00100000:0.0:1780298641.645882:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609288576:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 800us (868us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.645889:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8506 00000100:00100000:3.0:1780298641.646627:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.648275:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.648308:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609288960 00000100:00100000:3.0:1780298641.648318:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609288960 00000100:00100000:3.0:1780298641.648347:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8509 00000100:00100000:3.0:1780298641.648360:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609288960:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.648367:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.648374:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e630324cb lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d18b9 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.648396:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433ddc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604821888:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.648480:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.649030:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433ddc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604821888:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.649042:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e630324cb lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d18b9 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.649051:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e630324cb lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d18b9 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.649057:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3e00, rc 0) 00000100:00100000:3.0:1780298641.649078:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609288960:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 724us (806us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.649082:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8509 00000100:00100000:3.0:1780298641.650033:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609289088 00000100:00100000:3.0:1780298641.650037:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609289088 00002000:00100000:3.0:1780298641.650054:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (3375104->3379199) req@ffff9cf348004380 x1866777609289088/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 488/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298641.650061:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf348004380 with x1866777609289088 ext(3375104->3379199) 00002000:00010000:3.0:1780298641.650068:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf348004380 00000100:00100000:3.0:1780298641.650083:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5841 00000100:00100000:3.0:1780298641.650087:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348004380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609289088:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298641.651280:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609289216 00000100:00100000:0.0:1780298641.651284:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609289216 00002000:00100000:0.0:1780298641.651313:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (3379200->3383295) req@ffff9cf348005880 x1866777609289216/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 488/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.651320:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf348005880 with x1866777609289216 ext(3379200->3383295) 00002000:00010000:0.0:1780298641.651323:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf348005880 00000100:00100000:0.0:1780298641.651338:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5842 00000100:00100000:0.0:1780298641.651344:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348005880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609289216:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298641.651414:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348005880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609289216:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 70us (170us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298641.651418:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5842 00002000:00100000:0.0:1780298641.651422:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (3379200->3383295) req@ffff9cf348005880 x1866777609289216/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 488/4536 e 0 to 0 dl 1780298652 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.651428:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf348005880 with x1866777609289216 ext(3379200->3383295) 00002000:00010000:0.0:1780298641.651431:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf348005880 00000100:00100000:0.0:1780298641.652325:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.653522:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609289472 00000100:00100000:0.0:1780298641.653526:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609289472 00000100:00100000:0.0:1780298641.653549:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8510 00000100:00100000:0.0:1780298641.653554:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348005f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609289472:12345-192.168.203.25@tcp:16:fsx.0 00000100:00100000:0.0:1780298641.654311:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348005f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609289472:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 756us (820us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298641.654319:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8510 00000100:00100000:0.0:1780298641.656846:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609289728 00000100:00100000:0.0:1780298641.656850:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609289728 00000100:00100000:0.0:1780298641.656872:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8512 00000100:00100000:0.0:1780298641.656877:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348006680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609289728:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.656882:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.656895:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad103424/0xaf35132e630324d9 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d18c7 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.656920:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604822144:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.656948:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733fb600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.657572:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604822144:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.657583:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ad103424/0xaf35132e630324d9 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d18c7 expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.657593:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad103424/0xaf35132e630324d9 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d18c7 expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.657599:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9aa00, rc 0) 00000100:00100000:0.0:1780298641.657619:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348006680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609289728:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 743us (801us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.657624:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8512 00000100:00100000:3.0:1780298641.658330:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.660240:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.660279:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609290112 00000100:00100000:0.0:1780298641.660283:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609290112 00002000:00100000:0.0:1780298641.660309:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x12:0x0] (507904->511999) req@ffff9cf2433df480 x1866777609290112/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 488/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.660317:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2433df480 with x1866777609290112 ext(507904->511999) 00002000:00010000:0.0:1780298641.660320:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2433df480 00000100:00100000:0.0:1780298641.660336:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5845 00000100:00100000:0.0:1780298641.660341:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609290112:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298641.660406:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609290112:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 65us (166us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298641.660411:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5845 00002000:00100000:0.0:1780298641.660415:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x12:0x0] (507904->511999) req@ffff9cf2433df480 x1866777609290112/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 488/4536 e 0 to 0 dl 1780298652 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.660421:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2433df480 with x1866777609290112 ext(507904->511999) 00002000:00010000:0.0:1780298641.660423:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2433df480 00000100:00100000:3.0:1780298641.661371:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.661396:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609290240 00000100:00100000:0.0:1780298641.661400:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609290240 00000100:00100000:0.0:1780298641.661422:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8514 00000100:00100000:0.0:1780298641.661427:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609290240:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.661433:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.661440:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e630324e7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d18d5 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.661466:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604822272:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.661502:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733fbd80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.662091:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604822272:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.662103:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e630324e7 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d18d5 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.662113:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e630324e7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d18d5 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.662120:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9ac00, rc 0) 00000100:00100000:0.0:1780298641.662158:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609290240:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 731us (788us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.662163:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8514 00000100:00100000:3.0:1780298641.662985:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.664952:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609290496 00000100:00100000:0.0:1780298641.664956:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609290496 00002000:00100000:0.0:1780298641.664973:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (2129920->2142207) req@ffff9cf348318e00 x1866777609290496/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 488/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.664979:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf348318e00 with x1866777609290496 ext(2129920->2142207) 00002000:00010000:0.0:1780298641.664982:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf348318e00 00000100:00100000:0.0:1780298641.664997:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5846 00000100:00100000:0.0:1780298641.665002:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348318e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609290496:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298641.665085:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348318e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609290496:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 83us (164us total) trans 0 rc 12288/12288 00000100:00100000:0.0:1780298641.665090:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5846 00002000:00100000:0.0:1780298641.665093:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (2129920->2142207) req@ffff9cf348318e00 x1866777609290496/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 488/12728 e 0 to 0 dl 1780298652 ref 1 fl Complete:/600/0 rc 12288/12288 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.665099:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf348318e00 with x1866777609290496 ext(2129920->2142207) 00002000:00010000:0.0:1780298641.665101:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf348318e00 00000100:00100000:3.0:1780298641.666096:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.666144:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609290624 00000100:00100000:0.0:1780298641.666148:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609290624 00000100:00100000:0.0:1780298641.666176:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8516 00000100:00100000:0.0:1780298641.666182:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609290624:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.666188:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.666195:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e630324f5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d18e3 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.666217:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604822528:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.666250:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733fbd80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.666801:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604822528:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.666813:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e630324f5 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d18e3 expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.666823:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e630324f5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d18e3 expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.666829:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b000, rc 0) 00000100:00100000:0.0:1780298641.666851:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609290624:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 671us (758us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.666856:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8516 00000100:00100000:3.0:1780298641.667743:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609290752 00000100:00100000:3.0:1780298641.667747:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609290752 00000100:00100000:3.0:1780298641.667770:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8517 00000100:00100000:3.0:1780298641.667777:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34831a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609290752:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.667783:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.667790:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e630324fc lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d18ea expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.667811:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433dce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604822656:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.667842:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.669384:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.671167:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.671198:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609291008 00000100:00100000:0.0:1780298641.671204:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609291008 00000100:00100000:0.0:1780298641.671238:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8519 00000100:00100000:0.0:1780298641.671246:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:29360:x1866777609291008:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.671256:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.671265:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7fd0135/0xaf35132e6303250a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d18f8 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.671281:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(51 130) 00010000:00010000:0.0:1780298641.671286:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(51 130) 00010000:00010000:0.0:1780298641.671307:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7fd0135/0xaf35132e6303250a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [2838528->2977791] (req 2838528->2842623) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d18f8 expref: 10 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.671323:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e7fd0135/0xaf35132e6303250a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [2838528->2977791] (req 2838528->2842623) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d18f8 expref: 10 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.671332:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9be00, rc 0) 00000100:00100000:0.0:1780298641.671359:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:29360:x1866777609291008:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 115us (194us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298641.671364:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8519 00000100:00100000:0.0:1780298641.672214:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.673443:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609291264 00000100:00100000:0.0:1780298641.673447:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609291264 00002000:00100000:0.0:1780298641.673464:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (2842624->2846719) req@ffff9cf2558f2d80 x1866777609291264/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 488/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.673471:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2558f2d80 with x1866777609291264 ext(2842624->2846719) 00002000:00010000:0.0:1780298641.673474:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2558f2d80 00000100:00100000:0.0:1780298641.673495:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5848 00000100:00100000:0.0:1780298641.673503:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609291264:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298641.673591:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609291264:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 90us (182us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298641.673596:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5848 00002000:00100000:0.0:1780298641.673601:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (2842624->2846719) req@ffff9cf2558f2d80 x1866777609291264/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 488/4536 e 0 to 0 dl 1780298652 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.673607:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2558f2d80 with x1866777609291264 ext(2842624->2846719) 00002000:00010000:0.0:1780298641.673610:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2558f2d80 00000100:00100000:3.0:1780298641.674532:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.675761:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.675794:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609291520 00000100:00100000:0.0:1780298641.675798:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609291520 00000100:00100000:0.0:1780298641.675828:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8520 00000100:00100000:0.0:1780298641.675836:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242861880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609291520:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.675845:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.675858:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d71a6aba/0xaf35132e63032511 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d18ff expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.675903:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bf100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604822912:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.675953:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733fb600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.676623:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bf100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604822912:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.676646:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d71a6aba/0xaf35132e63032511 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d18ff expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.676672:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d71a6aba/0xaf35132e63032511 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d18ff expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.676689:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9bc00, rc 0) 00000100:00100000:0.0:1780298641.676723:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242861880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609291520:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 888us (962us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.676730:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8520 00000100:00100000:3.0:1780298641.677540:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.679099:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609291776 00000100:00100000:0.0:1780298641.679102:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609291776 00000100:00100000:0.0:1780298641.679145:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8522 00000100:00100000:0.0:1780298641.679151:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609291776:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.679158:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.679166:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e6303251f lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d190d expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.679181:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e63032485 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 659456->671743) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1873 expref: 8 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.679187:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000b77c5ae2/0xaf35132e630324c4 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d18b2 expref: 8 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.679192:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(70 177) 00010000:00010000:0.0:1780298641.679195:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(70 177) 00010000:00010000:0.0:1780298641.679198:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e6303251f lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2793472->2797567] (req 2793472->2797567) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d190d expref: 8 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.679213:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000b77c5ae2/0xaf35132e630324c4 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d18b2 expref: 8 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.679224:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000b77c5ae2/0xaf35132e630324c4 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d18b2 expref: 8 pid: 8356 timeout: 931 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.679230:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bf100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604823168:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298641.679264:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e63032485 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 659456->671743) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1873 expref: 8 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.679271:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e63032485 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 659456->671743) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1873 expref: 8 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.679278:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604823296:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298641.679294:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733fb600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.679915:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bf100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604823168:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298641.680065:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604823296:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298641.680088:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e6303251f lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2793472->2797567] (req 2793472->2797567) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d190d expref: 8 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.680099:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:0.0:1780298641.680102:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9ac00, rc 0) 00000100:00100000:0.0:1780298641.680155:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:29360:x1866777609291776:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1001us (1088us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298641.680165:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8522 00000100:00100000:3.0:1780298641.680882:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.680909:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609292032 00000100:00100000:0.0:1780298641.680913:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609292032 00010000:00010000:0.0:1780298641.680925:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e63032485 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 659456->671743) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1873 expref: 9 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.680943:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14902 00000100:00100000:0.0:1780298641.680947:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242860700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609292032:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298641.680953:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298641.680959:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e63032485 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 659456->671743) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1873 expref: 9 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.680966:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e63032485 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 659456->671743) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1873 expref: 9 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.680975:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e63032485 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 659456->671743) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1873 expref: 9 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.680982:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(70 177) 00010000:00010000:0.0:1780298641.680984:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(70 177) 00010000:00010000:0.0:1780298641.680990:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e6303251f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2793472->36347903] (req 2793472->2797567) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d190d expref: 6 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.680995:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e6303251f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2793472->36347903] (req 2793472->2797567) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d190d expref: 6 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.681009:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e6303251f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2793472->36347903] (req 2793472->2797567) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d190d expref: 6 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.681020:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bc380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777604823424:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298641.681048:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733fb600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.681707:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bc380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777604823424:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298641.681719:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298641.681737:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609292160 00000100:00100000:3.0:1780298641.681742:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609292160 00000100:00100000:0.0:1780298641.681745:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242860700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2400:x1866777609292032:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 796us (862us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298641.681751:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14902 00000100:00100000:3.0:1780298641.681785:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5850 00000100:00100000:3.0:1780298641.681793:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348318000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609292160:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:3.0:1780298641.681930:0:16972:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:3.0:1780298641.681932:0:16972:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:3.0:1780298641.681934:0:16972:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:3.0:1780298641.681936:0:16972:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:3.0:1780298641.681937:0:16972:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 632 -> 632 00000100:00100000:3.0:1780298641.681960:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348318000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609292160:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 170us (265us total) trans 4294969572 rc 0/0 00000100:00100000:3.0:1780298641.681964:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5850 00000100:00100000:0.0:1780298641.682860:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609292288 00000100:00100000:0.0:1780298641.682864:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609292288 00000100:00100000:0.0:1780298641.682886:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609292416 00000100:00100000:0.0:1780298641.682888:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609292416 00000100:00100000:0.0:1780298641.683669:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348318380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604823552:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.683688:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009856b175/0xaf35132e6303252d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d191b expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.683717:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009856b175/0xaf35132e6303252d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d191b expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.683729:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244f3b200, rc 0) 00000100:00100000:0.0:1780298641.683762:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348319180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609292416:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 849us (915us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.683770:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8524 00000100:00100000:3.0:1780298641.684723:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609292544 00000100:00100000:3.0:1780298641.684727:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609292544 00000100:00100000:3.0:1780298641.684750:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8525 00000100:00100000:3.0:1780298641.684758:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34831aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609292544:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.684764:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.684771:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005c47b04a/0xaf35132e63032534 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1922 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.684796:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242860000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604823808:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.684834:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609292672 00000100:00100000:0.0:1780298641.684839:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609292672 00000100:00100000:3.0:1780298641.684864:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.684877:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8526 00000100:00100000:0.0:1780298641.684885:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348318a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609292672:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.684893:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.684902:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e6303253b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1929 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.684928:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604823936:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.684963:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733fb600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.685392:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242860000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604823808:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.685409:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005c47b04a/0xaf35132e63032534 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1922 expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.685423:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005c47b04a/0xaf35132e63032534 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1922 expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.685443:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3400, rc 0) 00000100:00100000:3.0:1780298641.685469:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34831aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609292544:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 712us (815us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.685476:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8525 00000100:00100000:0.0:1780298641.685612:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604823936:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.685627:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e6303253b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1929 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.685640:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e6303253b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1929 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.685649:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b000, rc 0) 00000100:00100000:0.0:1780298641.685677:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348318a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609292672:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 793us (887us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.685684:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8526 00000100:00100000:0.0:1780298641.686438:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609292928 00000100:00100000:0.0:1780298641.686443:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609292928 00000100:00100000:0.0:1780298641.686480:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61883 00000100:00100000:0.0:1780298641.686489:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34831ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+79:29360:x1866777609292928:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:0.0:1780298641.686530:0:6477:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298641.686547:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36200/0xaf35132e630323ba lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 39 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d17a8 expref: 79 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.686560:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298641.686564:0:6477:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1780298641.686699:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9bc00/0xaf35132e63032542 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 38 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.686712:0:6477:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b600/0xaf35132e630323ac lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 38 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d17a1 expref: 60 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.686726:0:6477:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9bc00/0xaf35132e63032542 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 38 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.686748:0:6477:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b600/0xaf35132e630323ac lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 38 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d17a1 expref: 60 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.686760:0:6477:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b600/0xaf35132e630323ac lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 38 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d17a1 expref: 60 pid: 9976 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.686771:0:6477:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bc380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6477:1866777604824064:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298641.686810:0:6477:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733fbd80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.687407:0:6477:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bc380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6477:1866777604824064:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298641.687420:0:6477:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9bc00/0xaf35132e63032542 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 38 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.687599:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609293056 00000100:00100000:0.0:1780298641.687603:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609293056 00010000:00010000:0.0:1780298641.687620:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b600/0xaf35132e630323ac lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 38 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d17a1 expref: 61 pid: 9976 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.687649:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14903 00000100:00100000:0.0:1780298641.687657:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b1e050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+61:2400:x1866777609293056:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298641.687666:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298641.687671:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b600/0xaf35132e630323ac lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 39 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d17a1 expref: 61 pid: 9976 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.687681:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b600/0xaf35132e630323ac lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 39 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d17a1 expref: 61 pid: 9976 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.687693:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b600/0xaf35132e630323ac lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 39 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d17a1 expref: 61 pid: 9976 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.687703:0:8374:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298641.687707:0:8374:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9bc00/0xaf35132e63032542 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 38 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.687716:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9bc00/0xaf35132e63032542 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 38 type: IBT flags: 0x40290000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.687723:0:8374:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9bc00/0xaf35132e63032542 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 38 type: IBT flags: 0x40290000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.687741:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298641.687767:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b1e050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+60:2400:x1866777609293056:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 111us (204us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298641.687779:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14903 00000100:00100000:3.0:1780298641.688560:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609293184 00000100:00100000:3.0:1780298641.688564:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609293184 00000100:00100000:3.0:1780298641.688587:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8527 00000100:00100000:3.0:1780298641.688592:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348318700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:29360:x1866777609293184:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.688598:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.688606:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e63032549 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1930 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.688621:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e6303251f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2793472->36347903] (req 2793472->2797567) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d190d expref: 7 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.688627:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(70 177) 00010000:00010000:3.0:1780298641.688630:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(71 177) 00010000:00010000:3.0:1780298641.688633:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e63032549 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1930 expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.688645:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e6303251f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2793472->36347903] (req 2793472->2797567) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d190d expref: 7 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.688651:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e6303251f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2793472->36347903] (req 2793472->2797567) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d190d expref: 7 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.688665:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242860000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604824192:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298641.688693:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.689254:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242860000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604824192:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298641.689267:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e63032549 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1930 expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.689279:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2400, rc 0) 00000100:00100000:3.0:1780298641.689321:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348318700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609293184:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 728us (781us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298641.689328:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8527 00000100:00100000:0.0:1780298641.689585:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609293312 00000100:00100000:0.0:1780298641.689589:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609293312 00010000:00010000:0.0:1780298641.689605:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e6303251f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2793472->36347903] (req 2793472->2797567) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d190d expref: 8 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.689630:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14904 00000100:00100000:0.0:1780298641.689637:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f0700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609293312:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298641.689646:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298641.689664:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e6303251f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2793472->36347903] (req 2793472->2797567) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d190d expref: 8 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.689675:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e6303251f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2793472->36347903] (req 2793472->2797567) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d190d expref: 8 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.689690:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e6303251f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2793472->36347903] (req 2793472->2797567) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d190d expref: 8 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.689697:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(71 177) 00010000:00010000:0.0:1780298641.689699:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(71 177) 00010000:00010000:0.0:1780298641.689704:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e63032549 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1930 expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.689713:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e63032549 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1930 expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.689735:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e63032549 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1930 expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.689752:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bc380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777604824320:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298641.689781:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733fb600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.690482:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bc380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777604824320:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298641.690489:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298641.690512:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f0700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609293312:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 877us (970us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298641.690517:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14904 00000100:00100000:3.0:1780298641.690667:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609293440 00000100:00100000:3.0:1780298641.690671:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609293440 00000100:00100000:3.0:1780298641.690696:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8528 00000100:00100000:3.0:1780298641.690701:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609293440:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.690707:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.690717:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e63032550 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1937 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.690738:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000342a3832/0xaf35132e630324e0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [503808->733183] (req 503808->511999) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d18ce expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.690747:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000019780e8d/0xaf35132e6303249a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [733184->2412543] (req 733184->741375) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1888 expref: 9 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.690764:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e6303240e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [2412544->2424831] (req 2412544->2416639) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d17fc expref: 11 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.690769:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032454 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [2977792->36532223] (req 2977792->2990079) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1842 expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.690774:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e7fd0135/0xaf35132e6303250a lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [2838528->2977791] (req 2838528->2842623) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d18f8 expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.690778:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 5 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(51 130) 00010000:00010000:3.0:1780298641.690780:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(56 130) 00010000:00010000:3.0:1780298641.690783:0:20252:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e63032550 lrc: 8/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1937 expref: 11 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.690801:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e7fd0135/0xaf35132e6303250a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [2838528->2977791] (req 2838528->2842623) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d18f8 expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.690808:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e7fd0135/0xaf35132e6303250a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [2838528->2977791] (req 2838528->2842623) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d18f8 expref: 9 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.690816:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242860000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604824448:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298641.690853:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032454 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [2977792->36532223] (req 2977792->2990079) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1842 expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.690869:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032454 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [2977792->36532223] (req 2977792->2990079) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1842 expref: 9 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.690876:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242863100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604824576:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298641.690899:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e6303240e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [2412544->2424831] (req 2412544->2416639) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d17fc expref: 11 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.690906:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e6303240e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [2412544->2424831] (req 2412544->2416639) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d17fc expref: 11 pid: 19992 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.690912:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242861f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604824704:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298641.690987:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000019780e8d/0xaf35132e6303249a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [733184->2412543] (req 733184->741375) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1888 expref: 9 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.690995:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000019780e8d/0xaf35132e6303249a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [733184->2412543] (req 733184->741375) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1888 expref: 9 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.691003:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242860700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604824832:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298641.691082:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000342a3832/0xaf35132e630324e0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [503808->733183] (req 503808->511999) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d18ce expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.691093:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000342a3832/0xaf35132e630324e0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [503808->733183] (req 503808->511999) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d18ce expref: 9 pid: 19992 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.691116:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604824960:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298641.692348:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.692363:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.692379:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.692466:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14905 00000100:00100000:0.0:1780298641.692478:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bc380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609293696:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298641.692486:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298641.692501:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e6303240e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 9 type: EXT [2412544->2424831] (req 2412544->2416639) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d17fc expref: 11 pid: 19992 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.692522:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e6303240e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 8 type: EXT [2412544->2424831] (req 2412544->2416639) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d17fc expref: 11 pid: 19992 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.692537:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e6303240e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 10 type: EXT [2412544->2424831] (req 2412544->2416639) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d17fc expref: 11 pid: 19992 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.692561:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298641.692590:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bc380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609293696:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 113us (238us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298641.692612:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14905 00000100:00100000:3.0:1780298641.692613:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.692632:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609293824 00000100:00100000:3.0:1780298641.692636:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609293824 00010000:00010000:3.0:1780298641.692651:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e7fd0135/0xaf35132e6303250a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [2838528->2977791] (req 2838528->2842623) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d18f8 expref: 10 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.692683:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14908 00000100:00100000:3.0:1780298641.692699:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609293824:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298641.692709:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298641.692720:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e7fd0135/0xaf35132e6303250a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [2838528->2977791] (req 2838528->2842623) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d18f8 expref: 8 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.692732:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e7fd0135/0xaf35132e6303250a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [2838528->2977791] (req 2838528->2842623) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d18f8 expref: 8 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.692750:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7fd0135/0xaf35132e6303250a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [2838528->2977791] (req 2838528->2842623) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d18f8 expref: 8 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.692762:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298641.692772:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609294080 00000100:00100000:0.0:1780298641.692776:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609294080 00000100:00100000:3.0:1780298641.692783:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609293824:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 91us (175us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298641.692789:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14908 00002000:00100000:0.0:1780298641.692798:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (503808->511999) req@ffff9cf2558f1500 x1866777609294080/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 8680/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.692805:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2558f1500 with x1866777609294080 ext(503808->511999) 00010000:00010000:0.0:1780298641.692810:0:8362:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000342a3832/0xaf35132e630324e0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [503808->733183] (req 503808->511999) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d18ce expref: 7 pid: 19992 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.692821:0:8362:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000342a3832/0xaf35132e630324e0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [503808->733183] (req 503808->511999) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d18ce expref: 7 pid: 19992 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1780298641.692827:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9cf2558f1500 00000100:00100000:0.0:1780298641.692844:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 177 00000100:00100000:0.0:1780298641.692851:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609294080:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:0.0:1780298641.693282:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609294080:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 432us (538us total) trans 4294969151 rc 0/0 00000100:00100000:0.0:1780298641.693287:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 177 00002000:00100000:0.0:1780298641.693292:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (503808->511999) req@ffff9cf2558f1500 x1866777609294080/t4294969151(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 8680/448 e 0 to 0 dl 1780298652 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.693310:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2558f1500 with x1866777609294080 ext(503808->511999) 00010000:00010000:0.0:1780298641.693314:0:8362:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000342a3832/0xaf35132e630324e0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [503808->733183] (req 503808->511999) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d18ce expref: 9 pid: 19992 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.693326:0:8362:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000342a3832/0xaf35132e630324e0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [503808->733183] (req 503808->511999) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d18ce expref: 9 pid: 19992 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1780298641.693330:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9cf2558f1500 00000100:00100000:0.0:1780298641.694031:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609294208 00000100:00100000:0.0:1780298641.694034:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609294208 00010000:00010000:0.0:1780298641.694047:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000342a3832/0xaf35132e630324e0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [503808->733183] (req 503808->511999) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d18ce expref: 9 pid: 19992 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.694070:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14909 00000100:00100000:0.0:1780298641.694078:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f3800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609294208:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298641.694087:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:0.0:1780298641.694093:0:8374:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 128 -> 144 00010000:00010000:0.0:1780298641.694097:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000342a3832/0xaf35132e630324e0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [503808->733183] (req 503808->511999) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d18ce expref: 9 pid: 19992 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.694103:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000342a3832/0xaf35132e630324e0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [503808->733183] (req 503808->511999) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d18ce expref: 9 pid: 19992 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.694112:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000342a3832/0xaf35132e630324e0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [503808->733183] (req 503808->511999) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d18ce expref: 9 pid: 19992 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.694119:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(56 130) 00010000:00010000:0.0:1780298641.694153:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(56 130) 00010000:00010000:0.0:1780298641.694159:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e63032550 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1937 expref: 9 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.694165:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e63032550 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1937 expref: 9 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.694184:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e63032550 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1937 expref: 9 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.694193:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bc000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:8374:1866777604825088:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298641.694236:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733fb600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.694908:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bc000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:8374:1866777604825088:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298641.694919:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298641.694946:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f3800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609294208:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 868us (944us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298641.694953:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14909 00000100:00100000:3.0:1780298641.695098:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.695141:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.695165:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609294464 00000100:00100000:0.0:1780298641.695170:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609294464 00002000:00010000:0.0:1780298641.695189:0:19943:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00002000:00100000:0.0:1780298641.695192:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24519fb80 with x1866777609294464 ext(0->18446744073709551615) 00002000:00010000:0.0:1780298641.695200:0:19943:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24519fb80. 00000100:00100000:0.0:1780298641.695218:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5851 00000100:00100000:0.0:1780298641.695224:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609294464:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:0.0:1780298641.695827:0:19943:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 3413913 -> 0 00002000:00010000:0.0:1780298641.695830:0:19943:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298641.695832:0:19943:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298641.695834:0:19943:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298641.695836:0:19943:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 144 -> 0 00000100:00100000:0.0:1780298641.695868:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609294464:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 643us (727us total) trans 4294969152 rc 0/0 00000100:00100000:0.0:1780298641.695873:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5851 00002000:00010000:0.0:1780298641.695876:0:19943:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00002000:00100000:0.0:1780298641.695878:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24519fb80 with x1866777609294464 ext(0->18446744073709551615) 00002000:00010000:0.0:1780298641.695882:0:19943:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24519fb80. 00000100:00100000:3.0:1780298641.698887:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.702246:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609294848 00000100:00100000:3.0:1780298641.702252:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609294848 00002000:00100000:3.0:1780298641.702274:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (94208->98303) req@ffff9cf2558f1880 x1866777609294848/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:347/0 lens 488/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298641.702284:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2558f1880 with x1866777609294848 ext(94208->98303) 00002000:00010000:3.0:1780298641.702287:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2558f1880 00000100:00100000:3.0:1780298641.702319:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5854 00000100:00100000:3.0:1780298641.702327:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609294848:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298641.702413:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609294848:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 87us (199us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1780298641.702421:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5854 00002000:00100000:3.0:1780298641.702429:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (94208->98303) req@ffff9cf2558f1880 x1866777609294848/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:347/0 lens 488/4536 e 0 to 0 dl 1780298652 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298641.702439:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2558f1880 with x1866777609294848 ext(94208->98303) 00002000:00010000:3.0:1780298641.702443:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2558f1880 00000100:00100000:3.0:1780298641.703209:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.703237:0:9976:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609294976 00000100:00100000:0.0:1780298641.703242:0:9976:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609294976 00000100:00100000:0.0:1780298641.703269:0:9976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61885 00000100:00100000:0.0:1780298641.703275:0:9976:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+60:29360:x1866777609294976:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.703283:0:9976:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.703304:0:9976:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ac00/0xaf35132e63032565 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 39 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1945 expref: -99 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.703358:0:9976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b600/0xaf35132e6303256c lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 40 type: IBT flags: 0x40000000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.703368:0:9976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b600/0xaf35132e6303256c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 40 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.703375:0:9976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b600/0xaf35132e6303256c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 40 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1780298641.703388:0:9976:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 217337 00000004:00200000:0.0:1780298641.703391:0:9976:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1780298641.703401:0:9976:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b600/0xaf35132e6303256c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 40 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.703419:0:9976:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ac00/0xaf35132e63032565 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 40 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1945 expref: 62 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.703430:0:9976:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b600/0xaf35132e6303256c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 39 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1945 expref: 61 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.703438:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298641.703441:0:9976:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1780298641.703444:0:9976:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b600, rc 0) 00000100:00100000:0.0:1780298641.703474:0:9976:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+61:29360:x1866777609294976:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 199us (266us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298641.703482:0:9976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61885 00000100:00100000:0.0:1780298641.704311:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609295104 00000100:00100000:0.0:1780298641.704316:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609295104 00000100:00100000:0.0:1780298641.704342:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609295232 00000100:00100000:0.0:1780298641.704344:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609295232 00000100:00100000:3.0:1780298641.704360:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8529 00000100:00100000:0.0:1780298641.704371:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8530 00000100:00100000:3.0:1780298641.704373:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34831bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609295104:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.704382:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298641.704382:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242860700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609295232:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.704390:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.704394:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005c47b04a/0xaf35132e63032573 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d194c expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.704399:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e6303257a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1953 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.704458:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604825216:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.704458:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604825344:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.704525:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.704525:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733fb600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.705253:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604825344:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.705274:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e6303257a lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1953 expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.705277:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604825216:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.705306:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e6303257a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1953 expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.705314:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005c47b04a/0xaf35132e63032573 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d194c expref: 8 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.705316:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b000, rc 0) 00010000:00010000:3.0:1780298641.705332:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005c47b04a/0xaf35132e63032573 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d194c expref: 8 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.705341:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242860700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609295232:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 963us (1031us total) trans 0 rc 301/301 00010000:00010000:3.0:1780298641.705344:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3400, rc 0) 00000100:00100000:0.0:1780298641.705348:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8530 00000100:00100000:3.0:1780298641.705373:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34831bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609295104:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1004us (1104us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.705381:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8529 00000100:00100000:3.0:1780298641.706369:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609295360 00000100:00100000:3.0:1780298641.706373:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609295360 00000100:00100000:3.0:1780298641.706399:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8531 00000100:00100000:3.0:1780298641.706406:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609295360:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.706412:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.706420:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e63032581 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d195a expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.706443:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604825472:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.706520:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.707078:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604825472:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.707089:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e63032581 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d195a expref: 8 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.707170:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e63032581 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d195a expref: 8 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.707179:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3e00, rc 0) 00000100:00100000:3.0:1780298641.707206:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609295360:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 802us (879us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.707211:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8531 00000100:00100000:0.0:1780298641.708158:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.709519:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609295744 00000100:00100000:3.0:1780298641.709522:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609295744 00000100:00100000:3.0:1780298641.709548:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5856 00000100:00100000:3.0:1780298641.709552:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b69c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609295744:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:3.0:1780298641.709651:0:8363:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 217337 -> 217337 00002000:00010000:3.0:1780298641.709654:0:8363:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:3.0:1780298641.709655:0:8363:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:3.0:1780298641.709657:0:8363:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:3.0:1780298641.709658:0:8363:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 288 -> 296 00000100:00100000:3.0:1780298641.709683:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b69c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609295744:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 130us (194us total) trans 4294969575 rc 0/0 00000100:00100000:3.0:1780298641.709687:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5856 00000100:00100000:0.0:1780298641.710404:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609295872 00000100:00100000:0.0:1780298641.710408:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609295872 00000100:00100000:0.0:1780298641.710438:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8533 00000100:00100000:0.0:1780298641.710447:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b6b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609295872:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298641.710453:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8534 00010000:00010000:0.0:1780298641.710454:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298641.710462:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b69880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609296000:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.710464:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e6303258f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1968 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.710469:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.710478:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000010b6573d/0xaf35132e63032596 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d196f expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.710495:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7be680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604825728:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.710517:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604825856:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.710540:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733fb600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.710543:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.711281:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604825856:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.711307:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000010b6573d/0xaf35132e63032596 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d196f expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.711322:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000010b6573d/0xaf35132e63032596 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d196f expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.711337:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3a00, rc 0) 00000100:00100000:3.0:1780298641.711377:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b69880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609296000:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 914us (984us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.711385:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8534 00000100:00100000:0.0:1780298641.712241:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609296256 00000100:00100000:0.0:1780298641.712245:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609296256 00000100:00100000:3.0:1780298641.712257:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609296128 00000100:00100000:3.0:1780298641.712261:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609296128 00000100:00100000:0.0:1780298641.712272:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8535 00000100:00100000:0.0:1780298641.712279:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b6a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609296256:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.712286:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.712294:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b0cbb9d/0xaf35132e6303259d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d197d expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.712305:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8536 00000100:00100000:3.0:1780298641.712316:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b6aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609296128:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.712322:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.712330:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e630325a4 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1976 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.712334:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604825984:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.712359:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604826112:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.712377:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733fb600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.712385:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.713035:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604825984:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.713048:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604826112:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.713057:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003b0cbb9d/0xaf35132e6303259d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d197d expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.713064:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e630325a4 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1976 expref: 8 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.713077:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b0cbb9d/0xaf35132e6303259d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d197d expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.713079:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e630325a4 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1976 expref: 8 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.713096:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3c00, rc 0) 00010000:00010000:0.0:1780298641.713099:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b800, rc 0) 00000100:00100000:3.0:1780298641.713140:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b6aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609296128:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 811us (891us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.713145:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b6a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609296256:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 845us (912us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.713146:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8536 00000100:00100000:0.0:1780298641.713155:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8535 00000100:00100000:0.0:1780298641.714205:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609296384 00000100:00100000:0.0:1780298641.714209:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609296384 00000100:00100000:0.0:1780298641.714237:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5857 00000100:00100000:0.0:1780298641.714243:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b6b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609296384:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298641.714340:0:19951:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 217337 -> 217337 00002000:00010000:0.0:1780298641.714342:0:19951:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298641.714345:0:19951:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298641.714346:0:19951:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298641.714347:0:19951:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 296 -> 296 00000100:00100000:0.0:1780298641.714369:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b6b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609296384:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 127us (192us total) trans 4294969576 rc 0/0 00000100:00100000:0.0:1780298641.714375:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5857 00000100:00100000:0.0:1780298641.715847:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609296512 00000100:00100000:0.0:1780298641.715851:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609296512 00002000:00100000:0.0:1780298641.715870:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (0->221183) req@ffff9cf242b69f80 x1866777609296512/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:347/0 lens 504/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.715877:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b69f80 with x1866777609296512 ext(0->221183) 00002000:00010000:0.0:1780298641.715881:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf242b69f80 00000100:00100000:0.0:1780298641.715896:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5858 00000100:00100000:0.0:1780298641.715900:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b69f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609296512:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298641.719337:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b69f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609296512:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 3431us (3517us total) trans 0 rc 212992/212992 00000100:00100000:0.0:1780298641.719347:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5858 00002000:00100000:0.0:1780298641.719356:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (0->221183) req@ffff9cf242b69f80 x1866777609296512/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:347/0 lens 504/440 e 0 to 0 dl 1780298652 ref 1 fl Complete:/600/0 rc 212992/212992 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.719366:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b69f80 with x1866777609296512 ext(0->221183) 00002000:00010000:0.0:1780298641.719370:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf242b69f80 00000100:00100000:3.0:1780298641.721468:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609296640 00000100:00100000:3.0:1780298641.721476:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609296640 00000100:00100000:3.0:1780298641.721520:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5859 00000100:00100000:3.0:1780298641.721529:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242861f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609296640:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:3.0:1780298641.721687:0:16972:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:3.0:1780298641.721691:0:16972:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:3.0:1780298641.721694:0:16972:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:3.0:1780298641.721696:0:16972:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:3.0:1780298641.721699:0:16972:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 0 -> 8 00000100:00100000:3.0:1780298641.721733:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242861f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2399:x1866777609296640:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 205us (315us total) trans 4294969153 rc 0/0 00000100:00100000:3.0:1780298641.721742:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5859 00000100:00100000:0.0:1780298641.723268:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609296768 00000100:00100000:0.0:1780298641.723273:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609296768 00000100:00100000:0.0:1780298641.723311:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8537 00000100:00100000:0.0:1780298641.723321:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b6bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609296768:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.723332:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.723346:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003b0cbb9d/0xaf35132e630325ab lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1984 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.723406:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604826240:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.723444:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609296896 00000100:00100000:3.0:1780298641.723449:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609296896 00000100:00100000:3.0:1780298641.723475:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8538 00000100:00100000:0.0:1780298641.723475:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733fb600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.723499:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e60e450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609296896:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.723508:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.723517:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e630325b2 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d198b expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.723547:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604826368:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.723620:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.724228:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604826240:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.724252:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003b0cbb9d/0xaf35132e630325ab lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1984 expref: 8 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.724269:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604826368:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.724274:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003b0cbb9d/0xaf35132e630325ab lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1984 expref: 8 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.724286:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e630325b2 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d198b expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.724288:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b800, rc 0) 00010000:00010000:3.0:1780298641.724313:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e630325b2 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d198b expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.724321:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3c00, rc 0) 00000100:00100000:0.0:1780298641.724336:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b6bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609296768:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1016us (1106us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.724343:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e60e450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609296896:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 846us (985us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.724349:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8537 00000100:00100000:3.0:1780298641.724359:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8538 00000100:00100000:3.0:1780298641.725217:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609297024 00000100:00100000:3.0:1780298641.725220:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609297024 00000100:00100000:3.0:1780298641.725242:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8539 00000100:00100000:3.0:1780298641.725248:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b68a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609297024:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.725255:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.725262:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e630325b9 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1992 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.725293:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604826496:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.725339:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609297152 00000100:00100000:3.0:1780298641.725343:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393000 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.725344:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609297152 00000100:00100000:0.0:1780298641.725375:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8540 00000100:00100000:0.0:1780298641.725382:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242863100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609297152:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.725389:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.725396:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e630325c0 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1999 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.725415:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604826624:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.725444:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733fb600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.725859:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604826496:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.725870:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e630325b9 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1992 expref: 8 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.725879:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e630325b9 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1992 expref: 8 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.725924:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3a00, rc 0) 00000100:00100000:3.0:1780298641.725945:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b68a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609297024:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 698us (790us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.725951:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8539 00000100:00100000:0.0:1780298641.726030:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604826624:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.726045:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e630325c0 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1999 expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.726059:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e630325c0 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1999 expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.726069:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b000, rc 0) 00000100:00100000:0.0:1780298641.726095:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242863100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609297152:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 714us (807us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.726100:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8540 00000100:00100000:3.0:1780298641.728315:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.728330:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609297536 00000100:00100000:0.0:1780298641.728334:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609297536 00000100:00100000:0.0:1780298641.728357:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8542 00000100:00100000:0.0:1780298641.728363:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609297536:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.728370:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.728378:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e630325ce lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d19a7 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.728405:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604826880:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.728432:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733fb600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.729103:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604826880:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.729118:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e630325ce lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d19a7 expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.729158:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e630325ce lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d19a7 expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.729169:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9ac00, rc 0) 00000100:00100000:0.0:1780298641.729195:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609297536:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 832us (886us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.729203:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8542 00000100:00100000:0.0:1780298641.730116:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609297792 00000100:00100000:0.0:1780298641.730121:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609297792 00000100:00100000:0.0:1780298641.730172:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8544 00000100:00100000:0.0:1780298641.730182:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242862680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609297792:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.730190:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.730198:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad103424/0xaf35132e630325dc lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d19b5 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.730223:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604827136:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.730265:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733fb600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.730843:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604827136:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.730858:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ad103424/0xaf35132e630325dc lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d19b5 expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.730867:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad103424/0xaf35132e630325dc lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d19b5 expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.730877:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9aa00, rc 0) 00000100:00100000:0.0:1780298641.730899:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242862680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609297792:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 720us (822us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.730904:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8544 00000100:00100000:3.0:1780298641.733754:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609298176 00000100:00100000:3.0:1780298641.733794:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609298176 00010000:00010000:3.0:1780298641.733812:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244f3ac00/0xaf35132e6303255e lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 39 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d193e expref: 82 pid: 7411 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.733832:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14910 00000100:00100000:3.0:1780298641.733837:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b1e450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+82:2399:x1866777609298176:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298641.733844:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298641.733848:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244f3ac00/0xaf35132e6303255e lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 40 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d193e expref: 82 pid: 7411 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.733864:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244f3ac00/0xaf35132e6303255e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 40 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d193e expref: 82 pid: 7411 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.733874:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244f3ac00/0xaf35132e6303255e lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 40 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d193e expref: 82 pid: 7411 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.733882:0:6462:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298641.733885:0:6462:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36000/0xaf35132e630325e3 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 39 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.733892:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36000/0xaf35132e630325e3 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 39 type: IBT flags: 0x40290000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.733897:0:6462:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36000/0xaf35132e630325e3 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 39 type: IBT flags: 0x40290000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.733911:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298641.733933:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b1e450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+81:2399:x1866777609298176:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 95us (201us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298641.733939:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14910 00000100:00100000:3.0:1780298641.734703:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609298304 00000100:00100000:3.0:1780298641.734708:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609298304 00000100:00100000:3.0:1780298641.734739:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8545 00000100:00100000:3.0:1780298641.734746:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609298304:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.734755:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.734766:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e630325ea lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d19bc expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.734797:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e63032549 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1930 expref: 13 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.734806:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(71 177) 00010000:00010000:3.0:1780298641.734810:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(72 177) 00010000:00010000:3.0:1780298641.734815:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e630325ea lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d19bc expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.734834:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e63032549 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1930 expref: 13 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.734844:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e63032549 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1930 expref: 13 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.734854:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604827392:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298641.734895:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.735419:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604827392:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298641.735434:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e630325ea lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d19bc expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.735447:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3e00, rc 0) 00000100:00100000:3.0:1780298641.735476:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:29360:x1866777609298304:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 730us (799us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298641.735492:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8545 00000100:00100000:3.0:1780298641.736774:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.736804:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609298560 00000100:00100000:0.0:1780298641.736808:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609298560 00000100:00100000:0.0:1780298641.736832:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8546 00000100:00100000:0.0:1780298641.736838:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:29360:x1866777609298560:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.736845:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.736852:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e630325f1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d19c3 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.736866:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e63032550 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1937 expref: 11 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.736872:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(56 130) 00010000:00010000:0.0:1780298641.736875:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(57 130) 00010000:00010000:0.0:1780298641.736878:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e630325f1 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d19c3 expref: 9 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.736891:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e63032550 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1937 expref: 11 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.736897:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e63032550 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1937 expref: 11 pid: 20252 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.736904:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604827648:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298641.736932:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733fb600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.737399:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604827648:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298641.737409:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e630325f1 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d19c3 expref: 9 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.737416:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b000, rc 0) 00000100:00100000:0.0:1780298641.737435:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:29360:x1866777609298560:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 599us (664us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298641.737440:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8546 00000100:00100000:3.0:1780298641.737679:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609298688 00000100:00100000:3.0:1780298641.737684:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609298688 00010000:00010000:3.0:1780298641.737701:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e63032550 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1937 expref: 12 pid: 20252 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.737730:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14912 00000100:00100000:3.0:1780298641.737738:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2399:x1866777609298688:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298641.737748:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298641.737760:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e63032550 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1937 expref: 12 pid: 20252 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.737770:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e63032550 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1937 expref: 12 pid: 20252 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.737783:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e63032550 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1937 expref: 12 pid: 20252 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.737791:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(57 130) 00010000:00010000:3.0:1780298641.737795:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(57 130) 00010000:00010000:3.0:1780298641.737799:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e630325f1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d19c3 expref: 8 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.737807:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e630325f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d19c3 expref: 8 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.737834:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e630325f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d19c3 expref: 8 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.737843:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519c700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6461:1866777604827776:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298641.737876:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.738473:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519c700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6461:1866777604827776:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298641.738482:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298641.738506:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609298688:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 769us (850us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298641.738514:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14912 00000100:00100000:3.0:1780298641.738669:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609298816 00000100:00100000:3.0:1780298641.738673:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609298816 00000100:00100000:0.0:1780298641.738712:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00002000:00010000:3.0:1780298641.738715:0:8363:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:3.0:1780298641.738719:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24310e680 with x1866777609298816 ext(4194304->18446744073709551615) 00002000:00010000:3.0:1780298641.738725:0:8363:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24310e680. 00000100:00100000:3.0:1780298641.738743:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609298944 00000100:00100000:3.0:1780298641.738745:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609298944 00002000:00010000:3.0:1780298641.738753:0:8363:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (201159->18446744073709551615) 00002000:00100000:3.0:1780298641.738755:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f7bdf80 with x1866777609298944 ext(201159->18446744073709551615) 00002000:00010000:3.0:1780298641.738759:0:8363:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f7bdf80. 00000100:00100000:0.0:1780298641.739818:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609299072 00000100:00100000:0.0:1780298641.739821:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609299072 00000100:00100000:0.0:1780298641.739845:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61887 00000100:00100000:0.0:1780298641.739850:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+63:29360:x1866777609299072:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.739859:0:6477:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.739868:0:6477:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9aa00/0xaf35132e630325f8 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 39 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d19ca expref: -99 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.739920:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ac00/0xaf35132e630325ff lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 40 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.739929:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ac00/0xaf35132e630325ff lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 40 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.739935:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ac00/0xaf35132e630325ff lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 40 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1780298641.739950:0:6477:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 4395463 00000004:00200000:0.0:1780298641.739953:0:6477:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1780298641.739970:0:6477:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ac00/0xaf35132e630325ff lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 40 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.739985:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9aa00/0xaf35132e630325f8 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 40 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d19ca expref: 65 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.739996:0:6477:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ac00/0xaf35132e630325ff lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 39 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d19ca expref: 64 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.740001:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298641.740003:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1780298641.740005:0:6477:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9ac00, rc 0) 00000100:00100000:0.0:1780298641.740025:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+64:29360:x1866777609299072:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 176us (246us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298641.740031:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61887 00000100:00100000:3.0:1780298641.741716:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.742817:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.742844:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609299456 00000100:00100000:0.0:1780298641.742848:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609299456 00002000:00010000:0.0:1780298641.742861:0:19950:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4258927->18446744073709551615) 00002000:00100000:0.0:1780298641.742864:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24519c700 with x1866777609299456 ext(4258927->18446744073709551615) 00002000:00010000:0.0:1780298641.742868:0:19950:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24519c700. 00000100:00100000:0.0:1780298641.742885:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5863 00000100:00100000:0.0:1780298641.742890:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609299456:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:3.0:1780298641.742932:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00002000:00010000:0.0:1780298641.742978:0:19950:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4258927 00002000:00010000:0.0:1780298641.742981:0:19950:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298641.742984:0:19950:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298641.742988:0:19950:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298641.742990:0:19950:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 304 -> 304 00000100:00100000:0.0:1780298641.743018:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609299456:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 127us (199us total) trans 4294969579 rc 0/0 00000100:00100000:0.0:1780298641.743030:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5863 00002000:00010000:0.0:1780298641.743034:0:19950:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4258927->18446744073709551615) 00002000:00100000:0.0:1780298641.743036:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24519c700 with x1866777609299456 ext(4258927->18446744073709551615) 00002000:00010000:0.0:1780298641.743039:0:19950:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24519c700. 00000100:00100000:3.0:1780298641.744371:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.745913:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609299840 00000100:00100000:0.0:1780298641.745918:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609299840 00002000:00100000:0.0:1780298641.745939:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (4255744->4259839) req@ffff9cf24310ea00 x1866777609299840/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 488/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.745949:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24310ea00 with x1866777609299840 ext(4255744->4259839) 00002000:00010000:0.0:1780298641.745952:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24310ea00 00000100:00100000:0.0:1780298641.745974:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5866 00000100:00100000:0.0:1780298641.745982:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609299840:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298641.746056:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609299840:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 75us (183us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298641.746064:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5866 00002000:00100000:0.0:1780298641.746069:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (4255744->4259839) req@ffff9cf24310ea00 x1866777609299840/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 488/4536 e 0 to 0 dl 1780298652 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.746078:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24310ea00 with x1866777609299840 ext(4255744->4259839) 00002000:00010000:0.0:1780298641.746081:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24310ea00 00000100:00100000:3.0:1780298641.747364:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.748467:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000001:00080000:0.0:1780298641.749372:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969580 is committed 00000001:00080000:0.0:1780298641.749385:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969577 is committed 00000100:00100000:0.0:1780298641.750200:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.751921:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.754615:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.755921:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.755961:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609300736 00000100:00100000:0.0:1780298641.755966:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609300736 00000100:00100000:0.0:1780298641.755996:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8548 00000100:00100000:0.0:1780298641.756003:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348004700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609300736:12345-192.168.203.25@tcp:16:fsx.0 00000100:00100000:0.0:1780298641.756809:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348004700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609300736:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 805us (887us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298641.756816:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8548 00000100:00100000:0.0:1780298641.757611:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.758845:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609300992 00000100:00100000:3.0:1780298641.758850:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609300992 00000100:00100000:0.0:1780298641.758882:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.758910:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609301120 00000100:00100000:3.0:1780298641.758913:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609301120 00000100:00100000:0.0:1780298641.758918:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8549 00000100:00100000:0.0:1780298641.758927:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609300992:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.758936:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.758947:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63032629 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d19df expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.758991:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bf800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604828032:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.759055:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733fb600 going to sleep for 11 seconds 00002000:00010000:0.0:1780298641.759884:0:12101:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks: 304 -> 336 00000100:00100000:0.0:1780298641.759892:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bf800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604828032:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.759902:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63032629 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d19df expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.759915:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63032629 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d19df expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.759925:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9aa00, rc 0) 00000100:00100000:0.0:1780298641.759952:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609300992:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1026us (1176us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.759964:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8549 00000100:00100000:3.0:1780298641.760951:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.760969:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.761061:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8552 00000100:00100000:0.0:1780298641.761072:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609301376:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.761079:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.761087:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e6303263e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d19f4 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.761119:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bf800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604828288:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.761166:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733fb600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.761848:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24310ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604828160:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.761862:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bf800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604828288:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.761872:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009856b175/0xaf35132e63032637 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d19ed expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.761878:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e6303263e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d19f4 expref: 12 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.761898:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009856b175/0xaf35132e63032637 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d19ed expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.761903:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e6303263e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d19f4 expref: 12 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.761907:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244f3b200, rc 0) 00010000:00010000:0.0:1780298641.761913:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9ac00, rc 0) 00000100:00100000:3.0:1780298641.761938:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348006680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2397:x1866777609301248:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 894us (986us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.761944:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609301376:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 873us (970us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.761947:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8551 00000100:00100000:0.0:1780298641.761955:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8552 00000100:00100000:3.0:1780298641.762961:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.762999:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609301632 00000100:00100000:0.0:1780298641.763002:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609301632 00000100:00100000:0.0:1780298641.763037:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61891 00000100:00100000:0.0:1780298641.763043:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433ddc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+67:29360:x1866777609301632:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:0.0:1780298641.763074:0:6477:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298641.763087:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244f3a400/0xaf35132e63032614 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 42 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d19d1 expref: 67 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.763097:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298641.763100:0:6477:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1780298641.763232:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b600/0xaf35132e63032645 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 41 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.763242:0:6477:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36800/0xaf35132e63032622 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 41 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d19d8 expref: 81 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.763252:0:6477:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b600/0xaf35132e63032645 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 41 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.763272:0:6477:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36800/0xaf35132e63032622 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 41 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d19d8 expref: 81 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.763281:0:6477:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36800/0xaf35132e63032622 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 41 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d19d8 expref: 81 pid: 6476 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.763289:0:6477:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bf800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6477:1866777604828416:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298641.763332:0:6477:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.763958:0:6477:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bf800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6477:1866777604828416:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298641.763969:0:6477:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b600/0xaf35132e63032645 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 41 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.764055:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:0.0:1780298641.764244:0:6477:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b600/0xaf35132e63032645 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 40 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.764250:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b600/0xaf35132e63032645 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 40 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1780298641.764287:0:6477:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294990980 00000004:00010000:0.0:1780298641.764315:0:6477:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 000000001f54057d reply state 0000000053bf2545 transno 4294990981 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b600/0xaf35132e63032645 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 40 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.764324:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b600/0xaf35132e63032645 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 40 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.764328:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298641.764330:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1780298641.764332:0:6477:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1780298641.764375:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433ddc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+69:29360:x1866777609301632:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1332us (1415us total) trans 4294990981 rc 0/0 00000100:00100000:0.0:1780298641.764381:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61891 00000020:00080000:0.0:1780298641.764384:0:6477:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1780298641.765080:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.765174:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609301888 00000100:00100000:0.0:1780298641.765178:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609301888 00002000:00010000:0.0:1780298641.765194:0:19928:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:0.0:1780298641.765200:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2433de680 with x1866777609301888 ext(4194304->18446744073709551615) 00000100:00100000:3.0:1780298641.765201:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00002000:00010000:0.0:1780298641.765207:0:19928:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2433de680. 00000100:00100000:0.0:1780298641.765249:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609302016 00000100:00100000:0.0:1780298641.765251:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609302016 00002000:00010000:0.0:1780298641.765262:0:19928:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (2411486->18446744073709551615) 00002000:00100000:0.0:1780298641.765266:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2433dc700 with x1866777609302016 ext(2411486->18446744073709551615) 00002000:00010000:0.0:1780298641.765272:0:19928:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2433dc700. 00000100:00100000:0.0:1780298641.765291:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5872 00000100:00100000:0.0:1780298641.765310:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609302016:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:0.0:1780298641.765417:0:19928:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4194304 -> 2411486 00002000:00010000:0.0:1780298641.765419:0:19928:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298641.765421:0:19928:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298641.765423:0:19928:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298641.765425:0:19928:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:0.0:1780298641.765449:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609302016:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 152us (252us total) trans 4294969156 rc 0/0 00000100:00100000:0.0:1780298641.765455:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5872 00002000:00010000:0.0:1780298641.765458:0:19928:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (2411486->18446744073709551615) 00002000:00100000:0.0:1780298641.765460:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2433dc700 with x1866777609302016 ext(2411486->18446744073709551615) 00002000:00010000:0.0:1780298641.765464:0:19928:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2433dc700. 00000100:00100000:0.0:1780298641.767579:0:9976:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609302272 00000100:00100000:0.0:1780298641.767584:0:9976:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609302272 00000100:00100000:0.0:1780298641.767615:0:9976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61893 00000100:00100000:0.0:1780298641.767623:0:9976:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+81:29360:x1866777609302272:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.767631:0:9976:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.767640:0:9976:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a200/0xaf35132e6303265a lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 42 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1a02 expref: -99 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.767678:0:9976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9be00/0xaf35132e63032661 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 43 type: IBT flags: 0x40000000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.767687:0:9976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9be00/0xaf35132e63032661 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 43 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.767695:0:9976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9be00/0xaf35132e63032661 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 43 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1780298641.767710:0:9976:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 6605790 00000004:00200000:0.0:1780298641.767713:0:9976:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1780298641.767723:0:9976:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9be00/0xaf35132e63032661 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 43 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.767740:0:9976:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a200/0xaf35132e6303265a lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 43 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1a02 expref: 83 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.767753:0:9976:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9be00/0xaf35132e63032661 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 42 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1a02 expref: 82 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.767759:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298641.767761:0:9976:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1780298641.767764:0:9976:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9be00, rc 0) 00000100:00100000:0.0:1780298641.767792:0:9976:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+82:29360:x1866777609302272:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 170us (256us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298641.767799:0:9976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61893 00000100:00100000:3.0:1780298641.768620:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.768670:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.768745:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8554 00000100:00100000:0.0:1780298641.768759:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34831a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609302528:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.768769:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.768779:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e6303266f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1a10 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.768818:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bf800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604828672:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.768853:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733fb600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.769512:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bf800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604828672:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.769529:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e6303266f lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1a10 expref: 12 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.769553:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e6303266f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1a10 expref: 12 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.769562:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9ac00, rc 0) 00000100:00100000:0.0:1780298641.769584:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34831a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609302528:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 827us (912us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.769599:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8554 00000100:00100000:3.0:1780298641.770439:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.770484:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609302656 00000100:00100000:0.0:1780298641.770488:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609302656 00000100:00100000:3.0:1780298641.770506:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.770522:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609302784 00000100:00100000:0.0:1780298641.770526:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609302784 00000100:00100000:0.0:1780298641.770554:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8556 00000100:00100000:0.0:1780298641.770566:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34831b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609302784:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.770578:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.770588:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad103424/0xaf35132e6303267d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1a1e expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.770623:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bf800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604828928:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.770655:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733fb600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.771237:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bf800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604828928:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.771258:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ad103424/0xaf35132e6303267d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1a1e expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.771268:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad103424/0xaf35132e6303267d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1a1e expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.771276:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9aa00, rc 0) 00000100:00100000:0.0:1780298641.771310:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34831b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609302784:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 750us (805us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.771315:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8556 00000100:00100000:3.0:1780298641.772189:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.772261:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.772330:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8558 00000100:00100000:0.0:1780298641.772345:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348318a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609303040:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.772355:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.772363:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b0cbb9d/0xaf35132e6303268b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1a2c expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.772395:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bf800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604829056:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.772429:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733fb600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.773212:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bf800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604829056:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.773226:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003b0cbb9d/0xaf35132e6303268b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1a2c expref: 12 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.773245:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b0cbb9d/0xaf35132e6303268b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1a2c expref: 12 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.773259:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b800, rc 0) 00000100:00100000:0.0:1780298641.773308:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348318a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609303040:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 967us (1046us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.773318:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8558 00000100:00100000:3.0:1780298641.774196:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.775243:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.775291:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609303296 00000100:00100000:0.0:1780298641.775303:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609303296 00010000:00010000:0.0:1780298641.775322:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e630325ea lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d19bc expref: 8 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.775349:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14914 00000100:00100000:0.0:1780298641.775357:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34831bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609303296:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298641.775366:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298641.775377:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e630325ea lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d19bc expref: 8 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.775394:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e630325ea lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d19bc expref: 8 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.775410:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e630325ea lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d19bc expref: 8 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.775421:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(73 177) 00010000:00010000:0.0:1780298641.775424:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(73 177) 00010000:00010000:0.0:1780298641.775431:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000097580845/0xaf35132e63032692 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2215936->2220031) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1a33 expref: 8 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.775440:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000097580845/0xaf35132e63032692 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2215936->2220031) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1a33 expref: 8 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.775463:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000097580845/0xaf35132e63032692 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2215936->2220031) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1a33 expref: 8 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.775473:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bf800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777604829440:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298641.775507:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733fb600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.776098:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bf800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777604829440:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298641.776109:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298641.776152:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34831bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609303296:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 796us (907us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298641.776161:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14914 00000100:00100000:3.0:1780298641.783052:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.795248:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609303424:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 12029us (12192us total) trans 0 rc 3973120/3973120 00000100:00100000:0.0:1780298641.795263:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5873 00002000:00100000:0.0:1780298641.795278:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (221184->4194303) req@ffff9cf2558f0700 x1866777609303424/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:347/0 lens 488/440 e 0 to 0 dl 1780298652 ref 1 fl Complete:/600/0 rc 3973120/3973120 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.795308:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2558f0700 with x1866777609303424 ext(221184->4194303) 00002000:00010000:0.0:1780298641.795316:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2558f0700 00000100:00100000:3.0:1780298641.797611:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.799931:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.801770:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.805857:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.805913:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609303936 00000100:00100000:0.0:1780298641.805923:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609303936 00010000:00010000:0.0:1780298641.805951:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000097580845/0xaf35132e63032692 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2215936->2220031) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1a33 expref: 9 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.805989:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14915 00000100:00100000:0.0:1780298641.806000:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f3b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609303936:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298641.806016:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298641.806043:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000097580845/0xaf35132e63032692 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2215936->2220031) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1a33 expref: 9 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.806054:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000097580845/0xaf35132e63032692 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2215936->2220031) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1a33 expref: 9 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.806078:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000097580845/0xaf35132e63032692 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2215936->2220031) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1a33 expref: 9 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.806090:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(73 177) 00010000:00010000:0.0:1780298641.806093:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(73 177) 00010000:00010000:0.0:1780298641.806099:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a860e235/0xaf35132e630326a7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3244032->36798463] (req 3244032->3256319) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1a48 expref: 7 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.806110:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a860e235/0xaf35132e630326a7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3244032->36798463] (req 3244032->3256319) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1a48 expref: 7 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.806154:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000a860e235/0xaf35132e630326a7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3244032->36798463] (req 3244032->3256319) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1a48 expref: 7 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.806164:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bc700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777604829952:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298641.806218:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733fb600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.807173:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bc700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777604829952:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298641.807192:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298641.807230:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f3b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609303936:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1231us (1390us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298641.807239:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14915 00000100:00100000:3.0:1780298641.807285:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.808468:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.810427:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609304320 00000100:00100000:3.0:1780298641.810431:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609304320 00000100:00100000:3.0:1780298641.810455:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8564 00000100:00100000:3.0:1780298641.810461:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609304320:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.810467:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.810478:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e630326b5 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1a56 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.810505:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f1180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604830208:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.810537:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.812194:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.813887:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.814868:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.816101:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.816180:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609304832 00000100:00100000:3.0:1780298641.816186:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609304832 00002000:00100000:3.0:1780298641.816208:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (970752->974847) req@ffff9cf37f7be680 x1866777609304832/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 488/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298641.816234:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f7be680 with x1866777609304832 ext(970752->974847) 00002000:00010000:3.0:1780298641.816237:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f7be680 00000100:00100000:3.0:1780298641.816263:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5876 00000100:00100000:3.0:1780298641.816272:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7be680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609304832:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298641.816365:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7be680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609304832:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 95us (266us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1780298641.816371:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5876 00002000:00100000:3.0:1780298641.816376:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (970752->974847) req@ffff9cf37f7be680 x1866777609304832/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 488/4536 e 0 to 0 dl 1780298652 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298641.816382:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f7be680 with x1866777609304832 ext(970752->974847) 00002000:00010000:3.0:1780298641.816385:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f7be680 00000100:00100000:3.0:1780298641.817352:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.817765:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.819576:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.819598:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.819626:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609305344 00000100:00100000:3.0:1780298641.819631:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609305344 00000100:00100000:3.0:1780298641.819677:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8570 00000100:00100000:3.0:1780298641.819689:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609305344:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.819696:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.819705:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000010b6573d/0xaf35132e630326df lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1a80 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.819739:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f754a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604830848:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.819785:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.820475:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f754a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604830848:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.820498:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000010b6573d/0xaf35132e630326df lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1a80 expref: 12 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.820525:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000010b6573d/0xaf35132e630326df lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1a80 expref: 12 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.820543:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3a00, rc 0) 00000100:00100000:3.0:1780298641.820569:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609305344:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 884us (969us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.820578:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8570 00000100:00100000:0.0:1780298641.821403:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.821424:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609305472 00000100:00100000:3.0:1780298641.821428:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609305472 00000100:00100000:3.0:1780298641.821448:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8571 00000100:00100000:3.0:1780298641.821453:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:29360:x1866777609305472:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.821459:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.821466:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e630326e6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1a87 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.821483:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e630325f1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d19c3 expref: 11 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.821492:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(57 130) 00010000:00010000:3.0:1780298641.821496:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(58 130) 00010000:00010000:3.0:1780298641.821501:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e630326e6 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [458752->471039] (req 458752->471039) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1a87 expref: 12 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.821531:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e630325f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d19c3 expref: 11 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.821540:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e630325f1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d19c3 expref: 11 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.821548:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f754a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604830976:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298641.821575:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.822102:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f754a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604830976:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298641.822115:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e630326e6 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [458752->471039] (req 458752->471039) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1a87 expref: 12 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.822147:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3c00, rc 0) 00000100:00100000:3.0:1780298641.822175:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:29360:x1866777609305472:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 721us (771us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298641.822182:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8571 00000100:00100000:0.0:1780298641.822356:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.822379:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609305600 00000100:00100000:3.0:1780298641.822382:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609305600 00010000:00010000:3.0:1780298641.822396:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e630325f1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d19c3 expref: 12 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.822414:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14916 00000100:00100000:3.0:1780298641.822418:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609305600:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298641.822431:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298641.822438:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e630325f1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d19c3 expref: 12 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.822445:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e630325f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d19c3 expref: 12 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.822457:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e630325f1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d19c3 expref: 12 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.822464:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(58 130) 00010000:00010000:3.0:1780298641.822466:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(58 130) 00010000:00010000:3.0:1780298641.822473:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e630326e6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [458752->34013183] (req 458752->471039) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1a87 expref: 11 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.822482:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e630326e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [458752->34013183] (req 458752->471039) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1a87 expref: 11 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.822504:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e630326e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [458752->34013183] (req 458752->471039) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1a87 expref: 11 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.822513:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f754a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777604831104:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298641.822546:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.823239:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f754a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777604831104:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298641.823249:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:3.0:1780298641.823251:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298641.823330:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609305600:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 909us (971us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298641.823337:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14916 00000100:00100000:0.0:1780298641.824270:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.824393:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.826078:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.826097:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.826191:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8575 00000100:00100000:3.0:1780298641.826204:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255490e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609306112:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.826211:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.826219:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005c47b04a/0xaf35132e63032702 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1a9c expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.826251:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f754a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604831488:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.826307:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.827095:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f754a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604831488:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.827119:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005c47b04a/0xaf35132e63032702 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1a9c expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.827159:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005c47b04a/0xaf35132e63032702 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1a9c expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.827170:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3400, rc 0) 00000100:00100000:0.0:1780298641.828207:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.829421:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.830387:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.830408:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609306752 00000100:00100000:3.0:1780298641.830413:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609306752 00000100:00100000:3.0:1780298641.830435:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8576 00000100:00100000:3.0:1780298641.830439:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255491f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:29360:x1866777609306752:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.830446:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.830453:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e63032710 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1aaa expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.830474:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000a860e235/0xaf35132e630326a7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3244032->36798463] (req 3244032->3256319) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1a48 expref: 11 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.830481:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000991eb57d/0xaf35132e630326c3 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [966656->3244031] (req 966656->970751) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1a64 expref: 11 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.830485:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 2 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(73 177) 00010000:00010000:3.0:1780298641.830488:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(75 177) 00010000:00010000:3.0:1780298641.830490:0:20252:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e63032710 lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1aaa expref: 11 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.830502:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000991eb57d/0xaf35132e630326c3 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [966656->3244031] (req 966656->970751) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1a64 expref: 11 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.830508:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000991eb57d/0xaf35132e630326c3 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [966656->3244031] (req 966656->970751) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1a64 expref: 11 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.830514:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f754a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604831872:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298641.830548:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000a860e235/0xaf35132e630326a7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3244032->36798463] (req 3244032->3256319) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1a48 expref: 11 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.830564:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000a860e235/0xaf35132e630326a7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3244032->36798463] (req 3244032->3256319) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1a48 expref: 11 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.830581:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f755180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604832000:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298641.830605:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.831199:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f754a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604831872:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298641.831210:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f755180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604832000:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298641.831223:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e63032710 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1aaa expref: 11 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.831231:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3200, rc 0) 00000100:00100000:3.0:1780298641.831306:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255491f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:29360:x1866777609306752:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 815us (870us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298641.831314:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8576 00000100:00100000:3.0:1780298641.831425:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609306880 00000100:00100000:3.0:1780298641.831428:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609306880 00010000:00010000:3.0:1780298641.831440:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000a860e235/0xaf35132e630326a7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3244032->36798463] (req 3244032->3256319) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1a48 expref: 11 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.831458:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14918 00000100:00100000:3.0:1780298641.831463:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b6b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609306880:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298641.831469:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298641.831478:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000a860e235/0xaf35132e630326a7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3244032->36798463] (req 3244032->3256319) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1a48 expref: 11 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.831529:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000a860e235/0xaf35132e630326a7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3244032->36798463] (req 3244032->3256319) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1a48 expref: 11 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.831547:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a860e235/0xaf35132e630326a7 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3244032->36798463] (req 3244032->3256319) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1a48 expref: 11 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.831555:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298641.831562:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.831581:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b6b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609306880:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 117us (216us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298641.831586:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14918 00000100:00100000:3.0:1780298641.833408:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609307264 00000100:00100000:3.0:1780298641.833456:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609307264 00010000:00010000:3.0:1780298641.833469:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e630326e6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [458752->34013183] (req 458752->471039) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1a87 expref: 13 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.833487:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14920 00000100:00100000:3.0:1780298641.833492:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609307264:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298641.833497:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298641.833505:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e630326e6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [458752->34013183] (req 458752->471039) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1a87 expref: 13 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.833513:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e630326e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [458752->34013183] (req 458752->471039) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1a87 expref: 13 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.833522:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e630326e6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [458752->34013183] (req 458752->471039) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1a87 expref: 13 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.833529:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(59 130) 00010000:00010000:3.0:1780298641.833535:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(59 130) 00010000:00010000:3.0:1780298641.833539:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000003c8dab73/0xaf35132e63032717 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ab1 expref: 11 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.833545:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003c8dab73/0xaf35132e63032717 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ab1 expref: 11 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.833559:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000003c8dab73/0xaf35132e63032717 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ab1 expref: 11 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.833564:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f755180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:20253:1866777604832384:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298641.833591:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.834120:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f755180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:20253:1866777604832384:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298641.834207:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298641.834316:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609307264:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 822us (927us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298641.834330:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14920 00000100:00100000:0.0:1780298641.834431:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609307520 00000100:00100000:0.0:1780298641.834436:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609307520 00002000:00010000:0.0:1780298641.834452:0:19951:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00002000:00100000:0.0:1780298641.834456:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24310ed80 with x1866777609307520 ext(0->18446744073709551615) 00002000:00010000:0.0:1780298641.834460:0:19951:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24310ed80. 00000100:00100000:0.0:1780298641.834479:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5879 00000100:00100000:0.0:1780298641.834486:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609307520:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:0.0:1780298641.834602:0:19951:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 2411486 -> 0 00002000:00010000:0.0:1780298641.834604:0:19951:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298641.834606:0:19951:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298641.834607:0:19951:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298641.834609:0:19951:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:0.0:1780298641.834632:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2399:x1866777609307520:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 148us (236us total) trans 4294969158 rc 0/0 00000100:00100000:0.0:1780298641.834638:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5879 00002000:00010000:0.0:1780298641.834640:0:19951:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00002000:00100000:0.0:1780298641.834642:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24310ed80 with x1866777609307520 ext(0->18446744073709551615) 00002000:00010000:0.0:1780298641.834645:0:19951:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24310ed80. 00000100:00100000:0.0:1780298641.835874:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609307648 00000100:00100000:0.0:1780298641.835878:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609307648 00000100:00100000:0.0:1780298641.835904:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61895 00000100:00100000:0.0:1780298641.835910:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f744a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+72:29360:x1866777609307648:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.835921:0:6477:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.835930:0:6477:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae600/0xaf35132e6303271e lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 42 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1ab8 expref: -99 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.835982:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baec00/0xaf35132e63032725 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 43 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.835994:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baec00/0xaf35132e63032725 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 43 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.836004:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baec00/0xaf35132e63032725 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 43 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1780298641.836023:0:6477:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 3300408 00000004:00200000:0.0:1780298641.836026:0:6477:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1780298641.836038:0:6477:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baec00/0xaf35132e63032725 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 43 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.836053:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae600/0xaf35132e6303271e lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 43 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ab8 expref: 74 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.836062:0:6477:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baec00/0xaf35132e63032725 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 42 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ab8 expref: 73 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.836067:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298641.836069:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1780298641.836071:0:6477:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baec00, rc 0) 00000100:00100000:0.0:1780298641.836092:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f744a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+73:29360:x1866777609307648:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 183us (254us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298641.836096:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61895 00000100:00100000:0.0:1780298641.837418:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.838435:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609307904 00000100:00100000:3.0:1780298641.838440:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609307904 00002000:00100000:3.0:1780298641.838459:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (2048000->2052095) req@ffff9cf37f745f80 x1866777609307904/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 488/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298641.838469:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f745f80 with x1866777609307904 ext(2048000->2052095) 00002000:00010000:3.0:1780298641.838473:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f745f80 00000100:00100000:3.0:1780298641.838491:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5881 00000100:00100000:3.0:1780298641.838498:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609307904:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298641.838582:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609307904:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 84us (175us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1780298641.838589:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5881 00002000:00100000:3.0:1780298641.838596:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (2048000->2052095) req@ffff9cf37f745f80 x1866777609307904/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 488/4536 e 0 to 0 dl 1780298652 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298641.838605:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f745f80 with x1866777609307904 ext(2048000->2052095) 00002000:00010000:3.0:1780298641.838608:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f745f80 00000100:00100000:0.0:1780298641.839666:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.840724:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.841643:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.842628:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.842653:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609308544 00000100:00100000:3.0:1780298641.842658:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609308544 00010000:00010000:3.0:1780298641.842674:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e63032710 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1aaa expref: 10 pid: 20252 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.842695:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14922 00000100:00100000:3.0:1780298641.842699:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255489500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609308544:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298641.842705:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298641.842715:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e63032710 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1aaa expref: 10 pid: 20252 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.842723:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e63032710 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1aaa expref: 10 pid: 20252 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.842733:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e63032710 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1aaa expref: 10 pid: 20252 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.842749:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(76 177) 00010000:00010000:3.0:1780298641.842752:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(76 177) 00010000:00010000:3.0:1780298641.842756:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e63032733 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1abf expref: 8 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.842761:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e63032733 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1abf expref: 8 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.842777:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e63032733 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1abf expref: 8 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.842784:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f755180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777604832768:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298641.842813:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.843434:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f755180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777604832768:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298641.843444:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298641.843468:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255489500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609308544:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 767us (838us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298641.843477:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14922 00000100:00100000:0.0:1780298641.843608:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.844519:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.845422:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.845545:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.847783:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.848756:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.848776:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.848778:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609309440 00000100:00100000:3.0:1780298641.848782:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609309440 00000100:00100000:3.0:1780298641.848804:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609309312 00000100:00100000:3.0:1780298641.848806:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609309312 00000100:00100000:3.0:1780298641.848832:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8581 00000100:00100000:3.0:1780298641.848842:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255489180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609309312:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.848859:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.848868:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e63032756 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1ad4 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.848893:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f755180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604833280:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.848927:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.850558:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.850641:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.852163:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.853239:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.855579:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.855604:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.855637:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609310208 00000100:00100000:0.0:1780298641.855644:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609310208 00000100:00100000:0.0:1780298641.855696:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8585 00000100:00100000:0.0:1780298641.855710:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f754000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609310208:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.855720:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.855729:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2d588e1/0xaf35132e63032780 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1afe expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.855768:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255488380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604833792:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.855804:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733fb600 going to sleep for 11 seconds 00002000:00010000:3.0:1780298641.856527:0:8358:0:(ofd_lvb.c:259:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size: 0 -> 1398613 00000100:00100000:3.0:1780298641.856543:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255488380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604833792:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.856566:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c2d588e1/0xaf35132e63032780 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1afe expref: 13 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.856582:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2d588e1/0xaf35132e63032780 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1afe expref: 13 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.856595:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf800, rc 0) 00000100:00100000:3.0:1780298641.856620:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f754000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609310208:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 914us (1041us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.856626:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8585 00000100:00100000:0.0:1780298641.857575:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609310336 00000100:00100000:0.0:1780298641.857581:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609310336 00000100:00100000:0.0:1780298641.857619:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8586 00000100:00100000:0.0:1780298641.857629:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609310336:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.857637:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.857647:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000075508ee2/0xaf35132e63032787 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1b05 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.857656:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.857685:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255489f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604833920:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.857728:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733fb600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.858243:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255489f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604833920:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.858259:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000075508ee2/0xaf35132e63032787 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1b05 expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.858273:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000075508ee2/0xaf35132e63032787 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1b05 expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.858280:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae200, rc 0) 00000100:00100000:0.0:1780298641.858313:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609310336:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 687us (771us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.858318:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8586 00000100:00100000:0.0:1780298641.858414:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f745180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604834048:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.858432:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e6303278e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1b0c expref: 13 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.858446:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e6303278e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1b0c expref: 13 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.858456:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9ac00, rc 0) 00000100:00100000:0.0:1780298641.858488:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f754a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609310464:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 739us (837us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.858494:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8587 00000100:00100000:3.0:1780298641.860158:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.860181:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609310720 00000100:00100000:0.0:1780298641.860185:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609310720 00010000:00010000:0.0:1780298641.860201:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e63032733 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1abf expref: 10 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.860222:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14924 00000100:00100000:0.0:1780298641.860227:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b1c050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609310720:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298641.860235:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298641.860243:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e63032733 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1abf expref: 10 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.860254:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e63032733 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1abf expref: 10 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.860271:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e63032733 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1abf expref: 10 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.860281:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(77 177) 00010000:00010000:0.0:1780298641.860285:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(77 177) 00010000:00010000:0.0:1780298641.860291:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e63032795 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [4067328->37621759] (req 4067328->4079615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1b13 expref: 9 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.860307:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e63032795 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [4067328->37621759] (req 4067328->4079615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1b13 expref: 9 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.860326:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e63032795 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [4067328->37621759] (req 4067328->4079615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1b13 expref: 9 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.860332:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255489f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777604834304:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298641.860365:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733fb600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.860942:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255489f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777604834304:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298641.860951:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298641.860972:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b1c050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609310720:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 745us (867us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298641.860977:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14924 00000100:00100000:3.0:1780298641.861139:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.862270:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609310976 00000100:00100000:3.0:1780298641.862283:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609310976 00000100:00100000:3.0:1780298641.862320:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8589 00000100:00100000:3.0:1780298641.862327:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f744700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609310976:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.862334:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.862343:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e6303279c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1b1a expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.862440:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f755f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604834560:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.862472:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.863190:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f755f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604834560:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.863217:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e6303279c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1b1a expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.863235:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e6303279c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1b1a expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.863246:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3200, rc 0) 00000100:00100000:3.0:1780298641.863274:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f744700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609310976:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 947us (1025us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.863282:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8589 00000100:00100000:0.0:1780298641.864226:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.864304:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.864335:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609311232 00000100:00100000:3.0:1780298641.864396:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609311232 00000100:00100000:3.0:1780298641.864423:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8592 00000100:00100000:3.0:1780298641.864430:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255489500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609311232:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.864437:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.864450:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e630327b1 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1b28 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.864483:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f755f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604834816:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.864518:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.865197:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f755f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604834816:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.865250:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e630327b1 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1b28 expref: 12 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.865266:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e630327b1 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1b28 expref: 12 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.865276:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3a00, rc 0) 00000100:00100000:3.0:1780298641.865321:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255489500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609311232:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 891us (1028us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.865328:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8592 00000100:00100000:0.0:1780298641.866224:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.866326:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.866341:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609311616 00000100:00100000:3.0:1780298641.866345:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609311616 00000100:00100000:3.0:1780298641.866400:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8594 00000100:00100000:3.0:1780298641.866408:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255489f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609311616:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.866414:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.866421:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c47b04a/0xaf35132e630327bf lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1b3d expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.866443:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f755f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604835072:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.866472:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.867181:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f755f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604835072:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.867196:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005c47b04a/0xaf35132e630327bf lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1b3d expref: 13 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.867208:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c47b04a/0xaf35132e630327bf lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1b3d expref: 13 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.867218:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3400, rc 0) 00000100:00100000:0.0:1780298641.867242:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255489f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609311616:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 837us (920us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.867247:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8594 00000100:00100000:0.0:1780298641.868038:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.868992:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.869026:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609311872 00000100:00100000:0.0:1780298641.869031:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609311872 00002000:00100000:0.0:1780298641.869054:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (1396736->1398612) req@ffff9cf37f755880 x1866777609311872/t0(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:347/0 lens 2368/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.869063:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f755880 with x1866777609311872 ext(1396736->1398612) 00010000:00010000:0.0:1780298641.869070:0:19950:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 0000000065a1d570/0xaf35132e6303273a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ac6 expref: 15 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.869083:0:19950:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 0000000065a1d570/0xaf35132e6303273a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ac6 expref: 15 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1780298641.869092:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9cf37f755880 00000100:00100000:0.0:1780298641.869115:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 178 00000100:00100000:0.0:1780298641.869150:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f755880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2399:x1866777609311872:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:0.0:1780298641.869646:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f755880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2399:x1866777609311872:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 525us (686us total) trans 4294969161 rc 0/0 00000100:00100000:0.0:1780298641.869651:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 178 00002000:00100000:0.0:1780298641.869656:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (1396736->1398612) req@ffff9cf37f755880 x1866777609311872/t4294969161(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:347/0 lens 2368/448 e 0 to 0 dl 1780298652 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.869663:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f755880 with x1866777609311872 ext(1396736->1398612) 00010000:00010000:0.0:1780298641.869668:0:19950:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 0000000065a1d570/0xaf35132e6303273a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ac6 expref: 17 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.869678:0:19950:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 0000000065a1d570/0xaf35132e6303273a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ac6 expref: 17 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1780298641.869683:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9cf37f755880 00000100:00100000:0.0:1780298641.870479:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.871761:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.871792:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609312128 00000100:00100000:0.0:1780298641.871797:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609312128 00002000:00100000:0.0:1780298641.871818:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (880640->884735) req@ffff9cf37f757100 x1866777609312128/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 488/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.871825:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f757100 with x1866777609312128 ext(880640->884735) 00002000:00010000:0.0:1780298641.871827:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f757100 00000100:00100000:0.0:1780298641.871846:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5886 00000100:00100000:0.0:1780298641.871853:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f757100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609312128:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298641.871967:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f757100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609312128:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 113us (203us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298641.871974:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5886 00002000:00100000:0.0:1780298641.871980:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (880640->884735) req@ffff9cf37f757100 x1866777609312128/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 488/4536 e 0 to 0 dl 1780298652 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.871989:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f757100 with x1866777609312128 ext(880640->884735) 00002000:00010000:0.0:1780298641.871992:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f757100 00000100:00100000:0.0:1780298641.873479:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.877261:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609312512 00000100:00100000:3.0:1780298641.877267:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609312512 00000100:00100000:3.0:1780298641.877323:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8596 00000100:00100000:3.0:1780298641.877337:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609312512:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.877348:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.877366:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e630327cd lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1b4b expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.877434:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f754380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604835456:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.877509:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.878387:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f754380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604835456:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.878412:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e630327cd lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1b4b expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.878428:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e630327cd lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1b4b expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.878439:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3a00, rc 0) 00000100:00100000:3.0:1780298641.878469:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609312512:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1138us (1255us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.878476:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8596 00000100:00100000:0.0:1780298641.879421:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.879448:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609312768 00000100:00100000:3.0:1780298641.879462:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609312768 00000100:00100000:3.0:1780298641.879497:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8598 00000100:00100000:3.0:1780298641.879506:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255489180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609312768:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.879516:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.879525:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e630327db lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1b59 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.879556:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f754380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604835584:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.879599:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.880341:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f754380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604835584:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.880358:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e630327db lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1b59 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.880372:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e630327db lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1b59 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.880382:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3200, rc 0) 00000100:00100000:3.0:1780298641.880411:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255489180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609312768:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 906us (989us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.880418:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8598 00000100:00100000:0.0:1780298641.882174:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.883690:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.883717:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609313152 00000100:00100000:0.0:1780298641.883718:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.883724:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609313152 00010000:00010000:3.0:1780298641.883748:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e630327c6 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 880640->884735) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1b44 expref: 14 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.883783:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14926 00000100:00100000:3.0:1780298641.883804:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2398:x1866777609313152:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298641.883819:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298641.883837:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e630327c6 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 880640->884735) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1b44 expref: 14 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.883857:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e630327c6 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 880640->884735) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1b44 expref: 14 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.883892:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e630327c6 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 880640->884735) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1b44 expref: 14 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.883906:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(61 130) 00010000:00010000:3.0:1780298641.883914:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(61 130) 00010000:00010000:3.0:1780298641.883923:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000003c8dab73/0xaf35132e630327e2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3887104->37441535] (req 3887104->3895295) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1b60 expref: 13 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.883933:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003c8dab73/0xaf35132e630327e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3887104->37441535] (req 3887104->3895295) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1b60 expref: 13 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.883961:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000003c8dab73/0xaf35132e630327e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3887104->37441535] (req 3887104->3895295) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1b60 expref: 13 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.883976:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f754380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:20253:1866777604835968:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298641.884022:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.884700:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f754380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:20253:1866777604835968:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298641.884717:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298641.884743:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2398:x1866777609313152:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 943us (1053us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298641.884750:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14926 00000100:00100000:0.0:1780298641.885266:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.885402:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609313408 00000100:00100000:3.0:1780298641.885408:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609313408 00000100:00100000:3.0:1780298641.885508:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8601 00000100:00100000:3.0:1780298641.885518:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f744000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609313408:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.885527:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.885537:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000010b6573d/0xaf35132e630327f0 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1b6e expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.885565:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f754380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604836224:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.885638:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.887658:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.887718:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609313536 00000100:00100000:3.0:1780298641.887724:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609313536 00000100:00100000:3.0:1780298641.887763:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8602 00000100:00100000:3.0:1780298641.887776:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243060380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609313536:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.887786:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.887796:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e630327f7 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1b75 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.887869:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f754700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604836352:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.887918:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.888724:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f754700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604836352:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.888749:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e630327f7 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1b75 expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.888765:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e630327f7 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1b75 expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.888777:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2800, rc 0) 00000100:00100000:3.0:1780298641.888843:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243060380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609313536:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1071us (1187us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.888850:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8602 00000100:00100000:0.0:1780298641.890054:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.891312:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609313920 00000100:00100000:0.0:1780298641.891316:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609313920 00002000:00100000:0.0:1780298641.891339:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (1380352->1388543) req@ffff9cf37f744380 x1866777609313920/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:347/0 lens 488/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.891346:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f744380 with x1866777609313920 ext(1380352->1388543) 00002000:00010000:0.0:1780298641.891350:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f744380 00000100:00100000:0.0:1780298641.891366:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5889 00000100:00100000:0.0:1780298641.891371:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f744380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2397:x1866777609313920:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298641.891546:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f744380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2397:x1866777609313920:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 173us (288us total) trans 0 rc 8192/8192 00000100:00100000:0.0:1780298641.891553:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5889 00002000:00100000:0.0:1780298641.891561:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (1380352->1388543) req@ffff9cf37f744380 x1866777609313920/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:347/0 lens 488/8632 e 0 to 0 dl 1780298652 ref 1 fl Complete:/600/0 rc 8192/8192 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.891570:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f744380 with x1866777609313920 ext(1380352->1388543) 00002000:00010000:0.0:1780298641.891573:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f744380 00000100:00100000:3.0:1780298641.892657:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.892802:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609314176 00000100:00100000:3.0:1780298641.892805:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609314176 00000100:00100000:3.0:1780298641.892831:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8606 00000100:00100000:3.0:1780298641.892838:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609314176:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.892846:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.892855:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e63032813 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1b91 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.892884:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f756a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604836736:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.892909:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.894747:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.896511:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.896543:0:15026:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609314688 00000100:00100000:3.0:1780298641.896547:0:15026:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609314688 00000100:00100000:3.0:1780298641.896575:0:15026:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 133 00000100:00100000:3.0:1780298641.896583:0:15026:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243060e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+85:29360:x1866777609314688:12345-192.168.203.25@tcp:35:fsx.0 00000100:00100000:3.0:1780298641.896782:0:15026:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243060e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+86:29360:x1866777609314688:12345-192.168.203.25@tcp:35:fsx.0 Request processed in 201us (273us total) trans 4294990986 rc 0/0 00000100:00100000:3.0:1780298641.896787:0:15026:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 133 00000100:00100000:3.0:1780298641.897715:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.897764:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609314944 00000100:00100000:0.0:1780298641.897769:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609314944 00000100:00100000:0.0:1780298641.897806:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61899 00000100:00100000:0.0:1780298641.897814:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f757800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+86:29360:x1866777609314944:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.897827:0:6477:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.897842:0:6477:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baec00/0xaf35132e63032828 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1ba6 expref: -99 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.897894:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae400/0xaf35132e6303282f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.897909:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae400/0xaf35132e6303282f lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.897922:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae400/0xaf35132e6303282f lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.897936:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae800/0xaf35132e63032836 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5950b203 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.897944:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae800/0xaf35132e63032836 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5950b203 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.897951:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae800/0xaf35132e63032836 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5950b203 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.898043:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae600/0xaf35132e6303283d lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 45 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.898054:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae600/0xaf35132e6303283d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 45 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.898066:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae600/0xaf35132e6303283d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 45 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1780298641.898074:0:6477:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 8086739 00010000:00010000:0.0:1780298641.898148:0:6477:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae400/0xaf35132e6303282f lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.898157:0:6477:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae400/0xaf35132e6303282f lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.898163:0:6477:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae400/0xaf35132e6303282f lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.898170:0:6477:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae400/0xaf35132e6303282f lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.898176:0:6477:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf242bae400 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:0.0:1780298641.898180:0:6477:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae400/0xaf35132e6303282f lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.898187:0:6477:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae400/0xaf35132e6303282f lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.898196:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9cf3600ff300) 00010000:00010000:0.0:1780298641.898199:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1780298641.898203:0:6477:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae400/0xaf35132e6303282f lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.898210:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae400/0xaf35132e6303282f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.898221:0:6477:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae800/0xaf35132e63032836 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5950b203 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.898227:0:6477:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae800/0xaf35132e63032836 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5950b203 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.898233:0:6477:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae800/0xaf35132e63032836 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5950b203 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.898238:0:6477:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf242bae800 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:0.0:1780298641.898242:0:6477:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae800/0xaf35132e63032836 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5950b203 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.898248:0:6477:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae800/0xaf35132e63032836 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5950b203 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.898254:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5950b203 (ffff9cf37338b100) 00010000:00010000:0.0:1780298641.898257:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1780298641.898259:0:6477:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae800/0xaf35132e63032836 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5950b203 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.898265:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae800/0xaf35132e63032836 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5950b203 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.898315:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baec00/0xaf35132e63032828 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ba6 expref: 89 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.898329:0:6477:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae600/0xaf35132e6303283d lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 45 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ba6 expref: 88 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.898336:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298641.898338:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1780298641.898341:0:6477:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae600, rc 0) 00000100:00100000:0.0:1780298641.898375:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f757800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+88:29360:x1866777609314944:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 562us (659us total) trans 4294990987 rc 0/0 00000100:00100000:0.0:1780298641.898382:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61899 00000100:00100000:3.0:1780298641.899334:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609315072 00000100:00100000:3.0:1780298641.899338:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609315072 00000100:00100000:3.0:1780298641.899365:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 2222 00000100:00100000:3.0:1780298641.899370:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243844a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+88:2398:x1866777609315072:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298641.899377:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298641.899387:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae600/0xaf35132e6303283d lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 46 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ba6 expref: 88 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.899396:0:6462:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298641.899398:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298641.899418:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243844a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+87:2398:x1866777609315072:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 49us (129us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298641.899423:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 2222 00000100:00100000:0.0:1780298641.899700:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609315200 00000100:00100000:0.0:1780298641.899705:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609315200 00000100:00100000:0.0:1780298641.899736:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8609 00000100:00100000:0.0:1780298641.899746:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243847480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609315200:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.899755:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.899766:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e6303284b lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1bad expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.899813:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604837120:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.899825:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609315328 00000100:00100000:3.0:1780298641.899828:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609315328 00000100:00100000:0.0:1780298641.899866:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344d716c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.899882:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8610 00000100:00100000:3.0:1780298641.899888:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243845c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609315328:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.899894:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.899902:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63032852 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1bb4 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.899924:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f756a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604837248:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.899998:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348ce1840 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.900541:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604837120:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.900565:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e6303284b lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1bad expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.900578:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e6303284b lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1bad expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.900586:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf000, rc 0) 00000100:00100000:0.0:1780298641.900616:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243847480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609315200:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 873us (963us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.900622:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f756a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604837248:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.900629:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8609 00010000:00010000:3.0:1780298641.900669:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63032852 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1bb4 expref: 14 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.900714:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63032852 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1bb4 expref: 14 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.900723:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2800, rc 0) 00000100:00100000:3.0:1780298641.900744:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243845c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609315328:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 858us (941us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.900751:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8610 00000100:00100000:0.0:1780298641.901572:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609315456 00000100:00100000:0.0:1780298641.901578:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609315456 00000100:00100000:0.0:1780298641.901617:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609315584 00000100:00100000:0.0:1780298641.901621:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609315584 00000100:00100000:0.0:1780298641.901661:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8612 00000100:00100000:0.0:1780298641.901675:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243845180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609315584:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.901689:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.901701:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e63032860 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1bc2 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.901736:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604837504:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.901779:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344d716c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.902443:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604837504:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.902461:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e63032860 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1bc2 expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.902476:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e63032860 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1bc2 expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.902487:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf200, rc 0) 00000100:00100000:0.0:1780298641.902523:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243845180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609315584:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 852us (929us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.902537:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8612 00000100:00100000:0.0:1780298641.905891:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609315968 00000100:00100000:0.0:1780298641.905895:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609315968 00002000:00100000:0.0:1780298641.905917:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (3530752->3534847) req@ffff9cf243844000 x1866777609315968/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 488/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.905928:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243844000 with x1866777609315968 ext(3530752->3534847) 00002000:00010000:0.0:1780298641.905931:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf243844000 00000100:00100000:0.0:1780298641.905946:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5890 00000100:00100000:0.0:1780298641.905950:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243844000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609315968:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298641.906072:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243844000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609315968:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 120us (209us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298641.906079:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5890 00002000:00100000:0.0:1780298641.906087:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (3530752->3534847) req@ffff9cf243844000 x1866777609315968/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 488/440 e 0 to 0 dl 1780298652 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.906094:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243844000 with x1866777609315968 ext(3530752->3534847) 00002000:00010000:0.0:1780298641.906097:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf243844000 00000100:00100000:0.0:1780298641.906999:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609316096 00000100:00100000:0.0:1780298641.907003:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609316096 00002000:00100000:0.0:1780298641.907030:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (3534848->3538943) req@ffff9cf243846d80 x1866777609316096/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 488/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.907040:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243846d80 with x1866777609316096 ext(3534848->3538943) 00002000:00010000:0.0:1780298641.907043:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf243846d80 00000100:00100000:0.0:1780298641.907065:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5891 00000100:00100000:0.0:1780298641.907072:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243846d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2398:x1866777609316096:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298641.907157:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243846d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2398:x1866777609316096:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 86us (187us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298641.907162:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5891 00002000:00100000:0.0:1780298641.907168:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (3534848->3538943) req@ffff9cf243846d80 x1866777609316096/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 488/440 e 0 to 0 dl 1780298652 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.907174:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243846d80 with x1866777609316096 ext(3534848->3538943) 00002000:00010000:0.0:1780298641.907176:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf243846d80 00000100:00100000:0.0:1780298641.908393:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609316224 00000100:00100000:3.0:1780298641.908396:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609316352 00000100:00100000:0.0:1780298641.908400:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609316224 00000100:00100000:3.0:1780298641.908411:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609316352 00000100:00100000:0.0:1780298641.908438:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8614 00000100:00100000:0.0:1780298641.908449:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243845500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609316224:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.908459:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298641.908481:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8615 00010000:00010000:0.0:1780298641.908483:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005d10d3aa/0xaf35132e6303286e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1bd0 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.908500:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243845f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2398:x1866777609316352:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.908516:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298641.908531:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604837888:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.908532:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e63032875 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1bd7 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.908588:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f756a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604838016:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.908603:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344d716c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.908653:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348ce1840 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.909319:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604837888:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.909350:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005d10d3aa/0xaf35132e6303286e lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1bd0 expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.909370:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005d10d3aa/0xaf35132e6303286e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1bd0 expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.909381:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baec00, rc 0) 00000100:00100000:0.0:1780298641.909419:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243845500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609316224:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 973us (1134us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.909429:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8614 00000100:00100000:3.0:1780298641.909437:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f756a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604838016:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.909468:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e63032875 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1bd7 expref: 15 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.909504:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e63032875 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1bd7 expref: 15 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.909514:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2000, rc 0) 00000100:00100000:3.0:1780298641.909594:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243845f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2398:x1866777609316352:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1102us (1255us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.909601:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8615 00000100:00100000:0.0:1780298641.910850:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609316608 00000100:00100000:0.0:1780298641.910857:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609316608 00000100:00100000:0.0:1780298641.910889:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8617 00000100:00100000:0.0:1780298641.910902:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481ff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2397:x1866777609316608:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.910911:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.910921:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e6303287c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1be5 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.910961:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604838144:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.911014:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344d716c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.911820:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604838144:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.911844:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e6303287c lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1be5 expref: 15 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.911862:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e6303287c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1be5 expref: 15 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.911883:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf200, rc 0) 00000100:00100000:0.0:1780298641.911912:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481ff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2397:x1866777609316608:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1015us (1098us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.911918:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8617 00000100:00100000:3.0:1780298641.913157:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609316864 00000100:00100000:3.0:1780298641.913166:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609316864 00000100:00100000:0.0:1780298641.913218:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8619 00000100:00100000:0.0:1780298641.913225:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481ff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2398:x1866777609316864:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.913232:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.913240:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d4d8e3d7/0xaf35132e63032891 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1bf3 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.913261:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604838528:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.913291:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344d716c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.913993:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604838528:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.914004:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d4d8e3d7/0xaf35132e63032891 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1bf3 expref: 17 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.914014:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d4d8e3d7/0xaf35132e63032891 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1bf3 expref: 17 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.914020:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf000, rc 0) 00000100:00100000:0.0:1780298641.914046:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481ff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2398:x1866777609316864:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 821us (1000us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.914053:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8619 00000100:00100000:0.0:1780298641.916260:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609317248 00000100:00100000:0.0:1780298641.916265:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609317248 00000100:00100000:0.0:1780298641.916314:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8622 00000100:00100000:0.0:1780298641.916322:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609317248:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.916331:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.916339:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007b6539a7/0xaf35132e630328a6 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1c08 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.916369:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604838784:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.916402:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344d716c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.917047:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604838784:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.917064:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007b6539a7/0xaf35132e630328a6 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1c08 expref: 17 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.917080:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007b6539a7/0xaf35132e630328a6 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1c08 expref: 17 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.917088:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae800, rc 0) 00000100:00100000:0.0:1780298641.917118:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609317248:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 799us (906us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.917145:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8622 00000100:00100000:0.0:1780298641.917989:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609317376 00000100:00100000:0.0:1780298641.917994:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609317376 00002000:00100000:0.0:1780298641.918021:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (2195456->2199551) req@ffff9cf3481fea00 x1866777609317376/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:347/0 lens 488/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.918031:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fea00 with x1866777609317376 ext(2195456->2199551) 00002000:00010000:0.0:1780298641.918037:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481fea00 00000100:00100000:0.0:1780298641.918058:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5892 00000100:00100000:0.0:1780298641.918066:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609317376:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298641.918238:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609317376:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 171us (301us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298641.918245:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5892 00002000:00100000:0.0:1780298641.918252:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (2195456->2199551) req@ffff9cf3481fea00 x1866777609317376/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:347/0 lens 488/440 e 0 to 0 dl 1780298652 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.918267:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fea00 with x1866777609317376 ext(2195456->2199551) 00002000:00010000:0.0:1780298641.918271:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481fea00 00000100:00100000:0.0:1780298641.919372:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609317504 00000100:00100000:0.0:1780298641.919377:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609317504 00000100:00100000:0.0:1780298641.919406:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8623 00000100:00100000:0.0:1780298641.919414:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609317504:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.919422:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.919434:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e630328ad lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1c0f expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.919483:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604838912:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.919560:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344d716c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.920399:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604838912:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.920423:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e630328ad lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1c0f expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.920443:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e630328ad lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1c0f expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.920468:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae400, rc 0) 00000100:00100000:0.0:1780298641.920507:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609317504:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1092us (1177us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.920519:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8623 00000100:00100000:0.0:1780298641.921695:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8625 00000100:00100000:0.0:1780298641.921705:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609317760:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.921715:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.921726:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e630328bb lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1c1d expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.921762:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604839168:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.921809:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344d716c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.922484:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604839168:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.922502:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e630328bb lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1c1d expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.922516:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e630328bb lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1c1d expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.922528:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf600, rc 0) 00000100:00100000:0.0:1780298641.922557:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609317760:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 854us (987us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.922571:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8625 00000100:00100000:3.0:1780298641.923651:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.923685:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609318016 00000100:00100000:0.0:1780298641.923690:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609318016 00000100:00100000:0.0:1780298641.923715:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8627 00000100:00100000:0.0:1780298641.923721:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f756a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609318016:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.923728:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.923734:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005d10d3aa/0xaf35132e630328c9 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1c2b expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.923762:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604839424:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.923805:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344d716c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.923873:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.924555:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604839424:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.924571:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005d10d3aa/0xaf35132e630328c9 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1c2b expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.924591:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005d10d3aa/0xaf35132e630328c9 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1c2b expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.924599:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baec00, rc 0) 00000100:00100000:0.0:1780298641.924626:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f756a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609318016:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 906us (980us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.924634:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8627 00000100:00100000:3.0:1780298641.925670:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609318272 00000100:00100000:3.0:1780298641.925679:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609318272 00000100:00100000:3.0:1780298641.925714:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8629 00000100:00100000:3.0:1780298641.925720:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:29360:x1866777609318272:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.925733:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.925746:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e630328d7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1c39 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.925766:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(62 130) 00010000:00010000:3.0:1780298641.925770:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(62 130) 00010000:00010000:3.0:1780298641.925777:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e630328d7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [434176->3530751] (req 434176->438271) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1c39 expref: 15 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.925789:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e630328d7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [434176->3530751] (req 434176->438271) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1c39 expref: 15 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.925816:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2000, rc 0) 00000100:00100000:3.0:1780298641.925862:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:29360:x1866777609318272:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 142us (218us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298641.925867:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8629 00000100:00100000:0.0:1780298641.927965:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609318528 00000100:00100000:0.0:1780298641.927969:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609318528 00000100:00100000:3.0:1780298641.928019:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609318656 00000100:00100000:0.0:1780298641.928026:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8630 00000100:00100000:3.0:1780298641.928028:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609318656 00000100:00100000:0.0:1780298641.928043:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609318528:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.928054:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.928066:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e630328de lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1c40 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.928076:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8631 00000100:00100000:3.0:1780298641.928094:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2398:x1866777609318656:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.928104:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.928114:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e630328e5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1c47 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.928173:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604839680:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.928244:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344d716c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.928247:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f755f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604839808:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.928314:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348ce1840 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.928994:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604839680:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.929029:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e630328de lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1c40 expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.929047:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e630328de lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1c40 expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.929062:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bafa00, rc 0) 00000100:00100000:0.0:1780298641.929104:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609318528:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1066us (1173us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.929112:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8630 00002000:00010000:3.0:1780298641.929191:0:20252:0:(ofd_lvb.c:259:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size: 1398613 -> 3892435 00000100:00100000:3.0:1780298641.929210:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f755f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604839808:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.929235:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e630328e5 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1c47 expref: 17 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.929318:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e630328e5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1c47 expref: 17 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.929330:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2200, rc 0) 00000100:00100000:3.0:1780298641.929375:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2398:x1866777609318656:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1286us (1413us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.929383:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8631 00000100:00100000:0.0:1780298641.931877:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609318912 00000100:00100000:0.0:1780298641.931883:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609318912 00000100:00100000:0.0:1780298641.931919:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8633 00000100:00100000:0.0:1780298641.931932:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2398:x1866777609318912:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.931941:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.931952:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007b6539a7/0xaf35132e630328ec lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1c55 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.932006:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604839936:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.932069:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344d716c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.932957:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604839936:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.932978:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007b6539a7/0xaf35132e630328ec lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1c55 expref: 17 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.933006:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007b6539a7/0xaf35132e630328ec lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1c55 expref: 17 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.933019:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae800, rc 0) 00000100:00100000:0.0:1780298641.933051:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2398:x1866777609318912:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1125us (1210us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.933060:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8633 00000100:00100000:0.0:1780298641.934292:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609319168 00000100:00100000:0.0:1780298641.934304:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609319168 00000100:00100000:0.0:1780298641.934357:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8635 00000100:00100000:0.0:1780298641.934368:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481ffb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2397:x1866777609319168:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.934376:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.934385:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e63032901 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1c63 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.934425:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604840192:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.934466:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344d716c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.935267:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604840192:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.935282:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e63032901 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1c63 expref: 17 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.935305:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e63032901 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1c63 expref: 17 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.935318:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf600, rc 0) 00000100:00100000:0.0:1780298641.935344:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481ffb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2397:x1866777609319168:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 978us (1075us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.935350:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8635 00000100:00100000:3.0:1780298641.936603:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609319424 00000100:00100000:3.0:1780298641.936615:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609319424 00000100:00100000:3.0:1780298641.936717:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8637 00000100:00100000:3.0:1780298641.936729:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481ff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2397:x1866777609319424:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.936744:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.936759:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000039c5f58c/0xaf35132e6303290f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1c71 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.936819:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f755f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604840576:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.936883:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348ce1840 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.937680:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f755f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604840576:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.937704:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000039c5f58c/0xaf35132e6303290f lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1c71 expref: 17 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.937718:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000039c5f58c/0xaf35132e6303290f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1c71 expref: 17 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.937727:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3200, rc 0) 00000100:00100000:3.0:1780298641.937759:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481ff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2397:x1866777609319424:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1034us (1247us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.937770:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8637 00000100:00100000:3.0:1780298641.938926:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609319552 00000100:00100000:3.0:1780298641.938948:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609319552 00000100:00100000:3.0:1780298641.938995:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8638 00000100:00100000:3.0:1780298641.939005:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:29360:x1866777609319552:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.939017:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.939029:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000077813dbd/0xaf35132e63032916 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1c78 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.939052:0:19992:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f68a0cb8/0xaf35132e63032898 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2195456->4067327] (req 2195456->2199551) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1bfa expref: 9 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.939063:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(77 177) 00010000:00010000:3.0:1780298641.939068:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(78 177) 00010000:00010000:3.0:1780298641.939074:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000077813dbd/0xaf35132e63032916 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2674688->2686975] (req 2674688->2686975) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1c78 expref: 12 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.939098:0:19992:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f68a0cb8/0xaf35132e63032898 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2195456->4067327] (req 2195456->2199551) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1bfa expref: 9 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.939113:0:19992:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f68a0cb8/0xaf35132e63032898 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2195456->4067327] (req 2195456->2199551) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1bfa expref: 9 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.939164:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f755f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604840704:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298641.940110:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f755f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604840704:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298641.940169:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000077813dbd/0xaf35132e63032916 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2674688->2686975] (req 2674688->2686975) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1c78 expref: 12 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.940186:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2200, rc 0) 00000100:00100000:0.0:1780298641.940221:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:29360:x1866777609319552:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1217us (1336us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298641.940229:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8638 00000100:00100000:3.0:1780298641.940483:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.941915:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fe680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777604840832:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298641.941927:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298641.941957:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f755180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609319680:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1359us (1474us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298641.941964:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14929 00000100:00100000:3.0:1780298641.942179:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.942215:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609319808 00000100:00100000:0.0:1780298641.942220:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609319808 00002000:00100000:0.0:1780298641.942249:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (2674688->2678783) req@ffff9cf37f755f80 x1866777609319808/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 488/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.942259:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f755f80 with x1866777609319808 ext(2674688->2678783) 00002000:00010000:0.0:1780298641.942264:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f755f80 00000100:00100000:0.0:1780298641.942286:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5894 00000100:00100000:0.0:1780298641.942306:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f755f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609319808:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298641.942443:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f755f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609319808:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 150us (264us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298641.942450:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5894 00002000:00100000:0.0:1780298641.942457:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (2674688->2678783) req@ffff9cf37f755f80 x1866777609319808/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 488/440 e 0 to 0 dl 1780298652 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.942466:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f755f80 with x1866777609319808 ext(2674688->2678783) 00002000:00010000:0.0:1780298641.942470:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f755f80 00000100:00100000:3.0:1780298641.943471:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.943504:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609319936 00000100:00100000:0.0:1780298641.943508:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609319936 00002000:00100000:0.0:1780298641.943525:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (2682880->2686975) req@ffff9cf37f754a80 x1866777609319936/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 488/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.943531:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f754a80 with x1866777609319936 ext(2682880->2686975) 00002000:00010000:0.0:1780298641.943534:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f754a80 00000100:00100000:0.0:1780298641.943549:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5895 00000100:00100000:0.0:1780298641.943554:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f754a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609319936:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298641.943630:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f754a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609319936:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 76us (161us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298641.943635:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5895 00002000:00100000:0.0:1780298641.943638:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (2682880->2686975) req@ffff9cf37f754a80 x1866777609319936/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 488/440 e 0 to 0 dl 1780298652 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.943643:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f754a80 with x1866777609319936 ext(2682880->2686975) 00002000:00010000:0.0:1780298641.943645:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f754a80 00000100:00100000:3.0:1780298641.944918:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.944975:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609320192 00000100:00100000:0.0:1780298641.944979:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609320192 00000100:00100000:0.0:1780298641.945023:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8640 00000100:00100000:0.0:1780298641.945036:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f754700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2398:x1866777609320192:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.945046:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.945057:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e63032924 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1c86 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.945089:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604841088:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.945141:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344d716c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.946013:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604841088:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.946034:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e63032924 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1c86 expref: 17 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.946048:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e63032924 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1c86 expref: 17 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.946058:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf600, rc 0) 00000100:00100000:0.0:1780298641.946083:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f754700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2398:x1866777609320192:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1050us (1166us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.946090:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8640 00000100:00100000:3.0:1780298641.946997:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609320320 00000100:00100000:3.0:1780298641.947002:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609320320 00000100:00100000:3.0:1780298641.947076:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8641 00000100:00100000:3.0:1780298641.947086:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609320320:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.947093:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.947102:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e6303292b lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1c8d expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.947159:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f757480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604841216:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.947212:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348ce1840 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.947986:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f757480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604841216:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.948002:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e6303292b lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1c8d expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.948060:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e6303292b lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1c8d expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.948072:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2800, rc 0) 00000100:00100000:3.0:1780298641.948101:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609320320:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1017us (1185us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.948110:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8641 00000100:00100000:0.0:1780298641.949209:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8643 00000100:00100000:3.0:1780298641.949220:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8644 00000100:00100000:0.0:1780298641.949223:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609320576:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.949231:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298641.949236:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2398:x1866777609320704:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.949240:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007b6539a7/0xaf35132e63032939 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1c9b expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.949245:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298641.949269:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604841472:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.949326:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344d716c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.950006:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604841472:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.950027:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007b6539a7/0xaf35132e63032939 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1c9b expref: 13 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.950042:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007b6539a7/0xaf35132e63032939 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1c9b expref: 13 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.950053:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae800, rc 0) 00000100:00100000:0.0:1780298641.950085:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609320576:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 867us (990us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.950094:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8643 00000100:00100000:0.0:1780298641.951253:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609320832 00000100:00100000:0.0:1780298641.951259:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609320832 00000100:00100000:0.0:1780298641.951303:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8645 00000100:00100000:0.0:1780298641.951314:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609320832:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.951324:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.951335:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005d10d3aa/0xaf35132e63032947 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1ca9 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.951373:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604841728:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.951430:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344d716c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.952064:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604841728:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.952081:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005d10d3aa/0xaf35132e63032947 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ca9 expref: 13 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.952096:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005d10d3aa/0xaf35132e63032947 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ca9 expref: 13 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.952106:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baec00, rc 0) 00000100:00100000:0.0:1780298641.952163:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609320832:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 851us (952us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.952171:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8645 00000100:00100000:3.0:1780298641.953587:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.954335:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b69880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604842112:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.954351:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bf17d487/0xaf35132e6303295c lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1cbe expref: 17 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.954368:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bf17d487/0xaf35132e6303295c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1cbe expref: 17 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.954381:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37400, rc 0) 00000100:00100000:0.0:1780298641.954405:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f757480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609321216:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 729us (819us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.954411:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8648 00000100:00100000:0.0:1780298641.955452:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609321344 00000100:00100000:0.0:1780298641.955457:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609321344 00000100:00100000:0.0:1780298641.955488:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5896 00000100:00100000:0.0:1780298641.955495:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609321344:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298641.955659:0:19951:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 2685154 -> 614479 00002000:00010000:0.0:1780298641.955663:0:19951:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298641.955666:0:19951:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298641.955668:0:19951:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298641.955670:0:19951:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 296 -> 320 00000100:00100000:0.0:1780298641.955700:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2398:x1866777609321344:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 204us (282us total) trans 4294969587 rc 0/0 00000100:00100000:0.0:1780298641.955708:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5896 00000100:00100000:3.0:1780298641.956604:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609321472 00000100:00100000:3.0:1780298641.956610:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609321472 00000100:00100000:3.0:1780298641.956644:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8649 00000100:00100000:3.0:1780298641.956654:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609321472:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.956662:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298641.956663:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609321600 00010000:00010000:3.0:1780298641.956669:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e63032963 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1cc5 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.956669:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609321600 00000100:00100000:0.0:1780298641.956701:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8650 00000100:00100000:0.0:1780298641.956708:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2398:x1866777609321600:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.956715:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.956722:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007b6539a7/0xaf35132e6303296a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1ccc expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.956734:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f756a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604842240:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.956750:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604842368:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.956777:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348ce1840 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.956777:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344d716c0 going to sleep for 11 seconds 00002000:00010000:3.0:1780298641.957544:0:8358:0:(ofd_lvb.c:259:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size: 614479 -> 2685154 00000100:00100000:3.0:1780298641.957554:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f756a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604842240:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.957570:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e63032963 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1cc5 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.957587:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604842368:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.957588:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e63032963 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1cc5 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.957604:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3200, rc 0) 00010000:00010000:0.0:1780298641.957613:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007b6539a7/0xaf35132e6303296a lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ccc expref: 17 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.957628:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007b6539a7/0xaf35132e6303296a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ccc expref: 17 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.957635:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609321472:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 983us (1089us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.957648:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8649 00010000:00010000:0.0:1780298641.957658:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae800, rc 0) 00000100:00100000:0.0:1780298641.957684:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2398:x1866777609321600:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 976us (1047us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.957692:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8650 00000100:00100000:0.0:1780298641.958645:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609321728 00000100:00100000:0.0:1780298641.958650:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609321728 00000100:00100000:0.0:1780298641.958679:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8651 00000100:00100000:0.0:1780298641.958686:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609321728:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.958694:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.958703:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e63032971 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1cd3 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.958730:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604842496:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.958769:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344d716c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.959515:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604842496:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.959537:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e63032971 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1cd3 expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.959554:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e63032971 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1cd3 expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.959563:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf600, rc 0) 00000100:00100000:0.0:1780298641.959598:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609321728:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 912us (984us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.959610:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8651 00000100:00100000:0.0:1780298641.960723:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609321984 00000100:00100000:0.0:1780298641.960727:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609321984 00000100:00100000:0.0:1780298641.960757:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5897 00000100:00100000:0.0:1780298641.960763:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2398:x1866777609321984:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298641.960826:0:8364:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 2685154 -> 614479 00002000:00010000:0.0:1780298641.960829:0:8364:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298641.960831:0:8364:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298641.960832:0:8364:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298641.960834:0:8364:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 320 -> 320 00000100:00100000:0.0:1780298641.960856:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2398:x1866777609321984:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 93us (163us total) trans 4294969588 rc 0/0 00000100:00100000:0.0:1780298641.960862:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5897 00000100:00100000:3.0:1780298641.962026:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609322240 00000100:00100000:3.0:1780298641.962031:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609322240 00000100:00100000:0.0:1780298641.962069:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8654 00000100:00100000:0.0:1780298641.962083:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548bad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2397:x1866777609322240:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.962092:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.962101:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e63032986 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1ce8 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.962160:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604842880:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.962191:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344d716c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.962933:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604842880:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.962948:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e63032986 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ce8 expref: 17 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.962971:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e63032986 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ce8 expref: 17 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.962981:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baec00, rc 0) 00000100:00100000:0.0:1780298641.963000:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548bad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2397:x1866777609322240:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 923us (984us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.963004:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8654 00000100:00100000:3.0:1780298641.964073:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609322368 00000100:00100000:3.0:1780298641.964077:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609322368 00000100:00100000:3.0:1780298641.964109:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8655 00000100:00100000:3.0:1780298641.964115:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609322368:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.964139:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.964147:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e6303298d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1cef expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.964213:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f756a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604843008:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.964263:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348ce1840 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.964903:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f756a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604843008:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.964921:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e6303298d lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1cef expref: 15 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.964936:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e6303298d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1cef expref: 15 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.964948:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2800, rc 0) 00000100:00100000:3.0:1780298641.964976:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609322368:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 860us (942us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.965020:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8655 00000100:00100000:3.0:1780298641.968308:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609323008 00000100:00100000:3.0:1780298641.968312:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609323008 00000100:00100000:3.0:1780298641.968332:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8657 00000100:00100000:3.0:1780298641.968336:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:29360:x1866777609323008:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.968343:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.968349:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e630329a2 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1cfd expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.968365:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000077813dbd/0xaf35132e63032916 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2674688->4067327] (req 2674688->2686975) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1c78 expref: 15 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.968371:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e63032795 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [4067328->37621759] (req 4067328->4079615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1b13 expref: 15 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.968376:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 2 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(78 177) 00010000:00010000:3.0:1780298641.968378:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(80 177) 00010000:00010000:3.0:1780298641.968381:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e630329a2 lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1cfd expref: 15 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.968392:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e63032795 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [4067328->37621759] (req 4067328->4079615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1b13 expref: 15 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.968398:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e63032795 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [4067328->37621759] (req 4067328->4079615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1b13 expref: 15 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.968404:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f756a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604843392:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298641.968435:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000077813dbd/0xaf35132e63032916 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2674688->4067327] (req 2674688->2686975) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1c78 expref: 15 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.968442:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000077813dbd/0xaf35132e63032916 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2674688->4067327] (req 2674688->2686975) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1c78 expref: 15 pid: 19992 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.968459:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f754380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604843520:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298641.968482:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348ce1840 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.969117:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f756a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604843392:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298641.969139:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f754380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604843520:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298641.969153:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e630329a2 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1cfd expref: 15 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.969160:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3200, rc 0) 00000100:00100000:3.0:1780298641.969179:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:29360:x1866777609323008:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 843us (900us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298641.969184:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8657 00000100:00100000:0.0:1780298641.969426:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609323136 00000100:00100000:0.0:1780298641.969429:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609323136 00010000:00010000:0.0:1780298641.969444:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e63032795 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [4067328->37621759] (req 4067328->4079615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1b13 expref: 15 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.969464:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14931 00000100:00100000:0.0:1780298641.969469:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2398:x1866777609323136:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298641.969475:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298641.969483:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e63032795 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [4067328->37621759] (req 4067328->4079615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1b13 expref: 15 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.969490:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e63032795 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [4067328->37621759] (req 4067328->4079615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1b13 expref: 15 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.969500:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e63032795 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [4067328->37621759] (req 4067328->4079615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1b13 expref: 15 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.969511:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298641.969530:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2398:x1866777609323136:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 62us (133us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298641.969535:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14931 00000100:00100000:3.0:1780298641.970944:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.970974:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609323392 00000100:00100000:0.0:1780298641.970978:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609323392 00010000:00010000:0.0:1780298641.970989:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000077813dbd/0xaf35132e63032916 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2674688->4067327] (req 2674688->2686975) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1c78 expref: 16 pid: 19992 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.971009:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14932 00000100:00100000:0.0:1780298641.971013:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f754380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2398:x1866777609323392:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298641.971022:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:0.0:1780298641.971029:0:20253:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 320 -> 344 00010000:00010000:0.0:1780298641.971035:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000077813dbd/0xaf35132e63032916 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2674688->4067327] (req 2674688->2686975) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1c78 expref: 16 pid: 19992 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.971045:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000077813dbd/0xaf35132e63032916 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2674688->4067327] (req 2674688->2686975) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1c78 expref: 16 pid: 19992 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.971057:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000077813dbd/0xaf35132e63032916 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2674688->4067327] (req 2674688->2686975) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1c78 expref: 16 pid: 19992 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.971068:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(80 177) 00010000:00010000:0.0:1780298641.971071:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(80 177) 00010000:00010000:0.0:1780298641.971076:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e630329a2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1cfd expref: 15 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.971085:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e630329a2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1cfd expref: 15 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.971107:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e630329a2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1cfd expref: 15 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.971115:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777604843648:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298641.971170:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344d716c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.971842:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777604843648:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298641.971851:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298641.971875:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f754380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2398:x1866777609323392:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 861us (931us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298641.971880:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14932 00000100:00100000:3.0:1780298641.971963:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.971990:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609323520 00000100:00100000:0.0:1780298641.971993:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609323520 00000100:00100000:0.0:1780298641.972023:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8658 00000100:00100000:0.0:1780298641.972032:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f756a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:29360:x1866777609323520:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.972040:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.972048:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e630329a9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1d04 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.972068:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e630328d7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [434176->3530751] (req 434176->438271) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1c39 expref: 17 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.972078:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000097580845/0xaf35132e63032867 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [3530752->3883007] (req 3530752->3538943) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1bc9 expref: 17 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.972083:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003c8dab73/0xaf35132e630327e2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [3887104->37441535] (req 3887104->3895295) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1b60 expref: 17 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.972087:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 3 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(62 130) 00010000:00010000:0.0:1780298641.972089:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(65 130) 00010000:00010000:0.0:1780298641.972092:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e630329a9 lrc: 6/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d04 expref: 17 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.972103:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003c8dab73/0xaf35132e630327e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [3887104->37441535] (req 3887104->3895295) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1b60 expref: 17 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.972110:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003c8dab73/0xaf35132e630327e2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [3887104->37441535] (req 3887104->3895295) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1b60 expref: 17 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.972116:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604843776:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298641.972174:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000097580845/0xaf35132e63032867 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [3530752->3883007] (req 3530752->3538943) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1bc9 expref: 17 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.972180:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000097580845/0xaf35132e63032867 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [3530752->3883007] (req 3530752->3538943) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1bc9 expref: 17 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.972192:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243062d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604843904:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298641.972222:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e630328d7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [434176->3530751] (req 434176->438271) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1c39 expref: 17 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.972230:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e630328d7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [434176->3530751] (req 434176->438271) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1c39 expref: 17 pid: 19992 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.972237:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243060000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604844032:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298641.972254:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344d716c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.972924:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604843776:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298641.972929:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243062d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604843904:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298641.973185:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243060000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604844032:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298641.973203:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e630329a9 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d04 expref: 17 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.973212:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf600, rc 0) 00000100:00100000:0.0:1780298641.973236:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f756a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:29360:x1866777609323520:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1207us (1275us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298641.973243:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8658 00000100:00100000:3.0:1780298641.973311:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.973338:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609323648 00000100:00100000:0.0:1780298641.973342:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609323648 00002000:00100000:0.0:1780298641.973355:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (3530752->3892434) req@ffff9cf37f755180 x1866777609323648/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 504/0 e 0 to 0 dl 1780298652 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.973362:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f755180 with x1866777609323648 ext(3530752->3892434) 00010000:00010000:0.0:1780298641.973367:0:8362:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 0000000097580845/0xaf35132e63032867 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [3530752->3883007] (req 3530752->3538943) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1bc9 expref: 17 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.973374:0:8362:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 0000000097580845/0xaf35132e63032867 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [3530752->3883007] (req 3530752->3538943) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1bc9 expref: 17 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.973379:0:8362:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000003c8dab73/0xaf35132e630327e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [3887104->37441535] (req 3887104->3895295) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1b60 expref: 17 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.973384:0:8362:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000003c8dab73/0xaf35132e630327e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [3887104->37441535] (req 3887104->3895295) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1b60 expref: 17 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1780298641.973388:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 2 locks timeout for req ffff9cf37f755180 00000100:00100000:0.0:1780298641.973403:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 180 00000100:00100000:0.0:1780298641.973409:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f755180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2399:x1866777609323648:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:3.0:1780298641.973599:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609323776 00000100:00100000:3.0:1780298641.973604:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609323776 00010000:00010000:3.0:1780298641.973621:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e630328d7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [434176->3530751] (req 434176->438271) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1c39 expref: 19 pid: 19992 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.973649:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14933 00000100:00100000:3.0:1780298641.973656:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+19:2400:x1866777609323776:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298641.973665:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298641.973672:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e630328d7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [434176->3530751] (req 434176->438271) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1c39 expref: 19 pid: 19992 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.973684:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e630328d7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [434176->3530751] (req 434176->438271) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1c39 expref: 19 pid: 19992 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.973693:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e630328d7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [434176->3530751] (req 434176->438271) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1c39 expref: 19 pid: 19992 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.973700:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298641.973720:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2400:x1866777609323776:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 66us (185us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298641.973726:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14933 00000100:00100000:0.0:1780298641.974267:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f755180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2399:x1866777609323648:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 858us (957us total) trans 4294969163 rc 0/0 00000100:00100000:0.0:1780298641.974272:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 180 00002000:00100000:0.0:1780298641.974276:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (3530752->3892434) req@ffff9cf37f755180 x1866777609323648/t4294969163(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:347/0 lens 504/448 e 0 to 0 dl 1780298652 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298641.974283:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f755180 with x1866777609323648 ext(3530752->3892434) 00010000:00010000:0.0:1780298641.974289:0:8362:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 0000000097580845/0xaf35132e63032867 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [3530752->3883007] (req 3530752->3538943) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1bc9 expref: 18 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.974306:0:8362:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 0000000097580845/0xaf35132e63032867 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [3530752->3883007] (req 3530752->3538943) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1bc9 expref: 18 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.974312:0:8362:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000003c8dab73/0xaf35132e630327e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [3887104->37441535] (req 3887104->3895295) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1b60 expref: 18 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.974319:0:8362:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000003c8dab73/0xaf35132e630327e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [3887104->37441535] (req 3887104->3895295) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1b60 expref: 18 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1780298641.974323:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 2 locks timeout for req ffff9cf37f755180 00000100:00100000:0.0:1780298641.975329:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.975348:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.975369:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609323904 00000100:00100000:3.0:1780298641.975373:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609323904 00010000:00010000:3.0:1780298641.975389:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000003c8dab73/0xaf35132e630327e2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3887104->37441535] (req 3887104->3895295) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1b60 expref: 19 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.975412:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14934 00000100:00100000:3.0:1780298641.975419:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243062d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+19:2399:x1866777609323904:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298641.975426:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:3.0:1780298641.975433:0:6461:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 24 -> 64 00010000:00010000:3.0:1780298641.975440:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000003c8dab73/0xaf35132e630327e2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [3887104->37441535] (req 3887104->3895295) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1b60 expref: 19 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.975451:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000003c8dab73/0xaf35132e630327e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [3887104->37441535] (req 3887104->3895295) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1b60 expref: 19 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.975475:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003c8dab73/0xaf35132e630327e2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [3887104->37441535] (req 3887104->3895295) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1b60 expref: 19 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.975505:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298641.975529:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243062d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2399:x1866777609323904:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 110us (178us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298641.975538:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14934 00000100:00100000:0.0:1780298641.976464:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609324160 00000100:00100000:0.0:1780298641.976469:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609324160 00000100:00100000:3.0:1780298641.976485:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609324288 00002000:00010000:0.0:1780298641.976488:0:19950:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00000100:00100000:3.0:1780298641.976490:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609324288 00002000:00100000:0.0:1780298641.976494:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf347cd7480 with x1866777609324160 ext(4194304->18446744073709551615) 00002000:00010000:0.0:1780298641.976499:0:19950:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf347cd7480. 00002000:00010000:3.0:1780298641.976503:0:8363:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (2818642->18446744073709551615) 00002000:00100000:3.0:1780298641.976506:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf347cd4380 with x1866777609324288 ext(2818642->18446744073709551615) 00002000:00010000:3.0:1780298641.976510:0:8363:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf347cd4380. 00000100:00100000:0.0:1780298641.976524:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5898 00000100:00100000:3.0:1780298641.976532:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5899 00000100:00100000:0.0:1780298641.976535:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2400:x1866777609324160:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:3.0:1780298641.976544:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609324288:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:0.0:1780298641.976637:0:19950:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 2685154 -> 4194304 00002000:00010000:0.0:1780298641.976639:0:19950:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298641.976641:0:19950:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298641.976643:0:19950:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298641.976645:0:19950:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 344 -> 320 00000100:00100000:0.0:1780298641.976670:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609324160:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 140us (234us total) trans 4294969590 rc 0/0 00000100:00100000:0.0:1780298641.976676:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5898 00002000:00010000:0.0:1780298641.976679:0:19950:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:0.0:1780298641.976681:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf347cd7480 with x1866777609324160 ext(4194304->18446744073709551615) 00002000:00010000:0.0:1780298641.976684:0:19950:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf347cd7480. 00002000:00010000:3.0:1780298641.976712:0:8363:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 3892435 -> 2818642 00002000:00010000:3.0:1780298641.976715:0:8363:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:3.0:1780298641.976717:0:8363:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:3.0:1780298641.976719:0:8363:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:3.0:1780298641.976720:0:8363:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 64 -> 24 00000100:00100000:3.0:1780298641.976775:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2400:x1866777609324288:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 204us (283us total) trans 4294969164 rc 0/0 00000100:00100000:3.0:1780298641.976780:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5899 00002000:00010000:3.0:1780298641.976783:0:8363:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (2818642->18446744073709551615) 00002000:00100000:3.0:1780298641.976785:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf347cd4380 with x1866777609324288 ext(2818642->18446744073709551615) 00002000:00010000:3.0:1780298641.976792:0:8363:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf347cd4380. 00000100:00100000:0.0:1780298641.977531:0:9976:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609324416 00000100:00100000:0.0:1780298641.977535:0:9976:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609324416 00000100:00100000:0.0:1780298641.977559:0:9976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61901 00000100:00100000:0.0:1780298641.977565:0:9976:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+86:29360:x1866777609324416:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.977574:0:9976:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.977582:0:9976:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baec00/0xaf35132e630329b0 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 44 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1d0b expref: -99 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.977630:0:9976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf000/0xaf35132e630329b7 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 45 type: IBT flags: 0x40000000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.977638:0:9976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf000/0xaf35132e630329b7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 45 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.977644:0:9976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf000/0xaf35132e630329b7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 45 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1780298641.977658:0:9976:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 7012946 00000004:00200000:0.0:1780298641.977660:0:9976:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1780298641.977670:0:9976:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf000/0xaf35132e630329b7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 45 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.977684:0:9976:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baec00/0xaf35132e630329b0 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 45 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d0b expref: 88 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.977693:0:9976:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf000/0xaf35132e630329b7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 44 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d0b expref: 87 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.977698:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298641.977699:0:9976:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1780298641.977701:0:9976:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf000, rc 0) 00000100:00100000:0.0:1780298641.977721:0:9976:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+87:29360:x1866777609324416:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 157us (220us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298641.977725:0:9976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61901 00000100:00100000:0.0:1780298641.978754:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609324544 00000100:00100000:0.0:1780298641.978758:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609324544 00000100:00100000:3.0:1780298641.978768:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609324672 00000100:00100000:3.0:1780298641.978773:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609324672 00000100:00100000:0.0:1780298641.978783:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8659 00000100:00100000:0.0:1780298641.978789:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609324544:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.978795:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.978803:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e630329be lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1d12 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.978809:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8660 00000100:00100000:3.0:1780298641.978821:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609324672:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.978828:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298641.978835:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604844288:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.978836:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e630329c5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1d19 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.978865:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242863100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604844416:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.978874:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344d716c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.978892:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348ce1840 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.979671:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242863100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604844416:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.979685:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604844288:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.979692:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e630329c5 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d19 expref: 17 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.979700:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e630329be lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d12 expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.979711:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e630329c5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d19 expref: 17 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.979713:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e630329be lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d12 expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.979736:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf200, rc 0) 00010000:00010000:3.0:1780298641.979737:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2800, rc 0) 00000100:00100000:0.0:1780298641.979760:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609324544:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 971us (1047us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.979766:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609324672:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 945us (1022us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.979768:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8659 00000100:00100000:3.0:1780298641.979778:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8660 00000100:00100000:3.0:1780298641.980867:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609324800 00000100:00100000:3.0:1780298641.980872:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609324800 00000100:00100000:3.0:1780298641.980895:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609324928 00000100:00100000:3.0:1780298641.980897:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609324928 00000100:00100000:3.0:1780298641.980924:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8662 00000100:00100000:3.0:1780298641.980935:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2399:x1866777609324928:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298641.980946:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298641.980953:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ced6368f/0xaf35132e630329d3 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1d27 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.980977:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242863100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604844672:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298641.981003:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348ce1840 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.981812:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242863100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604844672:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298641.981828:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ced6368f/0xaf35132e630329d3 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d27 expref: 17 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.981839:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ced6368f/0xaf35132e630329d3 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d27 expref: 17 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298641.981846:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2a00, rc 0) 00000100:00100000:3.0:1780298641.981873:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2399:x1866777609324928:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 942us (1002us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298641.981885:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8662 00000100:00100000:0.0:1780298641.983039:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:1.0:1780298641.984342:0:6476:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37200, rc 0) 00000100:00100000:1.0:1780298641.984374:0:6476:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+76:29360:x1866777609325184:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 210us (303us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298641.984381:0:6476:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61902 00000100:00100000:3.0:1780298641.985395:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.985422:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609325440 00000100:00100000:0.0:1780298641.985425:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609325440 00000100:00100000:0.0:1780298641.985450:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61903 00000100:00100000:0.0:1780298641.985459:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242863100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+76:29360:x1866777609325440:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:0.0:1780298641.985486:0:6477:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298641.985498:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37200/0xaf35132e630329e1 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 46 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d2e expref: 76 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.985509:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298641.985512:0:6477:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1780298641.985586:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae200/0xaf35132e630329e8 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 45 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.985594:0:6477:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf000/0xaf35132e630329b7 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 45 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d0b expref: 86 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.985603:0:6477:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae200/0xaf35132e630329e8 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 45 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.985618:0:6477:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf000/0xaf35132e630329b7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 45 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d0b expref: 86 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.985631:0:6477:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf000/0xaf35132e630329b7 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 45 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d0b expref: 86 pid: 9976 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.985638:0:6477:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243060a80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6477:1866777604844800:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298641.985667:0:6477:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298641.986502:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298641.986527:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609325568 00000100:00100000:1.0:1780298641.986532:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609325568 00010000:00010000:1.0:1780298641.986550:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf000/0xaf35132e630329b7 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 45 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d0b expref: 87 pid: 9976 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298641.986579:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14936 00000100:00100000:1.0:1780298641.986586:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242860000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+87:2399:x1866777609325568:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298641.986594:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298641.986598:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf000/0xaf35132e630329b7 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 46 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d0b expref: 87 pid: 9976 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298641.986605:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf000/0xaf35132e630329b7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 46 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d0b expref: 87 pid: 9976 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298641.986614:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf000/0xaf35132e630329b7 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 46 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d0b expref: 87 pid: 9976 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298641.986621:0:6463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298641.986624:0:6463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae200/0xaf35132e630329e8 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 45 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298641.986630:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae200/0xaf35132e630329e8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 45 type: IBT flags: 0x40290000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298641.986634:0:6463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae200/0xaf35132e630329e8 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 45 type: IBT flags: 0x40290000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298641.986647:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298641.986669:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242860000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+86:2399:x1866777609325568:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 84us (166us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298641.986678:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14936 00000100:00100000:3.0:1780298641.987467:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298641.987500:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609325696 00000100:00100000:1.0:1780298641.987504:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609325696 00002000:00010000:1.0:1780298641.987516:0:19943:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:1.0:1780298641.987519:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242861880 with x1866777609325696 ext(4194304->18446744073709551615) 00002000:00010000:1.0:1780298641.987522:0:19943:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf242861880. 00000100:00100000:1.0:1780298641.987539:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5901 00000100:00100000:1.0:1780298641.987544:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242861880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2399:x1866777609325696:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:1.0:1780298641.987601:0:19943:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:1.0:1780298641.987605:0:19943:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:1.0:1780298641.987607:0:19943:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:1.0:1780298641.987609:0:19943:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:1.0:1780298641.987611:0:19943:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 320 -> 320 00000100:00100000:0.0:1780298641.987618:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298641.987645:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242861880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2399:x1866777609325696:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 92us (169us total) trans 4294969592 rc 0/0 00000100:00100000:1.0:1780298641.987654:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5901 00002000:00010000:1.0:1780298641.987658:0:19943:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:1.0:1780298641.987663:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242861880 with x1866777609325696 ext(4194304->18446744073709551615) 00002000:00010000:1.0:1780298641.987671:0:19943:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf242861880. 00000100:00100000:3.0:1780298641.988585:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:2.0:1780298641.988743:0:7411:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b000/0xaf35132e630329f6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 46 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298641.988756:0:7411:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b000/0xaf35132e630329f6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 46 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298641.988769:0:7411:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 7396701 00000004:00200000:2.0:1780298641.988772:0:7411:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298641.988781:0:7411:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b000/0xaf35132e630329f6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 46 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298641.988797:0:7411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a200/0xaf35132e630329ef lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 46 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d35 expref: 80 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298641.988810:0:7411:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b000/0xaf35132e630329f6 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 45 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d35 expref: 79 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298641.988817:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298641.988820:0:7411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298641.988822:0:7411:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b000, rc 0) 00000100:00100000:2.0:1780298641.988849:0:7411:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242862680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+79:29360:x1866777609325952:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 176us (264us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298641.988856:0:7411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61904 00000100:00100000:3.0:1780298641.990002:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298641.990030:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609326080 00000100:00100000:1.0:1780298641.990033:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609326080 00000100:00100000:1.0:1780298641.990060:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5903 00000100:00100000:1.0:1780298641.990068:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2397:x1866777609326080:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298641.990182:0:19951:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:1.0:1780298641.990184:0:19951:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:1.0:1780298641.990186:0:19951:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:1.0:1780298641.990187:0:19951:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:1.0:1780298641.990189:0:19951:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 320 -> 320 00000100:00100000:1.0:1780298641.990210:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+19:2397:x1866777609326080:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 144us (210us total) trans 4294969593 rc 0/0 00000100:00100000:1.0:1780298641.990215:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5903 00000100:00100000:0.0:1780298641.990856:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298641.990893:0:9976:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609326208 00000100:00100000:2.0:1780298641.990898:0:9976:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609326208 00000100:00100000:2.0:1780298641.990927:0:9976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61905 00000100:00100000:2.0:1780298641.990934:0:9976:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243061c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+86:29360:x1866777609326208:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298641.990942:0:9976:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298641.990950:0:9976:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b800/0xaf35132e630329fd lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 46 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1d3c expref: -99 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298641.990988:0:9976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9aa00/0xaf35132e63032a04 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 47 type: IBT flags: 0x40000000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298641.990997:0:9976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9aa00/0xaf35132e63032a04 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 47 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298641.991004:0:9976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9aa00/0xaf35132e63032a04 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 47 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298641.991028:0:9976:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 7396701 00000004:00200000:2.0:1780298641.991031:0:9976:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298641.991040:0:9976:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9aa00/0xaf35132e63032a04 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 47 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298641.991056:0:9976:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b800/0xaf35132e630329fd lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 47 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d3c expref: 88 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298641.991067:0:9976:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9aa00/0xaf35132e63032a04 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 46 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d3c expref: 87 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298641.991073:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298641.991076:0:9976:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298641.991078:0:9976:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9aa00, rc 0) 00000100:00100000:2.0:1780298641.991103:0:9976:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243061c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+87:29360:x1866777609326208:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 170us (246us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298641.991109:0:9976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61905 00000100:00100000:3.0:1780298641.991980:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.992011:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298641.992056:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609326336 00000100:00100000:0.0:1780298641.992057:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609326464 00000100:00100000:1.0:1780298641.992062:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609326336 00000100:00100000:0.0:1780298641.992063:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609326464 00000100:00100000:1.0:1780298641.992100:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8663 00000100:00100000:0.0:1780298641.992108:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8664 00000100:00100000:1.0:1780298641.992114:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609326336:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298641.992120:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348007100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2399:x1866777609326464:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.992150:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298641.992154:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.992158:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e63032a0b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1d4a expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298641.992165:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63032a12 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1d43 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.992188:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243063b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604844928:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298641.992203:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b6a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604845056:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.992229:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:1.0:1780298641.992235:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298641.993007:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b6a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604845056:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.993027:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243063b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604844928:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298641.993028:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63032a12 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d43 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.993045:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e63032a0b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d4a expref: 17 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298641.993049:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63032a12 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d43 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.993060:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e63032a0b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d4a expref: 17 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298641.993082:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36a00, rc 0) 00010000:00010000:0.0:1780298641.993085:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf200, rc 0) 00000100:00100000:1.0:1780298641.993118:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609326336:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1009us (1137us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.993120:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348007100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2399:x1866777609326464:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1002us (1108us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298641.993143:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8664 00000100:00100000:1.0:1780298641.993157:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8663 00000100:00100000:2.0:1780298641.994089:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298641.994118:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298641.994150:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609326592 00000100:00100000:1.0:1780298641.994156:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609326592 00000100:00100000:3.0:1780298641.994169:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609326720 00000100:00100000:3.0:1780298641.994175:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609326720 00000100:00100000:1.0:1780298641.994191:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8665 00000100:00100000:1.0:1780298641.994203:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609326592:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298641.994204:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8666 00010000:00010000:1.0:1780298641.994214:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298641.994215:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609326720:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.994222:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298641.994226:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e63032a19 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1d51 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.994230:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e63032a20 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1d58 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.994270:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243063b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604845184:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298641.994277:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b6a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604845312:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298641.994313:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393480 going to sleep for 11 seconds 00000100:00100000:1.0:1780298641.994323:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12a80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298641.994981:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243063b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604845184:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.994997:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e63032a20 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d58 expref: 17 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298641.995007:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b6a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604845312:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298641.995009:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e63032a20 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d58 expref: 17 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.995018:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baec00, rc 0) 00010000:00010000:1.0:1780298641.995035:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e63032a19 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d51 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.995044:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609326720:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 830us (924us total) trans 0 rc 301/301 00010000:00010000:1.0:1780298641.995050:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e63032a19 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d51 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.995051:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8666 00010000:00010000:1.0:1780298641.995064:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37000, rc 0) 00000100:00100000:1.0:1780298641.995090:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609326592:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 891us (1002us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298641.995098:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8665 00000100:00100000:3.0:1780298641.995988:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298641.996026:0:6476:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609326976 00000100:00100000:1.0:1780298641.996030:0:6476:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609326976 00000100:00100000:1.0:1780298641.996053:0:6476:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61906 00000100:00100000:1.0:1780298641.996060:0:6476:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348004700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+79:29360:x1866777609326976:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:1.0:1780298641.996087:0:6476:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298641.996097:0:6476:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b000/0xaf35132e630329f6 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 47 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d35 expref: 79 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298641.996105:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298641.996108:0:6476:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298641.996235:0:6476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37600/0xaf35132e63032a27 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 46 type: IBT flags: 0x40000000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298641.996244:0:6476:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9aa00/0xaf35132e63032a04 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 46 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d3c expref: 86 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298641.996253:0:6476:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37600/0xaf35132e63032a27 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 46 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298641.996269:0:6476:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9aa00/0xaf35132e63032a04 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 46 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d3c expref: 86 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298641.996279:0:6476:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9aa00/0xaf35132e63032a04 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 46 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d3c expref: 86 pid: 9976 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298641.996286:0:6476:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b6a300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6476:1866777604845440:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298641.996326:0:6476:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12f00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298641.996904:0:6476:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b6a300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6476:1866777604845440:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298641.996917:0:6476:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37600/0xaf35132e63032a27 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 46 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298641.996984:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.997016:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609327104 00000100:00100000:0.0:1780298641.997021:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609327104 00010000:00010000:0.0:1780298641.997041:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9aa00/0xaf35132e63032a04 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 46 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d3c expref: 87 pid: 9976 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.997067:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14937 00000100:00100000:0.0:1780298641.997076:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b1d050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+87:2399:x1866777609327104:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298641.997084:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298641.997088:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9aa00/0xaf35132e63032a04 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 47 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d3c expref: 87 pid: 9976 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.997095:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9aa00/0xaf35132e63032a04 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 47 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d3c expref: 87 pid: 9976 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.997105:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9aa00/0xaf35132e63032a04 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 47 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d3c expref: 87 pid: 9976 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.997111:0:20253:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298641.997114:0:20253:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37600/0xaf35132e63032a27 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 46 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.997121:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37600/0xaf35132e63032a27 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 46 type: IBT flags: 0x40290000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.997146:0:20253:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37600/0xaf35132e63032a27 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 46 type: IBT flags: 0x40290000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.997159:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298641.997173:0:6476:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37600/0xaf35132e63032a27 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 45 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.997180:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b1d050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+86:2399:x1866777609327104:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 106us (197us total) trans 0 rc 0/0 00010000:00010000:1.0:1780298641.997185:0:6476:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37600/0xaf35132e63032a27 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 45 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298641.997190:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14937 00000001:00080000:1.0:1780298641.997240:0:6476:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294990992 00000004:00010000:1.0:1780298641.997261:0:6476:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 000000009aeae017 reply state 00000000b29eb423 transno 4294990993 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37600/0xaf35132e63032a27 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 45 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298641.997272:0:6476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37600/0xaf35132e63032a27 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 45 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298641.997280:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298641.997282:0:6476:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1780298641.997285:0:6476:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1780298641.997350:0:6476:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348004700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+81:29360:x1866777609326976:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1290us (1363us total) trans 4294990993 rc 0/0 00000100:00100000:1.0:1780298641.997357:0:6476:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61906 00000020:00080000:1.0:1780298641.997361:0:6476:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1780298641.998094:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298641.998143:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298641.998162:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609327232 00000100:00100000:2.0:1780298641.998168:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609327232 00000100:00100000:1.0:1780298641.998169:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609327360 00000100:00100000:1.0:1780298641.998176:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609327360 00002000:00010000:2.0:1780298641.998186:0:8363:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (376626->18446744073709551615) 00002000:00100000:2.0:1780298641.998190:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf348006680 with x1866777609327232 ext(376626->18446744073709551615) 00002000:00010000:1.0:1780298641.998193:0:19949:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00002000:00010000:2.0:1780298641.998196:0:8363:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf348006680. 00002000:00100000:1.0:1780298641.998199:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e56d880 with x1866777609327360 ext(0->18446744073709551615) 00002000:00010000:1.0:1780298641.998207:0:19949:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf36e56d880. 00000100:00100000:2.0:1780298641.998218:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5904 00000100:00100000:2.0:1780298641.998224:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348006680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+19:2400:x1866777609327232:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:0.0:1780298641.998238:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5905 00000100:00100000:0.0:1780298641.998259:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2400:x1866777609327360:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:0.0:1780298641.998551:0:8364:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 3202397 -> 0 00002000:00010000:0.0:1780298641.998555:0:8364:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298641.998557:0:8364:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298641.998559:0:8364:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298641.998562:0:8364:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 24 -> 0 00002000:00010000:2.0:1780298641.998572:0:8363:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 376626 00002000:00010000:2.0:1780298641.998577:0:8363:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298641.998579:0:8363:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298641.998581:0:8363:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298641.998583:0:8363:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 320 -> 296 00000100:00100000:0.0:1780298641.998595:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+19:2400:x1866777609327360:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 348us (468us total) trans 4294969166 rc 0/0 00000100:00100000:0.0:1780298641.998602:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5905 00002000:00010000:0.0:1780298641.998606:0:8364:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00000100:00100000:2.0:1780298641.998612:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348006680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2400:x1866777609327232:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 385us (545us total) trans 4294969594 rc 0/0 00002000:00100000:0.0:1780298641.998612:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e56d880 with x1866777609327360 ext(0->18446744073709551615) 00002000:00010000:0.0:1780298641.998619:0:8364:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf36e56d880. 00000100:00100000:2.0:1780298641.998624:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5904 00002000:00010000:2.0:1780298641.998628:0:8363:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (376626->18446744073709551615) 00002000:00100000:2.0:1780298641.998631:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf348006680 with x1866777609327232 ext(376626->18446744073709551615) 00002000:00010000:2.0:1780298641.998638:0:8363:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf348006680. 00000100:00100000:2.0:1780298641.999384:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298641.999417:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609327488 00000100:00100000:0.0:1780298641.999422:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609327488 00000100:00100000:0.0:1780298641.999458:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61907 00000100:00100000:0.0:1780298641.999465:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+86:29360:x1866777609327488:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298641.999475:0:6477:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298641.999485:0:6477:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae400/0xaf35132e63032a2e lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 46 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1d5f expref: -99 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.999534:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafa00/0xaf35132e63032a35 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 47 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.999547:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafa00/0xaf35132e63032a35 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 47 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.999555:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafa00/0xaf35132e63032a35 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 47 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1780298641.999572:0:6477:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 376626 00000004:00200000:0.0:1780298641.999575:0:6477:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1780298641.999586:0:6477:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafa00/0xaf35132e63032a35 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 47 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.999604:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae400/0xaf35132e63032a2e lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 47 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d5f expref: 88 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.999617:0:6477:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafa00/0xaf35132e63032a35 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 46 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d5f expref: 87 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298641.999624:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298641.999626:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1780298641.999629:0:6477:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bafa00, rc 0) 00000100:00100000:0.0:1780298641.999656:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+87:29360:x1866777609327488:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 192us (274us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298641.999662:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61907 00000100:00100000:3.0:1780298642.000492:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.000531:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609327616 00000100:00100000:1.0:1780298642.000536:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609327616 00000100:00100000:1.0:1780298642.000578:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8667 00000100:00100000:1.0:1780298642.000587:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609327616:12345-192.168.203.25@tcp:101:fsx.0 00000100:00080000:2.0:1780298642.000592:0:6473:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:1.0:1780298642.000595:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.000605:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e63032a3c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1d66 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.000625:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.000647:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b6a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604845568:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.000658:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609327744 00000100:00100000:0.0:1780298642.000664:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609327744 00000100:00080000:2.0:1780298642.000701:0:6473:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1780298642.000721:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8668 00000100:00100000:1.0:1780298642.000722:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12a80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.000735:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2398:x1866777609327744:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.000744:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.000755:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a3f03a/0xaf35132e63032a43 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1d6d expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.000784:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243062d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604845696:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.000818:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c780 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.001327:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b6a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604845568:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.001346:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e63032a3c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d66 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.001370:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e63032a3c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d66 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.001378:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36c00, rc 0) 00000100:00100000:1.0:1780298642.001404:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609327616:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 819us (912us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.001409:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8667 00000100:00100000:0.0:1780298642.001467:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243062d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604845696:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.001482:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000001a3f03a/0xaf35132e63032a43 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d6d expref: 17 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.001494:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a3f03a/0xaf35132e63032a43 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d6d expref: 17 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.001500:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae600, rc 0) 00000100:00100000:0.0:1780298642.001523:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2398:x1866777609327744:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 795us (899us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.001528:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8668 00000100:00100000:2.0:1780298642.002393:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.002432:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609327872 00000100:00100000:0.0:1780298642.002437:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609327872 00000100:00100000:0.0:1780298642.002473:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8669 00000100:00100000:0.0:1780298642.002483:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243809180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609327872:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.002492:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298642.002500:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:0.0:1780298642.002502:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63032a4a lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1d74 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.002536:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243062d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604845824:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.002538:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609328000 00000100:00100000:1.0:1780298642.002544:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609328000 00000100:00100000:1.0:1780298642.002581:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8670 00000100:00100000:1.0:1780298642.002591:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609328000:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298642.002598:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c780 going to sleep for 11 seconds 00010000:00010000:1.0:1780298642.002602:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.002612:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009cb6f854/0xaf35132e63032a51 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1d7b expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.002640:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b6a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604845952:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.002675:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12a80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.003198:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243062d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604845824:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.003211:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63032a4a lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d74 expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.003222:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63032a4a lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d74 expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.003229:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baee00, rc 0) 00000100:00100000:0.0:1780298642.003252:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243809180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609327872:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 773us (861us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.003257:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8669 00000100:00100000:1.0:1780298642.003404:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b6a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604845952:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.003424:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009cb6f854/0xaf35132e63032a51 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d7b expref: 17 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.003439:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009cb6f854/0xaf35132e63032a51 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d7b expref: 17 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.003449:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36e00, rc 0) 00000100:00100000:1.0:1780298642.003483:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609328000:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 893us (982us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.003491:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8670 00000100:00100000:3.0:1780298642.004381:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.004417:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609328128 00000100:00100000:1.0:1780298642.004423:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609328128 00000100:00100000:1.0:1780298642.004466:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8671 00000100:00100000:1.0:1780298642.004476:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609328128:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.004485:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.004494:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bf17d487/0xaf35132e63032a58 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1d82 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.004518:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e630329a2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1cfd expref: 19 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.004528:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298641 v=(80 177) 00010000:00010000:1.0:1780298642.004532:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(81 177) 00010000:00010000:1.0:1780298642.004536:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000bf17d487/0xaf35132e63032a58 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [352256->360447] (req 352256->360447) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d82 expref: 10 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.004558:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e630329a2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1cfd expref: 19 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.004572:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e630329a2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1cfd expref: 19 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.004585:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b6a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604846080:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298642.004625:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.005204:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b6a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604846080:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298642.005219:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bf17d487/0xaf35132e63032a58 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [352256->360447] (req 352256->360447) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d82 expref: 10 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.005231:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37400, rc 0) 00000100:00100000:1.0:1780298642.005261:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609328128:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 787us (879us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.005272:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8671 00000100:00100000:3.0:1780298642.005455:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.005494:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609328256 00000100:00100000:0.0:1780298642.005499:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609328256 00010000:00010000:0.0:1780298642.005520:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e630329a2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1cfd expref: 20 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.005554:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14938 00000100:00100000:0.0:1780298642.005562:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bf800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2399:x1866777609328256:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298642.005572:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298642.005585:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e630329a2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1cfd expref: 20 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.005597:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e630329a2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1cfd expref: 20 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.005620:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e630329a2 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1cfd expref: 20 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.005630:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(81 177) 00010000:00010000:0.0:1780298642.005634:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(81 177) 00010000:00010000:0.0:1780298642.005642:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000bf17d487/0xaf35132e63032a58 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 352256->360447) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d82 expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.005651:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000bf17d487/0xaf35132e63032a58 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 352256->360447) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d82 expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.005674:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000bf17d487/0xaf35132e63032a58 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 352256->360447) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d82 expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.005684:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243062d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777604846208:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298642.005721:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c780 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.006412:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243062d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777604846208:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298642.006423:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298642.006449:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bf800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+19:2399:x1866777609328256:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 888us (995us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.006456:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14938 00000100:00100000:3.0:1780298642.006682:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.006718:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609328384 00000100:00100000:1.0:1780298642.006723:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609328384 00002000:00100000:1.0:1780298642.006748:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (352256->360447) req@ffff9cf255492300 x1866777609328384/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298642.006758:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255492300 with x1866777609328384 ext(352256->360447) 00002000:00010000:1.0:1780298642.006762:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255492300 00000100:00100000:1.0:1780298642.006784:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5906 00000100:00100000:1.0:1780298642.006791:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255492300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609328384:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298642.006955:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255492300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609328384:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 164us (272us total) trans 0 rc 8192/8192 00000100:00100000:1.0:1780298642.006963:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5906 00002000:00100000:1.0:1780298642.006970:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (352256->360447) req@ffff9cf255492300 x1866777609328384/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/8632 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 8192/8192 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298642.006979:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255492300 with x1866777609328384 ext(352256->360447) 00002000:00010000:1.0:1780298642.006982:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255492300 00000100:00100000:3.0:1780298642.008090:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.008146:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609328512 00000100:00100000:0.0:1780298642.008151:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609328512 00000100:00100000:0.0:1780298642.008180:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8672 00000100:00100000:0.0:1780298642.008189:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255493480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609328512:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.008200:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.008212:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e63032a5f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1d89 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.008267:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243062d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604846336:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.008328:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c780 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.009065:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243062d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604846336:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.009085:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e63032a5f lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d89 expref: 17 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.009101:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e63032a5f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d89 expref: 17 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.009112:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baec00, rc 0) 00000100:00100000:0.0:1780298642.009171:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255493480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609328512:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 983us (1081us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.009179:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8672 00000100:00100000:3.0:1780298642.010030:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.010076:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609328640 00000100:00100000:2.0:1780298642.010083:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609328640 00000100:00100000:2.0:1780298642.010116:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8673 00000100:00100000:2.0:1780298642.010142:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255490e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2399:x1866777609328640:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.010154:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.010165:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad103424/0xaf35132e63032a66 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1d90 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.010200:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243808380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604846464:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.010247:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.011002:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243808380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604846464:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.011020:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ad103424/0xaf35132e63032a66 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d90 expref: 17 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.011035:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad103424/0xaf35132e63032a66 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d90 expref: 17 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.011046:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9aa00, rc 0) 00000100:00100000:2.0:1780298642.011076:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255490e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2399:x1866777609328640:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 953us (1047us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.011083:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8673 00000100:00100000:0.0:1780298642.012182:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.012216:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609328768 00000100:00100000:1.0:1780298642.012222:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609328768 00000100:00100000:1.0:1780298642.012256:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8674 00000100:00100000:1.0:1780298642.012264:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243062d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609328768:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.012274:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.012286:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f68a0cb8/0xaf35132e63032a6d lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1d97 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.012324:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000bf17d487/0xaf35132e63032a58 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 352256->360447) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d82 expref: 11 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.012333:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(81 177) 00010000:00010000:1.0:1780298642.012337:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(81 177) 00010000:00010000:1.0:1780298642.012343:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f68a0cb8/0xaf35132e63032a6d lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [974848->983039] (req 974848->983039) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d97 expref: 11 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.012374:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000bf17d487/0xaf35132e63032a58 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 352256->360447) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d82 expref: 11 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.012388:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000bf17d487/0xaf35132e63032a58 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 352256->360447) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d82 expref: 11 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.012399:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b6a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604846592:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298642.012440:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.013115:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b6a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604846592:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298642.013151:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f68a0cb8/0xaf35132e63032a6d lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [974848->983039] (req 974848->983039) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d97 expref: 11 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.013164:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37800, rc 0) 00000100:00100000:1.0:1780298642.013194:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243062d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:29360:x1866777609328768:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 931us (1012us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.013201:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8674 00000100:00100000:3.0:1780298642.013397:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.013434:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609328896 00000100:00100000:2.0:1780298642.013439:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609328896 00010000:00010000:2.0:1780298642.013455:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000bf17d487/0xaf35132e63032a58 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 352256->360447) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d82 expref: 11 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.013483:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14939 00000100:00100000:2.0:1780298642.013492:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255491f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609328896:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298642.013502:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298642.013516:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000bf17d487/0xaf35132e63032a58 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 352256->360447) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d82 expref: 11 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.013529:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000bf17d487/0xaf35132e63032a58 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 352256->360447) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d82 expref: 11 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.013546:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bf17d487/0xaf35132e63032a58 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 352256->360447) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d82 expref: 11 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.013557:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(81 177) 00010000:00010000:2.0:1780298642.013561:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(81 177) 00010000:00010000:2.0:1780298642.013569:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f68a0cb8/0xaf35132e63032a6d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [974848->34529279] (req 974848->983039) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d97 expref: 10 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.013579:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f68a0cb8/0xaf35132e63032a6d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [974848->34529279] (req 974848->983039) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d97 expref: 10 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.013602:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000f68a0cb8/0xaf35132e63032a6d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [974848->34529279] (req 974848->983039) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d97 expref: 10 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.013612:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243808380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777604846720:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298642.013653:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.014331:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243808380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777604846720:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298642.014342:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298642.014370:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255491f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609328896:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 880us (974us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.014382:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14939 00000100:00100000:0.0:1780298642.014954:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.014990:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609329024 00000100:00100000:1.0:1780298642.014994:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609329024 00000100:00100000:1.0:1780298642.015030:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8675 00000100:00100000:1.0:1780298642.015040:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243060000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609329024:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298642.015049:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:1.0:1780298642.015054:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.015065:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e63032a74 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1d9e expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.015085:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609329152 00000100:00100000:2.0:1780298642.015090:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609329152 00000100:00100000:1.0:1780298642.015101:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b6a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604846848:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.015150:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8676 00000100:00100000:2.0:1780298642.015162:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243063b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2399:x1866777609329152:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298642.015169:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12a80 going to sleep for 11 seconds 00010000:00010000:2.0:1780298642.015172:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.015182:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000019780e8d/0xaf35132e63032a7b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1da5 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.015214:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243808380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604846976:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.015252:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00002000:00010000:1.0:1780298642.015848:0:8358:0:(ofd_lvb.c:259:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size: 376626 -> 979279 00000100:00100000:1.0:1780298642.015863:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b6a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604846848:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.015882:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e63032a74 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d9e expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.015899:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e63032a74 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d9e expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.015909:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37000, rc 0) 00000100:00100000:1.0:1780298642.015940:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243060000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609329024:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 903us (988us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.015949:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8675 00000100:00100000:2.0:1780298642.015967:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243808380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604846976:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.015991:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000019780e8d/0xaf35132e63032a7b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1da5 expref: 17 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.016006:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000019780e8d/0xaf35132e63032a7b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1da5 expref: 17 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.016017:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9a200, rc 0) 00000100:00100000:2.0:1780298642.016050:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243063b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2399:x1866777609329152:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 891us (1006us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.016058:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8676 00000100:00100000:0.0:1780298642.016952:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.016985:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609329280 00000100:00100000:2.0:1780298642.016989:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609329280 00000100:00100000:2.0:1780298642.017017:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8677 00000100:00100000:2.0:1780298642.017023:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243062300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609329280:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.017030:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.017037:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63032a82 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1dac expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.017060:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243808380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604847104:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.017096:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.017096:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.017155:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609329408 00000100:00100000:1.0:1780298642.017159:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609329408 00000100:00100000:1.0:1780298642.017190:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8678 00000100:00100000:1.0:1780298642.017196:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609329408:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.017203:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.017211:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009cb6f854/0xaf35132e63032a89 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1db3 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.017236:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b6a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604847232:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.017273:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.017688:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243808380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604847104:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.017703:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63032a82 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1dac expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.017716:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63032a82 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1dac expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.017726:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9be00, rc 0) 00000100:00100000:2.0:1780298642.017754:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243062300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609329280:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 731us (802us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.017761:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8677 00000100:00100000:1.0:1780298642.017881:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b6a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604847232:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.017899:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009cb6f854/0xaf35132e63032a89 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1db3 expref: 17 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.017913:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009cb6f854/0xaf35132e63032a89 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1db3 expref: 17 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.017923:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36e00, rc 0) 00000100:00100000:1.0:1780298642.017953:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609329408:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 756us (897us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.017962:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8678 00000100:00100000:0.0:1780298642.019176:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.019212:0:7411:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609329536 00000100:00100000:2.0:1780298642.019218:0:7411:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609329536 00000100:00080000:3.0:1780298642.019256:0:6473:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1780298642.019268:0:7411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61908 00000100:00100000:2.0:1780298642.019278:0:7411:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243061880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+81:29360:x1866777609329536:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.019304:0:7411:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.019319:0:7411:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9aa00/0xaf35132e63032a90 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 47 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1dba expref: -99 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.019410:0:7411:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b800/0xaf35132e63032a97 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 48 type: IBT flags: 0x40000000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.019425:0:7411:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b800/0xaf35132e63032a97 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 48 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.019436:0:7411:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b800/0xaf35132e63032a97 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 48 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298642.019460:0:7411:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 376626 00000004:00200000:2.0:1780298642.019473:0:7411:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298642.019491:0:7411:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b800/0xaf35132e63032a97 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 48 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.019512:0:7411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9aa00/0xaf35132e63032a90 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 48 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1dba expref: 83 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.019524:0:7411:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b800/0xaf35132e63032a97 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 47 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1dba expref: 82 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.019532:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.019535:0:7411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298642.019538:0:7411:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b800, rc 0) 00000100:00100000:2.0:1780298642.019569:0:7411:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243061880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+82:29360:x1866777609329536:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 293us (394us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.019576:0:7411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61908 00000100:00100000:0.0:1780298642.020490:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.020527:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609329664 00000100:00100000:2.0:1780298642.020531:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609329664 00000100:00100000:2.0:1780298642.020555:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8679 00000100:00100000:2.0:1780298642.020560:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e60f050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+19:2400:x1866777609329664:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.020566:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.020573:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009fd8d343/0xaf35132e63032a9e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1dc1 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.020600:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243808380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604847360:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.020636:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.021224:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243808380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604847360:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.021237:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009fd8d343/0xaf35132e63032a9e lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1dc1 expref: 20 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.021258:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009fd8d343/0xaf35132e63032a9e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1dc1 expref: 20 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.021265:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b000, rc 0) 00000100:00100000:2.0:1780298642.021286:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e60f050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+19:2400:x1866777609329664:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 726us (798us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.021290:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8679 00000100:00100000:0.0:1780298642.022179:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.022212:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609329792 00000100:00100000:2.0:1780298642.022216:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609329792 00000100:00100000:2.0:1780298642.022239:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8680 00000100:00100000:2.0:1780298642.022244:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243063800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+19:29360:x1866777609329792:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.022251:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.022258:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000991eb57d/0xaf35132e63032aa5 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1dc8 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.022272:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f68a0cb8/0xaf35132e63032a6d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [974848->34529279] (req 974848->983039) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d97 expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.022279:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(81 177) 00010000:00010000:2.0:1780298642.022282:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(82 177) 00010000:00010000:2.0:1780298642.022285:0:20252:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000991eb57d/0xaf35132e63032aa5 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3096576->3108863] (req 3096576->3108863) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1dc8 expref: 20 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.022313:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f68a0cb8/0xaf35132e63032a6d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [974848->34529279] (req 974848->983039) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d97 expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.022323:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f68a0cb8/0xaf35132e63032a6d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [974848->34529279] (req 974848->983039) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d97 expref: 9 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.022330:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243809180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604847488:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298642.022360:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.022960:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243809180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604847488:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298642.022971:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000991eb57d/0xaf35132e63032aa5 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3096576->3108863] (req 3096576->3108863) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1dc8 expref: 20 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.022979:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244f3a200, rc 0) 00000100:00100000:2.0:1780298642.023003:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243063800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:29360:x1866777609329792:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 757us (827us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.023010:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8680 00000100:00100000:3.0:1780298642.023260:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.023307:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609329920 00000100:00100000:2.0:1780298642.023312:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609329920 00002000:00100000:2.0:1780298642.023338:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (974848->979278) req@ffff9cf24310dc00 x1866777609329920/t0(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 4920/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.023346:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24310dc00 with x1866777609329920 ext(974848->979278) 00010000:00010000:2.0:1780298642.023350:0:8362:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000f68a0cb8/0xaf35132e63032a6d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [974848->34529279] (req 974848->983039) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d97 expref: 10 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.023359:0:8362:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000f68a0cb8/0xaf35132e63032a6d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [974848->34529279] (req 974848->983039) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d97 expref: 10 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1780298642.023365:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf24310dc00 00000100:00100000:2.0:1780298642.023380:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 181 00000100:00100000:2.0:1780298642.023386:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609329920:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:2.0:1780298642.023888:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2399:x1866777609329920:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 502us (628us total) trans 4294969595 rc 0/0 00000100:00100000:2.0:1780298642.023893:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 181 00002000:00100000:2.0:1780298642.023898:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (974848->979278) req@ffff9cf24310dc00 x1866777609329920/t4294969595(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 4920/448 e 0 to 0 dl 1780298653 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.023905:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24310dc00 with x1866777609329920 ext(974848->979278) 00010000:00010000:2.0:1780298642.023909:0:8362:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000f68a0cb8/0xaf35132e63032a6d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [974848->34529279] (req 974848->983039) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d97 expref: 12 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.023918:0:8362:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000f68a0cb8/0xaf35132e63032a6d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [974848->34529279] (req 974848->983039) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d97 expref: 12 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1780298642.023926:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf24310dc00 00000100:00100000:0.0:1780298642.024656:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.024689:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609330048 00000100:00100000:2.0:1780298642.024693:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609330048 00010000:00010000:2.0:1780298642.024711:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000f68a0cb8/0xaf35132e63032a6d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [974848->34529279] (req 974848->983039) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d97 expref: 12 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.024739:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14940 00000100:00100000:2.0:1780298642.024747:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243063480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609330048:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298642.024756:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:2.0:1780298642.024765:0:8374:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 296 -> 312 00010000:00010000:2.0:1780298642.024768:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f68a0cb8/0xaf35132e63032a6d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [974848->34529279] (req 974848->983039) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d97 expref: 12 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.024776:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f68a0cb8/0xaf35132e63032a6d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [974848->34529279] (req 974848->983039) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d97 expref: 12 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.024789:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f68a0cb8/0xaf35132e63032a6d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [974848->34529279] (req 974848->983039) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d97 expref: 12 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.024796:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(82 177) 00010000:00010000:2.0:1780298642.024799:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(82 177) 00010000:00010000:2.0:1780298642.024804:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000991eb57d/0xaf35132e63032aa5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3096576->36651007] (req 3096576->3108863) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1dc8 expref: 19 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.024809:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000991eb57d/0xaf35132e63032aa5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3096576->36651007] (req 3096576->3108863) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1dc8 expref: 19 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.024827:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000991eb57d/0xaf35132e63032aa5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3096576->36651007] (req 3096576->3108863) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1dc8 expref: 19 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.024835:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243809180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777604847616:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298642.024865:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.025581:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243809180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777604847616:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298642.025596:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298642.025627:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243063480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609330048:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 880us (970us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.025636:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14940 00000100:00100000:0.0:1780298642.025737:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.025777:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609330176 00000100:00100000:1.0:1780298642.025783:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609330176 00000100:00100000:1.0:1780298642.025824:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5907 00000100:00100000:1.0:1780298642.025833:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243062d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2399:x1866777609330176:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298642.025944:0:19951:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 979279 -> 3106995 00002000:00010000:1.0:1780298642.025948:0:19951:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:1.0:1780298642.025950:0:19951:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:1.0:1780298642.025953:0:19951:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:1.0:1780298642.025955:0:19951:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 312 -> 336 00000100:00100000:1.0:1780298642.025986:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243062d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2399:x1866777609330176:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 155us (251us total) trans 4294969596 rc 0/0 00000100:00100000:1.0:1780298642.025994:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5907 00000100:00100000:3.0:1780298642.026890:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.026928:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609330304 00000100:00100000:1.0:1780298642.026932:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609330304 00000100:00100000:1.0:1780298642.026963:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8681 00000100:00100000:1.0:1780298642.026971:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2398:x1866777609330304:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.026980:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.026989:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e63032aac lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1dcf expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.027028:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b6a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604847744:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.027079:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.027810:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b6a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604847744:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.027829:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e63032aac lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1dcf expref: 22 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.027847:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e63032aac lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1dcf expref: 22 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.027857:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37000, rc 0) 00000100:00100000:1.0:1780298642.027887:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2398:x1866777609330304:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 917us (998us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.027894:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8681 00000100:00100000:0.0:1780298642.028796:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.028836:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609330432 00000100:00100000:1.0:1780298642.028841:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609330432 00000100:00100000:1.0:1780298642.028874:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8682 00000100:00100000:1.0:1780298642.028882:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243060000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2399:x1866777609330432:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.028891:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.028901:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e63032ab3 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1dd6 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.028929:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b6a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604847872:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.028969:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12a80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.029654:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.029690:0:9976:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609330688 00000100:00100000:2.0:1780298642.029695:0:9976:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609330688 00000100:00100000:3.0:1780298642.029704:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00080000:0.0:1780298642.029756:0:6473:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1780298642.029757:0:9976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1375 00000100:00100000:2.0:1780298642.029769:0:9976:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609330688:12345-192.168.203.25@tcp:400:kworker.0 00000100:00100000:0.0:1780298642.029779:0:6471:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609330560 00000100:00100000:1.0:1780298642.029780:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b6a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604847872:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.029785:0:6471:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609330560 00010000:00010000:1.0:1780298642.029801:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e63032ab3 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1dd6 expref: 22 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.029809:0:9976:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609330688:12345-192.168.203.25@tcp:400:kworker.0 Request processed in 42us (155us total) trans 0 rc 0/0 00010000:00010000:1.0:1780298642.029817:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e63032ab3 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1dd6 expref: 22 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.029822:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.029822:0:9976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1375 00000100:00100000:0.0:1780298642.029829:0:6471:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 576 00010000:00010000:1.0:1780298642.029840:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36c00, rc 0) 00000100:00100000:0.0:1780298642.029845:0:6471:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347c9a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4a8786ce-7c8a-40a7-8357-9ee004d06219+12:2400:x1866777609330560:12345-192.168.203.25@tcp:400:kworker.0 00000100:00100000:2.0:1780298642.029847:0:9976:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609330816 00000100:00100000:2.0:1780298642.029851:0:9976:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609330816 00000100:00100000:1.0:1780298642.029874:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243060000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2399:x1866777609330432:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 992us (1079us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.029881:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8682 00000100:00100000:0.0:1780298642.029887:0:6471:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347c9a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4a8786ce-7c8a-40a7-8357-9ee004d06219+12:2400:x1866777609330560:12345-192.168.203.25@tcp:400:kworker.0 Request processed in 46us (182us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.029892:0:9976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1376 00000100:00100000:0.0:1780298642.029901:0:6471:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 576 00000100:00100000:2.0:1780298642.029906:0:9976:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+5:2400:x1866777609330816:12345-192.168.203.25@tcp:400:kworker.0 00000100:00080000:1.0:1780298642.029919:0:6473:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1780298642.029942:0:9976:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+5:2400:x1866777609330816:12345-192.168.203.25@tcp:400:kworker.0 Request processed in 40us (128us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.029953:0:9976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1376 00000100:00100000:0.0:1780298642.030881:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.030912:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609330944 00000100:00100000:2.0:1780298642.030916:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609330944 00000100:00100000:2.0:1780298642.030939:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8683 00000100:00100000:2.0:1780298642.030944:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243063b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2397:x1866777609330944:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.030951:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.030959:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000065a1d570/0xaf35132e63032aba lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1ddd expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.030982:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243809180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604848000:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.031014:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.031725:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243809180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604848000:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.031740:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000065a1d570/0xaf35132e63032aba lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ddd expref: 22 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.031750:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000065a1d570/0xaf35132e63032aba lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ddd expref: 22 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.031757:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244f3ac00, rc 0) 00000100:00100000:2.0:1780298642.031781:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243063b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2397:x1866777609330944:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 837us (902us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.031786:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8683 00000100:00100000:1.0:1780298642.032558:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.032605:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609331072 00000100:00100000:2.0:1780298642.032612:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609331072 00000100:00100000:2.0:1780298642.032645:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8684 00000100:00100000:2.0:1780298642.032654:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b6a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2399:x1866777609331072:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.032662:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.032672:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c60b4886/0xaf35132e63032ac1 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1de4 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.032703:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243809180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604848128:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.032746:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.033419:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243809180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604848128:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.033436:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c60b4886/0xaf35132e63032ac1 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1de4 expref: 22 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.033448:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c60b4886/0xaf35132e63032ac1 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1de4 expref: 22 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.033455:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244f3a800, rc 0) 00000100:00100000:2.0:1780298642.033478:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b6a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2399:x1866777609331072:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 827us (923us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.033482:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8684 00000100:00100000:3.0:1780298642.034263:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.034306:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609331200 00000100:00100000:1.0:1780298642.034311:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609331200 00000100:00100000:1.0:1780298642.034341:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8685 00000100:00100000:1.0:1780298642.034350:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:29360:x1866777609331200:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.034359:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.034368:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63032ac8 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1deb expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.034386:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(82 177) 00010000:00010000:1.0:1780298642.034390:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(82 177) 00010000:00010000:1.0:1780298642.034400:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63032ac8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [307200->3096575] (req 307200->311295) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1deb expref: 22 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.034415:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63032ac8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [307200->3096575] (req 307200->311295) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1deb expref: 22 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.034424:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36a00, rc 0) 00000100:00100000:1.0:1780298642.034453:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:29360:x1866777609331200:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 105us (191us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.034460:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8685 00000100:00100000:1.0:1780298642.035113:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.035231:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609331328 00000100:00100000:3.0:1780298642.035236:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609331328 00000100:00100000:3.0:1780298642.035272:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5908 00000100:00100000:3.0:1780298642.035280:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b6bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2399:x1866777609331328:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:3.0:1780298642.035431:0:19928:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 3106995 -> 3106995 00002000:00010000:3.0:1780298642.035434:0:19928:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:3.0:1780298642.035437:0:19928:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:3.0:1780298642.035439:0:19928:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:3.0:1780298642.035442:0:19928:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 336 -> 336 00000100:00100000:3.0:1780298642.035510:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b6bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+23:2399:x1866777609331328:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 230us (396us total) trans 4294969597 rc 0/0 00000100:00100000:3.0:1780298642.035517:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5908 00000100:00100000:0.0:1780298642.036401:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.036423:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.036437:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609331456 00000100:00100000:1.0:1780298642.036443:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609331456 00000100:00100000:3.0:1780298642.036466:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609331584 00000100:00100000:3.0:1780298642.036472:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609331584 00000100:00100000:1.0:1780298642.036488:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8686 00000100:00100000:1.0:1780298642.036498:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243062300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609331456:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.036507:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298642.036514:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8687 00010000:00010000:1.0:1780298642.036516:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e63032acf lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1df2 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.036526:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243060a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609331584:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.036538:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.036548:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a860e235/0xaf35132e63032ad6 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1df9 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.036571:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b6aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604848256:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.036585:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243809180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604848384:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.036627:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.036627:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.037377:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b6aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604848256:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.037380:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243809180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604848384:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.037397:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e63032acf lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1df2 expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.037397:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a860e235/0xaf35132e63032ad6 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1df9 expref: 17 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.037414:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e63032acf lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1df2 expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.037414:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a860e235/0xaf35132e63032ad6 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1df9 expref: 17 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.037424:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37200, rc 0) 00010000:00010000:0.0:1780298642.037424:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244f3b400, rc 0) 00000100:00100000:1.0:1780298642.037454:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243062300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609331456:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 958us (1055us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.037454:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243060a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609331584:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 933us (1030us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.037468:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8687 00000100:00100000:1.0:1780298642.037473:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8686 00000100:00100000:0.0:1780298642.038419:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.038461:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609331712 00000100:00100000:2.0:1780298642.038466:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609331712 00000100:00100000:0.0:1780298642.038481:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.038501:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609331840 00000100:00100000:2.0:1780298642.038504:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609331840 00000100:00100000:1.0:1780298642.038533:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8688 00000100:00100000:2.0:1780298642.038545:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8689 00000100:00100000:1.0:1780298642.038546:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243061f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609331712:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.038556:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298642.038562:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243060380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609331840:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.038564:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003c8dab73/0xaf35132e63032add lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1e00 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.038570:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.038589:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009856b175/0xaf35132e63032ae4 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1e07 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.038589:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b6aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604848512:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.038627:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243808380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604848640:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.038628:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.038656:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.039303:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b6aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604848512:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.039322:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003c8dab73/0xaf35132e63032add lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e00 expref: 12 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.039334:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003c8dab73/0xaf35132e63032add lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e00 expref: 12 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.039341:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36800, rc 0) 00000100:00100000:2.0:1780298642.039363:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243808380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604848640:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.039373:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243061f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609331712:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 832us (955us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.039379:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8688 00010000:00010000:2.0:1780298642.039385:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009856b175/0xaf35132e63032ae4 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e07 expref: 17 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.039400:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009856b175/0xaf35132e63032ae4 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e07 expref: 17 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.039411:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244f3b200, rc 0) 00000100:00100000:2.0:1780298642.039433:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243060380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609331840:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 877us (953us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.039440:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8689 00000100:00100000:2.0:1780298642.040346:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.040395:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.040424:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609331968 00000100:00100000:0.0:1780298642.040428:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609331968 00000100:00100000:1.0:1780298642.040431:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609332096 00000100:00100000:1.0:1780298642.040435:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609332096 00000100:00100000:0.0:1780298642.040468:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8690 00000100:00100000:1.0:1780298642.040477:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8691 00000100:00100000:0.0:1780298642.040480:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243808380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609331968:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298642.040490:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243809c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609332096:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.040490:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.040500:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.040502:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000441cddf1/0xaf35132e63032aeb lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1e0e expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.040509:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f68a0cb8/0xaf35132e63032af2 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1e15 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.040541:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b6aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604848768:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.040549:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243062680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604848896:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.040578:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12a80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.040582:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.041303:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b6aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604848768:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.041320:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f68a0cb8/0xaf35132e63032af2 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e15 expref: 17 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.041331:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243062680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604848896:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.041334:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f68a0cb8/0xaf35132e63032af2 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e15 expref: 17 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.041344:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37800, rc 0) 00010000:00010000:0.0:1780298642.041357:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000441cddf1/0xaf35132e63032aeb lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e0e expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.041371:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243809c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609332096:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 884us (975us total) trans 0 rc 301/301 00010000:00010000:0.0:1780298642.041376:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000441cddf1/0xaf35132e63032aeb lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e0e expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.041383:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8691 00010000:00010000:0.0:1780298642.041395:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf400, rc 0) 00000100:00100000:0.0:1780298642.041419:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243808380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609331968:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 943us (1073us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.041427:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8690 00000100:00100000:2.0:1780298642.042318:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.042343:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609332224 00000100:00100000:3.0:1780298642.042348:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609332224 00000100:00100000:3.0:1780298642.042372:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8692 00000100:00100000:3.0:1780298642.042377:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24380a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:29360:x1866777609332224:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.042384:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.042390:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63032af9 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1e1c expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.042405:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63032ac8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [307200->3096575] (req 307200->311295) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1deb expref: 22 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.042411:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(82 177) 00010000:00010000:3.0:1780298642.042414:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(83 177) 00010000:00010000:3.0:1780298642.042417:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63032af9 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1855488->1859583] (req 1855488->1859583) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e1c expref: 12 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.042447:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63032ac8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [307200->3096575] (req 307200->311295) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1deb expref: 22 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.042458:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63032ac8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [307200->3096575] (req 307200->311295) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1deb expref: 22 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.042466:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604849024:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298642.042496:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348ce1840 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.043044:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604849024:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298642.043056:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63032af9 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1855488->1859583] (req 1855488->1859583) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e1c expref: 12 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.043069:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2000, rc 0) 00000100:00100000:3.0:1780298642.043093:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24380a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:29360:x1866777609332224:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 716us (776us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298642.043098:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8692 00000100:00100000:2.0:1780298642.043200:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.043233:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609332352 00000100:00100000:1.0:1780298642.043238:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609332352 00010000:00010000:1.0:1780298642.043268:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63032ac8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [307200->3096575] (req 307200->311295) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1deb expref: 23 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.043304:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14941 00000100:00100000:1.0:1780298642.043313:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24380a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+23:2400:x1866777609332352:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298642.043323:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298642.043335:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63032ac8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [307200->3096575] (req 307200->311295) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1deb expref: 23 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.043346:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63032ac8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [307200->3096575] (req 307200->311295) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1deb expref: 23 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.043363:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63032ac8 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [307200->3096575] (req 307200->311295) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1deb expref: 23 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.043374:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(83 177) 00010000:00010000:1.0:1780298642.043377:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(83 177) 00010000:00010000:1.0:1780298642.043386:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63032af9 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1855488->3096575] (req 1855488->1859583) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e1c expref: 11 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.043395:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63032af9 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1855488->3096575] (req 1855488->1859583) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e1c expref: 11 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.043419:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63032af9 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1855488->3096575] (req 1855488->1859583) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e1c expref: 11 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.043428:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b6aa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777604849152:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298642.043467:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.044150:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b6aa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777604849152:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298642.044163:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298642.044189:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24380a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2400:x1866777609332352:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 878us (989us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.044198:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14941 00000100:00100000:2.0:1780298642.044404:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.044432:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609332480 00000100:00100000:3.0:1780298642.044437:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609332480 00002000:00100000:3.0:1780298642.044462:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (1855488->1859583) req@ffff9cf243808700 x1866777609332480/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298642.044473:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243808700 with x1866777609332480 ext(1855488->1859583) 00002000:00010000:3.0:1780298642.044478:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf243808700 00000100:00100000:3.0:1780298642.044497:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5909 00000100:00100000:3.0:1780298642.044508:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243808700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2399:x1866777609332480:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298642.044652:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243808700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2399:x1866777609332480:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 144us (249us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1780298642.044657:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5909 00002000:00100000:3.0:1780298642.044661:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (1855488->1859583) req@ffff9cf243808700 x1866777609332480/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/4536 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298642.044667:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243808700 with x1866777609332480 ext(1855488->1859583) 00002000:00010000:3.0:1780298642.044670:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf243808700 00000100:00100000:2.0:1780298642.045559:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.045594:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609332608 00000100:00100000:1.0:1780298642.045599:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609332608 00002000:00100000:1.0:1780298642.045619:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (1859584->1863679) req@ffff9cf243808e00 x1866777609332608/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298642.045627:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243808e00 with x1866777609332608 ext(1859584->1863679) 00002000:00010000:1.0:1780298642.045630:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf243808e00 00000100:00100000:1.0:1780298642.045650:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5910 00000100:00100000:1.0:1780298642.045658:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243808e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609332608:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298642.045747:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243808e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609332608:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 90us (187us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298642.045759:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5910 00002000:00100000:1.0:1780298642.045766:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (1859584->1863679) req@ffff9cf243808e00 x1866777609332608/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/4536 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298642.045774:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243808e00 with x1866777609332608 ext(1859584->1863679) 00002000:00010000:1.0:1780298642.045778:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf243808e00 00000100:00100000:0.0:1780298642.046642:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.046672:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609332736 00000100:00100000:2.0:1780298642.046676:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609332736 00000100:00100000:2.0:1780298642.046700:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8693 00000100:00100000:2.0:1780298642.046706:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243062680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2397:x1866777609332736:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.046712:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.046720:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63032b00 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1e23 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.046748:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243809880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604849280:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.046781:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.047366:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243809880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604849280:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.047382:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63032b00 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e23 expref: 23 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.047394:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63032b00 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e23 expref: 23 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.047401:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9aa00, rc 0) 00000100:00100000:0.0:1780298642.047424:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243062680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2397:x1866777609332736:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 719us (783us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.047429:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8693 00000100:00100000:0.0:1780298642.048093:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.048141:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609332864 00000100:00100000:2.0:1780298642.048145:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609332864 00000100:00100000:2.0:1780298642.048167:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8694 00000100:00100000:2.0:1780298642.048172:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f754700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2397:x1866777609332864:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.048178:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.048184:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009fd8d343/0xaf35132e63032b07 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1e2a expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.048203:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243808000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604849408:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.048233:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4900 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.048794:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243808000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604849408:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.048806:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009fd8d343/0xaf35132e63032b07 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e2a expref: 23 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.048816:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009fd8d343/0xaf35132e63032b07 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e2a expref: 23 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.048822:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b000, rc 0) 00000100:00100000:1.0:1780298642.048845:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f754700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2397:x1866777609332864:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 673us (753us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.048849:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8694 00000100:00100000:0.0:1780298642.049545:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.049580:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609332992 00000100:00100000:2.0:1780298642.049584:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609332992 00000100:00100000:2.0:1780298642.049606:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8695 00000100:00100000:2.0:1780298642.049611:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f757480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:29360:x1866777609332992:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.049616:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.049622:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e63032b0e lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1e31 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.049634:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(83 177) 00010000:00010000:2.0:1780298642.049636:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(83 177) 00010000:00010000:2.0:1780298642.049642:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e63032b0e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1064960->1851391] (req 1064960->1073151) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e31 expref: 23 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.049651:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e63032b0e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1064960->1851391] (req 1064960->1073151) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e31 expref: 23 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.049657:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9a200, rc 0) 00000100:00100000:2.0:1780298642.049677:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f757480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+23:29360:x1866777609332992:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 66us (133us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.049681:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8695 00000100:00100000:0.0:1780298642.050279:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.050317:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609333120 00000100:00100000:2.0:1780298642.050321:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609333120 00000100:00100000:2.0:1780298642.050347:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5911 00000100:00100000:2.0:1780298642.050351:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f756a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+23:2400:x1866777609333120:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298642.050504:0:8362:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 3106995 -> 3106995 00002000:00010000:2.0:1780298642.050506:0:8362:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.050508:0:8362:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.050509:0:8362:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.050511:0:8362:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 336 -> 352 00000100:00100000:2.0:1780298642.050538:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f756a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+24:2400:x1866777609333120:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 186us (258us total) trans 4294969598 rc 0/0 00000100:00100000:2.0:1780298642.050546:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5911 00000100:00100000:0.0:1780298642.051348:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.051439:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609333248 00000100:00100000:0.0:1780298642.051464:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.051479:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609333248 00000100:00100000:1.0:1780298642.051511:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609333376 00000100:00100000:3.0:1780298642.051515:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8696 00000100:00100000:1.0:1780298642.051515:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609333376 00000100:00100000:3.0:1780298642.051537:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f754380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2399:x1866777609333248:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298642.051544:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8697 00010000:00010000:3.0:1780298642.051546:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.051556:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e63032b15 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1e38 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.051557:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2399:x1866777609333376:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.051573:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.051585:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63032b1c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1e3f expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.051596:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604849536:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.051621:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548b8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604849664:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.051653:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.051654:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348ce1840 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.052412:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604849536:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.052432:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e63032b15 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e38 expref: 13 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.052443:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e63032b15 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e38 expref: 13 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.052448:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548b8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604849664:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.052463:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2800, rc 0) 00010000:00010000:2.0:1780298642.052484:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63032b1c lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e3f expref: 17 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.052495:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f754380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2399:x1866777609333248:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 960us (1147us total) trans 0 rc 301/301 00010000:00010000:2.0:1780298642.052501:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63032b1c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e3f expref: 17 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.052507:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8696 00010000:00010000:2.0:1780298642.052519:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf400, rc 0) 00000100:00100000:2.0:1780298642.052542:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2399:x1866777609333376:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 990us (1077us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.052549:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8697 00000100:00100000:2.0:1780298642.053556:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.053588:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.053589:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609333504 00000100:00100000:1.0:1780298642.053594:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609333504 00000100:00100000:3.0:1780298642.053616:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609333632 00000100:00100000:3.0:1780298642.053621:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609333632 00000100:00100000:1.0:1780298642.053625:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8698 00000100:00100000:1.0:1780298642.053635:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24380b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609333504:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.053643:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.053652:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63032b23 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1e46 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.053656:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8699 00000100:00100000:3.0:1780298642.053665:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243808a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609333632:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.053672:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.053680:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ced6368f/0xaf35132e63032b2a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1e4d expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.053686:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b6aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604849792:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.053707:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604849920:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.053730:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12a80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.053733:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348ce1840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.054404:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b6aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604849792:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.054409:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604849920:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.054423:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63032b23 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e46 expref: 13 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.054429:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ced6368f/0xaf35132e63032b2a lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e4d expref: 17 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.054442:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ced6368f/0xaf35132e63032b2a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e4d expref: 17 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.054446:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63032b23 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e46 expref: 13 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.054452:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2a00, rc 0) 00010000:00010000:1.0:1780298642.054456:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36a00, rc 0) 00000100:00100000:3.0:1780298642.054488:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243808a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609333632:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 823us (899us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.054491:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24380b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609333504:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 859us (936us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.054499:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8698 00000100:00100000:3.0:1780298642.054507:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8699 00000100:00100000:0.0:1780298642.055518:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.055574:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609333760 00000100:00100000:2.0:1780298642.055581:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609333760 00000100:00100000:2.0:1780298642.055615:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8700 00000100:00100000:2.0:1780298642.055624:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:29360:x1866777609333760:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.055634:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.055645:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63032b31 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1e54 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.055663:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e63032b0e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1064960->1851391] (req 1064960->1073151) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e31 expref: 23 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.055677:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63032af9 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1855488->3096575] (req 1855488->1859583) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e1c expref: 13 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.055685:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 2 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(83 177) 00010000:00010000:2.0:1780298642.055688:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(85 177) 00010000:00010000:2.0:1780298642.055693:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63032b31 lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1847296->1859583] (req 1847296->1859583) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e54 expref: 13 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.055719:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63032af9 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1855488->3096575] (req 1855488->1859583) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e1c expref: 13 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.055733:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63032af9 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1855488->3096575] (req 1855488->1859583) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e1c expref: 13 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.055745:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24380bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604850048:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298642.055806:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e63032b0e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1064960->1851391] (req 1064960->1073151) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e31 expref: 23 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.055817:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e63032b0e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1064960->1851391] (req 1064960->1073151) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e31 expref: 23 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.055829:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24380b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604850176:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298642.055856:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.056661:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24380bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604850048:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298642.057021:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.057054:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.057094:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609334016 00000100:00100000:2.0:1780298642.057095:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609333888 00000100:00100000:3.0:1780298642.057101:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609334016 00000100:00100000:2.0:1780298642.057103:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609333888 00010000:00010000:2.0:1780298642.057155:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63032af9 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1855488->3096575] (req 1855488->1859583) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e1c expref: 14 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.057163:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e63032b0e lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [1064960->1851391] (req 1064960->1073151) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e31 expref: 24 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.057194:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14942 00000100:00100000:1.0:1780298642.057199:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24380b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604850176:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298642.057211:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14943 00000100:00100000:2.0:1780298642.057216:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609333888:12345-192.168.203.25@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1780298642.057221:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548bb480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+24:2399:x1866777609334016:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298642.057231:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298642.057233:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298642.057247:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e63032b0e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 8 type: EXT [1064960->1851391] (req 1064960->1073151) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e31 expref: 24 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.057251:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63032af9 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 8 type: EXT [1855488->3096575] (req 1855488->1859583) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e1c expref: 14 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.057262:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e63032b0e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 8 type: EXT [1064960->1851391] (req 1064960->1073151) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e31 expref: 24 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.057286:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63032af9 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [1855488->3096575] (req 1855488->1859583) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e1c expref: 14 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.057322:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e63032b0e lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [1064960->1851391] (req 1064960->1073151) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e31 expref: 24 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.057335:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63032af9 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [1855488->3096575] (req 1855488->1859583) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e1c expref: 14 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.057336:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63032b31 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [1847296->1859583] (req 1847296->1859583) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e54 expref: 14 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.057337:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(85 177) 00010000:00010000:3.0:1780298642.057360:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(85 177) 00010000:00010000:3.0:1780298642.057385:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63032b31 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1847296->3096575] (req 1847296->1859583) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e54 expref: 13 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.057395:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63032b31 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1847296->3096575] (req 1847296->1859583) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e54 expref: 13 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.057420:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298642.057422:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9ac00, rc 0) 00010000:00010000:3.0:1780298642.057441:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63032b31 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1847296->3096575] (req 1847296->1859583) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e54 expref: 13 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.057457:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304c000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777604850304:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298642.057652:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:29360:x1866777609333760:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 2021us (2132us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.057654:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609333888:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 432us (590us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298642.057656:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348ce1840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.057673:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8700 00000100:00100000:2.0:1780298642.057683:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14942 00000100:00100000:3.0:1780298642.058618:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304c000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777604850304:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298642.058634:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298642.058700:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.058714:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548bb480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+23:2399:x1866777609334016:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1494us (1692us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298642.058721:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14943 00000100:00100000:1.0:1780298642.058750:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609334144 00000100:00100000:1.0:1780298642.058757:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609334144 00000100:00100000:1.0:1780298642.058808:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5912 00000100:00100000:1.0:1780298642.058814:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548bad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609334144:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298642.058997:0:19951:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 3106995 -> 3106995 00002000:00010000:1.0:1780298642.059001:0:19951:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:1.0:1780298642.059004:0:19951:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:1.0:1780298642.059006:0:19951:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:1.0:1780298642.059008:0:19951:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 352 -> 352 00000100:00100000:1.0:1780298642.059046:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548bad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609334144:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 231us (348us total) trans 4294969599 rc 0/0 00000100:00100000:1.0:1780298642.059053:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5912 00000100:00100000:0.0:1780298642.060050:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.060098:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609334272 00000100:00100000:2.0:1780298642.060105:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609334272 00000100:00100000:2.0:1780298642.060173:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8701 00000100:00100000:3.0:1780298642.060180:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.060186:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609334272:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.060198:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298642.060200:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609334400 00000100:00100000:0.0:1780298642.060204:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609334400 00010000:00010000:2.0:1780298642.060211:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63032b38 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1e5b expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.060231:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8702 00000100:00100000:0.0:1780298642.060239:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609334400:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.060246:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298642.060256:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24380ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604850432:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.060256:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63032b3f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1e62 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.060306:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548ba680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604850560:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.060329:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.060341:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.061118:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548ba680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604850560:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.061119:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24380ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604850432:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.061169:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63032b3f lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e62 expref: 17 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.061172:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63032b38 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e5b expref: 14 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.061200:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63032b38 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e5b expref: 14 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.061203:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63032b3f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e62 expref: 17 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.061216:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9aa00, rc 0) 00010000:00010000:1.0:1780298642.061221:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae400, rc 0) 00000100:00100000:2.0:1780298642.061245:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609334272:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1063us (1197us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.061254:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8701 00000100:00100000:1.0:1780298642.061261:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609334400:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1019us (1079us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.061267:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8702 00000100:00100000:3.0:1780298642.062469:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.062512:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609334528 00000100:00100000:1.0:1780298642.062518:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609334528 00000100:00100000:3.0:1780298642.062530:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.062558:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609334656 00000100:00100000:1.0:1780298642.062563:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609334656 00000100:00100000:1.0:1780298642.062594:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8703 00000100:00100000:0.0:1780298642.062603:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8704 00000100:00100000:1.0:1780298642.062609:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609334528:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298642.062620:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2399:x1866777609334656:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.062621:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.062631:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.062634:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f68a0cb8/0xaf35132e63032b46 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1e69 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.062641:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e63032b4d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1e70 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.062675:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b6aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604850688:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.062679:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548ba300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604850816:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.062733:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12a80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.062734:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.063521:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b6aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604850688:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.063540:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f68a0cb8/0xaf35132e63032b46 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e69 expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.063545:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548ba300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604850816:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.063557:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f68a0cb8/0xaf35132e63032b46 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e69 expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.063566:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37800, rc 0) 00010000:00010000:0.0:1780298642.063569:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e63032b4d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e70 expref: 17 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.063585:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e63032b4d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e70 expref: 17 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.063596:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609334528:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 993us (1200us total) trans 0 rc 301/301 00010000:00010000:0.0:1780298642.063607:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baec00, rc 0) 00000100:00100000:1.0:1780298642.063608:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8703 00000100:00100000:0.0:1780298642.063634:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2399:x1866777609334656:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1021us (1105us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.063641:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8704 00000100:00100000:2.0:1780298642.064663:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.064700:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609334784 00000100:00100000:1.0:1780298642.064705:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609334784 00000100:00100000:1.0:1780298642.064738:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8705 00000100:00100000:1.0:1780298642.064746:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24380ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:29360:x1866777609334784:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.064754:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.064764:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63032b54 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1e77 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.064782:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(85 177) 00010000:00010000:1.0:1780298642.064787:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(85 177) 00010000:00010000:1.0:1780298642.064797:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63032b54 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [159744->1847295] (req 159744->167935) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e77 expref: 14 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.064811:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63032b54 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [159744->1847295] (req 159744->167935) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e77 expref: 14 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.064820:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36a00, rc 0) 00000100:00100000:1.0:1780298642.064846:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24380ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:29360:x1866777609334784:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 102us (184us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.064852:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8705 00000100:00100000:3.0:1780298642.065544:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.065575:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609334912 00000100:00100000:2.0:1780298642.065580:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609334912 00000100:00100000:2.0:1780298642.065621:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5913 00000100:00100000:2.0:1780298642.065630:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609334912:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298642.065753:0:19928:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 3106995 -> 3106995 00002000:00010000:2.0:1780298642.065756:0:19928:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.065758:0:19928:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.065759:0:19928:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.065761:0:19928:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 352 -> 352 00000100:00100000:2.0:1780298642.065784:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2400:x1866777609334912:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 158us (241us total) trans 4294969600 rc 0/0 00000100:00100000:2.0:1780298642.065789:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5913 00000100:00100000:3.0:1780298642.066753:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.066785:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609335040 00000100:00100000:1.0:1780298642.066789:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609335040 00000100:00100000:1.0:1780298642.066813:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8706 00000100:00100000:1.0:1780298642.066819:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+23:2399:x1866777609335040:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.066826:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.066833:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003c8dab73/0xaf35132e63032b5b lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1e7e expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.066863:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b6aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604850944:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.066903:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.067578:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b6aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604850944:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.067599:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003c8dab73/0xaf35132e63032b5b lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e7e expref: 24 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.067612:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003c8dab73/0xaf35132e63032b5b lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e7e expref: 24 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.067621:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36800, rc 0) 00000100:00100000:1.0:1780298642.067650:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+23:2399:x1866777609335040:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 831us (899us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.067657:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8706 00000100:00100000:3.0:1780298642.068438:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.068479:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609335168 00000100:00100000:0.0:1780298642.068485:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609335168 00000100:00100000:0.0:1780298642.068508:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8707 00000100:00100000:0.0:1780298642.068513:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+23:2399:x1866777609335168:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.068520:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.068527:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63032b62 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1e85 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.068550:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548ba300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604851072:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.068581:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.069218:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548ba300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604851072:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.069230:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63032b62 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e85 expref: 24 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.069239:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63032b62 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e85 expref: 24 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.069245:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baee00, rc 0) 00000100:00100000:0.0:1780298642.069267:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+23:2399:x1866777609335168:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 754us (830us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.069271:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8707 00000100:00100000:2.0:1780298642.070948:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.070982:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609335424 00000100:00100000:0.0:1780298642.070987:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609335424 00000100:00080000:1.0:1780298642.071021:0:6473:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1780298642.071028:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61909 00000100:00100000:0.0:1780298642.071039:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24380b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+87:29360:x1866777609335424:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:0.0:1780298642.071082:0:6477:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298642.071102:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafa00/0xaf35132e63032a35 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 48 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d5f expref: 87 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.071118:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298642.071121:0:6477:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1780298642.071279:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafe00/0xaf35132e63032b69 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 47 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.071290:0:6477:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b800/0xaf35132e63032a97 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 47 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1dba expref: 81 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.071313:0:6477:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafe00/0xaf35132e63032b69 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 47 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.071333:0:6477:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b800/0xaf35132e63032a97 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 47 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1dba expref: 81 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.071348:0:6477:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b800/0xaf35132e63032a97 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 47 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1dba expref: 81 pid: 7411 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.071358:0:6477:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548ba300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6477:1866777604851200:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298642.071398:0:6477:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34633db40 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.072055:0:6477:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548ba300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6477:1866777604851200:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298642.072070:0:6477:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafe00/0xaf35132e63032b69 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 47 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.072248:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.072284:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609335552 00000100:00100000:0.0:1780298642.072288:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609335552 00010000:00010000:0.0:1780298642.072317:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b800/0xaf35132e63032a97 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 47 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1dba expref: 82 pid: 7411 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.072341:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14944 00000100:00100000:0.0:1780298642.072349:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b1dc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+82:2399:x1866777609335552:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298642.072358:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298642.072362:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b800/0xaf35132e63032a97 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 48 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1dba expref: 82 pid: 7411 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.072373:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b800/0xaf35132e63032a97 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 48 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1dba expref: 82 pid: 7411 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.072384:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b800/0xaf35132e63032a97 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 48 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1dba expref: 82 pid: 7411 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.072393:0:6461:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298642.072397:0:6461:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafe00/0xaf35132e63032b69 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 47 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.072405:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafe00/0xaf35132e63032b69 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 47 type: IBT flags: 0x40290000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.072412:0:6461:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafe00/0xaf35132e63032b69 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 47 type: IBT flags: 0x40290000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.072429:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298642.072455:0:6477:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafe00/0xaf35132e63032b69 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 46 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.072457:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b1dc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+81:2399:x1866777609335552:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 110us (210us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.072471:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14944 00010000:00010000:2.0:1780298642.072483:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafe00/0xaf35132e63032b69 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 46 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1780298642.072554:0:6477:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294990994 00000004:00010000:2.0:1780298642.072592:0:6477:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 000000000fac3a31 reply state 00000000bc3b18ee transno 4294990995 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafe00/0xaf35132e63032b69 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 46 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.072606:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafe00/0xaf35132e63032b69 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 46 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.072614:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.072617:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1780298642.072620:0:6477:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1780298642.072681:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24380b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+89:29360:x1866777609335424:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1643us (1733us total) trans 4294990995 rc 0/0 00000100:00100000:2.0:1780298642.072688:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61909 00000020:00080000:2.0:1780298642.072693:0:6477:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1780298642.073493:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.073529:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609335680 00000100:00100000:0.0:1780298642.073533:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609335680 00000100:00100000:0.0:1780298642.073562:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8708 00000100:00100000:0.0:1780298642.073570:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24380a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:29360:x1866777609335680:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.073581:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.073592:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e63032b70 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1e8c expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.073612:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63032b54 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [159744->1847295] (req 159744->167935) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e77 expref: 16 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.073621:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63032b31 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1847296->3096575] (req 1847296->1859583) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e54 expref: 16 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.073629:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000991eb57d/0xaf35132e63032aa5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3096576->36651007] (req 3096576->3108863) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1dc8 expref: 22 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.073637:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 3 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(85 177) 00010000:00010000:0.0:1780298642.073641:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(88 177) 00010000:00010000:0.0:1780298642.073647:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e63032b70 lrc: 6/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e8c expref: 16 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.073667:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000991eb57d/0xaf35132e63032aa5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3096576->36651007] (req 3096576->3108863) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1dc8 expref: 22 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.073679:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000991eb57d/0xaf35132e63032aa5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3096576->36651007] (req 3096576->3108863) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1dc8 expref: 22 pid: 20252 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.073690:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548b9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604851328:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298642.073741:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63032b31 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1847296->3096575] (req 1847296->1859583) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e54 expref: 16 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.073751:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63032b31 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1847296->3096575] (req 1847296->1859583) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e54 expref: 16 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.073761:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548bb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604851456:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298642.073791:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63032b54 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [159744->1847295] (req 159744->167935) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e77 expref: 16 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.073802:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63032b54 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [159744->1847295] (req 159744->167935) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e77 expref: 16 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.073812:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548ba300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604851584:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298642.073838:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.074695:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548b9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604851328:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298642.074706:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548bb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604851456:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298642.074888:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.074902:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548ba300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604851584:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298642.074928:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e63032b70 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e8c expref: 16 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.074941:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae000, rc 0) 00000100:00100000:1.0:1780298642.074947:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609335936 00000100:00100000:1.0:1780298642.074951:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609335936 00010000:00010000:1.0:1780298642.074979:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63032b31 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1847296->3096575] (req 1847296->1859583) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e54 expref: 17 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.074979:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24380a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:29360:x1866777609335680:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1405us (1482us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.074996:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8708 00000100:00100000:1.0:1780298642.075018:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14945 00000100:00100000:1.0:1780298642.075038:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2399:x1866777609335936:12345-192.168.203.25@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1780298642.075046:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:1.0:1780298642.075048:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298642.075065:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63032b31 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [1847296->3096575] (req 1847296->1859583) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e54 expref: 16 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.075082:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63032b31 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [1847296->3096575] (req 1847296->1859583) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e54 expref: 16 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.075086:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609335808 00000100:00100000:2.0:1780298642.075090:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609335808 00010000:00010000:1.0:1780298642.075098:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63032b31 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [1847296->3096575] (req 1847296->1859583) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e54 expref: 16 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.075107:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000991eb57d/0xaf35132e63032aa5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [3096576->36651007] (req 3096576->3108863) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1dc8 expref: 23 pid: 20252 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.075110:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298642.075160:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14946 00000100:00100000:2.0:1780298642.075169:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+23:2399:x1866777609335808:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298642.075178:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1780298642.075178:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2399:x1866777609335936:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 153us (292us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298642.075184:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:2.0:1780298642.075189:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000991eb57d/0xaf35132e63032aa5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3096576->36651007] (req 3096576->3108863) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1dc8 expref: 23 pid: 20252 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.075190:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14945 00010000:00010000:2.0:1780298642.075201:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000991eb57d/0xaf35132e63032aa5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3096576->36651007] (req 3096576->3108863) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1dc8 expref: 23 pid: 20252 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.075216:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000991eb57d/0xaf35132e63032aa5 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3096576->36651007] (req 3096576->3108863) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1dc8 expref: 23 pid: 20252 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.075221:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609336064 00000100:00100000:1.0:1780298642.075225:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609336064 00010000:00010000:2.0:1780298642.075229:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298642.075238:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63032b54 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [159744->1847295] (req 159744->167935) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e77 expref: 15 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.075254:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2399:x1866777609335808:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 88us (255us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.075262:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14946 00000100:00100000:1.0:1780298642.075272:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14947 00000100:00100000:1.0:1780298642.075278:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2398:x1866777609336064:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298642.075286:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298642.075306:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63032b54 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [159744->1847295] (req 159744->167935) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e77 expref: 15 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.075316:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63032b54 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [159744->1847295] (req 159744->167935) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e77 expref: 15 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.075326:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63032b54 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [159744->1847295] (req 159744->167935) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e77 expref: 15 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.075333:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(88 177) 00010000:00010000:1.0:1780298642.075335:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(88 177) 00010000:00010000:1.0:1780298642.075340:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e63032b70 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e8c expref: 14 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.075347:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e63032b70 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e8c expref: 14 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.075365:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e63032b70 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e8c expref: 14 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.075372:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b6aa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777604851712:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298642.075401:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.076046:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b6aa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777604851712:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298642.076058:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298642.076105:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2398:x1866777609336064:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 826us (925us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298642.076112:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.076120:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14947 00000100:00100000:2.0:1780298642.076173:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609336192 00000100:00100000:2.0:1780298642.076177:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609336192 00000100:00100000:2.0:1780298642.076212:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8709 00000100:00100000:2.0:1780298642.076221:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:29360:x1866777609336192:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.076230:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.076241:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e63032b77 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1e93 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.076261:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e630329a9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d04 expref: 18 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.076266:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298641 v=(65 130) 00010000:00010000:2.0:1780298642.076269:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(66 130) 00010000:00010000:2.0:1780298642.076271:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e63032b77 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e93 expref: 17 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.076285:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e630329a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d04 expref: 18 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.076304:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e630329a9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d04 expref: 18 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.076311:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24380b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604851840:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298642.076339:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4900 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.076891:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24380b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604851840:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298642.076905:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e63032b77 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e93 expref: 17 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.076913:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b000, rc 0) 00000100:00100000:2.0:1780298642.076935:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:29360:x1866777609336192:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 717us (827us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.076946:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8709 00000100:00100000:3.0:1780298642.077154:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.077180:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609336320 00000100:00100000:0.0:1780298642.077184:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609336320 00010000:00010000:0.0:1780298642.077213:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e630329a9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d04 expref: 19 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1780298642.077219:0:6473:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1780298642.077233:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14948 00000100:00100000:0.0:1780298642.077238:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304dc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+19:2400:x1866777609336320:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298642.077245:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298642.077254:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e630329a9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d04 expref: 19 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.077263:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e630329a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d04 expref: 19 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.077278:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e630329a9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1d04 expref: 19 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.077290:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(66 130) 00010000:00010000:0.0:1780298642.077304:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(66 130) 00010000:00010000:0.0:1780298642.077310:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e63032b77 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e93 expref: 16 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.077318:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e63032b77 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e93 expref: 16 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.077340:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e63032b77 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e93 expref: 16 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.077357:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548ba300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777604851968:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298642.077398:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.078005:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548ba300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777604851968:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298642.078016:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298642.078056:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304dc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2400:x1866777609336320:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 816us (905us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.078064:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14948 00000100:00100000:3.0:1780298642.078120:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.078185:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609336448 00000100:00100000:0.0:1780298642.078190:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609336448 00002000:00010000:0.0:1780298642.078211:0:8363:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:0.0:1780298642.078215:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24304e300 with x1866777609336448 ext(4194304->18446744073709551615) 00002000:00010000:0.0:1780298642.078223:0:8363:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24304e300. 00000100:00100000:0.0:1780298642.078248:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5914 00000100:00100000:0.0:1780298642.078256:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609336448:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:0.0:1780298642.078396:0:8363:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 3106995 -> 4194304 00002000:00010000:0.0:1780298642.078399:0:8363:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298642.078401:0:8363:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298642.078403:0:8363:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298642.078405:0:8363:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 352 -> 352 00000100:00100000:0.0:1780298642.078438:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2400:x1866777609336448:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 182us (318us total) trans 4294969601 rc 0/0 00000100:00100000:0.0:1780298642.078445:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5914 00002000:00010000:0.0:1780298642.078448:0:8363:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:0.0:1780298642.078451:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24304e300 with x1866777609336448 ext(4194304->18446744073709551615) 00002000:00010000:0.0:1780298642.078456:0:8363:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24304e300. 00000100:00100000:1.0:1780298642.078573:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.078610:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609336576 00000100:00100000:0.0:1780298642.078614:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609336576 00002000:00010000:0.0:1780298642.078629:0:19943:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (3275867->18446744073709551615) 00002000:00100000:0.0:1780298642.078633:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b6aa00 with x1866777609336576 ext(3275867->18446744073709551615) 00002000:00010000:0.0:1780298642.078639:0:19943:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf242b6aa00. 00000100:00100000:0.0:1780298642.078661:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5915 00000100:00100000:0.0:1780298642.078671:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b6aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609336576:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:0.0:1780298642.078789:0:19943:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 0 -> 3275867 00002000:00010000:0.0:1780298642.078792:0:19943:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298642.078794:0:19943:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298642.078797:0:19943:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298642.078799:0:19943:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:0.0:1780298642.078830:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b6aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609336576:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 161us (257us total) trans 4294969167 rc 0/0 00000100:00100000:0.0:1780298642.078838:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5915 00002000:00010000:0.0:1780298642.078841:0:19943:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (3275867->18446744073709551615) 00002000:00100000:0.0:1780298642.078844:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b6aa00 with x1866777609336576 ext(3275867->18446744073709551615) 00002000:00010000:0.0:1780298642.078850:0:19943:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf242b6aa00. 00000100:00100000:2.0:1780298642.079631:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.079664:0:7411:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609336704 00000100:00100000:1.0:1780298642.079668:0:7411:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609336704 00000100:00100000:1.0:1780298642.079696:0:7411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61910 00000100:00100000:1.0:1780298642.079703:0:7411:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24380b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+81:29360:x1866777609336704:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.079712:0:7411:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.079724:0:7411:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36800/0xaf35132e63032b7e lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 47 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1e9a expref: -99 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.079783:0:7411:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37800/0xaf35132e63032b85 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 48 type: IBT flags: 0x40000000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.079797:0:7411:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37800/0xaf35132e63032b85 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 48 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.079804:0:7411:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37800/0xaf35132e63032b85 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 48 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1780298642.079820:0:7411:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 7470171 00000004:00200000:1.0:1780298642.079824:0:7411:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1780298642.079835:0:7411:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37800/0xaf35132e63032b85 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 48 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.079850:0:7411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36800/0xaf35132e63032b7e lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 48 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e9a expref: 83 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.079858:0:7411:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37800/0xaf35132e63032b85 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 47 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e9a expref: 82 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.079863:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298642.079865:0:7411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1780298642.079867:0:7411:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37800, rc 0) 00000100:00100000:1.0:1780298642.079890:0:7411:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24380b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+82:29360:x1866777609336704:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 189us (260us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.079895:0:7411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61910 00000100:00100000:2.0:1780298642.080749:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.080785:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609336832 00000100:00100000:1.0:1780298642.080788:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609336832 00000100:00100000:1.0:1780298642.080816:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8710 00000100:00100000:1.0:1780298642.080823:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24380bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2398:x1866777609336832:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.080830:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.080840:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e63032b8c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1ea1 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.080844:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.080876:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b69880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604852096:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.080887:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609336960 00000100:00100000:3.0:1780298642.080900:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609336960 00000100:00100000:1.0:1780298642.080922:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12a80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.080957:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8711 00000100:00100000:3.0:1780298642.080981:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243808000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2398:x1866777609336960:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.080999:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.081054:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63032b93 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1ea8 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.081108:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604852224:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.081226:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348ce1840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.081551:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b69880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604852096:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.081570:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e63032b8c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ea1 expref: 23 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.081586:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e63032b8c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ea1 expref: 23 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.081596:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37200, rc 0) 00000100:00100000:1.0:1780298642.081629:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24380bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2398:x1866777609336832:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 807us (880us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.081636:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8710 00000100:00100000:3.0:1780298642.081895:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604852224:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.081918:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63032b93 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ea8 expref: 19 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.081937:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63032b93 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ea8 expref: 19 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.081948:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2800, rc 0) 00000100:00100000:3.0:1780298642.081983:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243808000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2398:x1866777609336960:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1007us (1142us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.081992:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8711 00000100:00100000:0.0:1780298642.082808:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.082934:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609337088 00000100:00100000:3.0:1780298642.082940:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609337088 00000100:00100000:2.0:1780298642.082969:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.083000:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609337216 00000100:00100000:1.0:1780298642.083006:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609337216 00000100:00100000:0.0:1780298642.083020:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8712 00000100:00100000:3.0:1780298642.083043:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8713 00000100:00100000:0.0:1780298642.083044:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2400:x1866777609337088:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.083058:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298642.083065:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243809880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2399:x1866777609337216:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.083068:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007b6539a7/0xaf35132e63032b9a lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1eaf expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.083076:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.083089:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ced6368f/0xaf35132e63032ba1 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1eb6 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.083100:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548bb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604852352:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.083183:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.083190:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604852480:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.083232:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348ce1840 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.083741:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548bb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604852352:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.083759:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007b6539a7/0xaf35132e63032b9a lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1eaf expref: 23 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.083771:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007b6539a7/0xaf35132e63032b9a lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1eaf expref: 23 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.083779:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae800, rc 0) 00000100:00100000:0.0:1780298642.083808:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2400:x1866777609337088:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 781us (1000us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.083816:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8712 00000100:00100000:0.0:1780298642.084002:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604852480:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.084018:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ced6368f/0xaf35132e63032ba1 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1eb6 expref: 19 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.084030:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ced6368f/0xaf35132e63032ba1 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1eb6 expref: 19 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.084040:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2a00, rc 0) 00000100:00100000:0.0:1780298642.084062:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243809880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2399:x1866777609337216:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1007us (1096us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.084068:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8713 00000100:00100000:2.0:1780298642.084874:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.084922:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609337344 00000100:00100000:2.0:1780298642.084928:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.084929:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609337344 00000100:00100000:0.0:1780298642.084957:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609337472 00000100:00100000:0.0:1780298642.084959:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609337472 00000100:00100000:1.0:1780298642.084969:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8714 00000100:00100000:1.0:1780298642.084983:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243809180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2399:x1866777609337344:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.084995:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298642.084995:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8715 00000100:00100000:0.0:1780298642.085008:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2400:x1866777609337472:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.085009:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63032ba8 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1ebd expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.085017:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.085027:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2d588e1/0xaf35132e63032baf lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1ec4 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.085047:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b69880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604852608:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.085064:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548bb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604852736:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.085104:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12a80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.085105:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.085879:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b69880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604852608:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.085898:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63032ba8 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ebd expref: 23 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.085907:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548bb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604852736:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.085919:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63032ba8 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ebd expref: 23 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.085931:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36a00, rc 0) 00010000:00010000:0.0:1780298642.085942:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c2d588e1/0xaf35132e63032baf lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ec4 expref: 19 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.085957:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2d588e1/0xaf35132e63032baf lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ec4 expref: 19 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.085966:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf800, rc 0) 00000100:00100000:1.0:1780298642.085967:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243809180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2399:x1866777609337344:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 986us (1091us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.085982:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8714 00000100:00100000:0.0:1780298642.086002:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2400:x1866777609337472:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 998us (1073us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.086008:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8715 00000100:00100000:2.0:1780298642.086978:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.087009:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609337600 00000100:00100000:0.0:1780298642.087014:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609337600 00000100:00100000:0.0:1780298642.087040:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8716 00000100:00100000:0.0:1780298642.087047:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481ffb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:29360:x1866777609337600:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.087055:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.087063:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002ee949fd/0xaf35132e63032bb6 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1ecb expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.087078:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e63032b70 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e8c expref: 14 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.087085:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(88 177) 00010000:00010000:0.0:1780298642.087088:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(89 177) 00010000:00010000:0.0:1780298642.087091:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002ee949fd/0xaf35132e63032bb6 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1622016->1630207] (req 1622016->1630207) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ecb expref: 23 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.087106:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e63032b70 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e8c expref: 14 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.087117:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e63032b70 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e8c expref: 14 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.087156:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548bb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604852864:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298642.087198:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.087814:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548bb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604852864:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298642.087824:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002ee949fd/0xaf35132e63032bb6 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1622016->1630207] (req 1622016->1630207) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ecb expref: 23 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.087832:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bafc00, rc 0) 00000100:00100000:0.0:1780298642.087852:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481ffb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+23:29360:x1866777609337600:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 808us (878us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.087857:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8716 00000100:00100000:3.0:1780298642.088094:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.088145:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609337728 00000100:00100000:0.0:1780298642.088150:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609337728 00010000:00010000:0.0:1780298642.088169:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e63032b70 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e8c expref: 15 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.088198:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14949 00000100:00100000:0.0:1780298642.088205:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2399:x1866777609337728:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298642.088214:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298642.088227:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e63032b70 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e8c expref: 15 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.088243:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e63032b70 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e8c expref: 15 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.088255:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e63032b70 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e8c expref: 15 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.088261:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(89 177) 00010000:00010000:0.0:1780298642.088264:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(89 177) 00010000:00010000:0.0:1780298642.088269:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000002ee949fd/0xaf35132e63032bb6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1622016->35176447] (req 1622016->1630207) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ecb expref: 22 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.088274:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002ee949fd/0xaf35132e63032bb6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1622016->35176447] (req 1622016->1630207) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ecb expref: 22 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.088300:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000002ee949fd/0xaf35132e63032bb6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1622016->35176447] (req 1622016->1630207) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ecb expref: 22 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.088312:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548bb480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777604852992:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298642.088346:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.089022:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548bb480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777604852992:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298642.089032:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298642.089059:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609337728:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 854us (965us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.089067:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14949 00000100:00100000:0.0:1780298642.089233:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.089261:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609337856 00000100:00100000:2.0:1780298642.089265:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609337856 00000100:00100000:2.0:1780298642.089307:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5916 00000100:00100000:2.0:1780298642.089313:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548bb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+23:2399:x1866777609337856:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298642.089487:0:8362:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:2.0:1780298642.089491:0:8362:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.089494:0:8362:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.089496:0:8362:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.089499:0:8362:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 352 -> 368 00000100:00100000:2.0:1780298642.089530:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548bb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+24:2399:x1866777609337856:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 217us (296us total) trans 4294969602 rc 0/0 00000100:00100000:2.0:1780298642.089535:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5916 00000100:00100000:2.0:1780298642.090867:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.090897:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.090918:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609337984 00000100:00100000:0.0:1780298642.090924:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609337984 00000100:00100000:3.0:1780298642.090928:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609338112 00000100:00100000:3.0:1780298642.090933:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609338112 00000100:00100000:0.0:1780298642.090954:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8717 00000100:00100000:0.0:1780298642.090963:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481ff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+24:2398:x1866777609337984:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298642.090966:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8718 00000100:00100000:3.0:1780298642.090978:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2398:x1866777609338112:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.090978:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.090986:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.090991:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e63032bbd lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1ed2 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.090998:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e63032bc4 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1ed9 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.091036:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548b9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604853120:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.091038:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604853248:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.091099:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348ce1840 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.091099:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.091950:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548b9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604853120:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.091981:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e63032bbd lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ed2 expref: 25 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.091999:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e63032bbd lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ed2 expref: 25 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.092013:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf600, rc 0) 00000100:00100000:1.0:1780298642.092020:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604853248:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.092061:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e63032bc4 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ed9 expref: 19 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.092062:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481ff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+24:2398:x1866777609337984:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1096us (1189us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.092085:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8717 00010000:00010000:1.0:1780298642.092113:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e63032bc4 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ed9 expref: 19 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.092153:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2600, rc 0) 00000100:00100000:1.0:1780298642.092198:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2398:x1866777609338112:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1224us (1300us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.092204:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8718 00000100:00100000:3.0:1780298642.093034:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.093071:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609338240 00000100:00100000:3.0:1780298642.093076:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.093076:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609338240 00000100:00100000:0.0:1780298642.093112:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609338368 00000100:00100000:0.0:1780298642.093114:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609338368 00000100:00100000:0.0:1780298642.093149:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8719 00000100:00100000:0.0:1780298642.093159:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+24:2398:x1866777609338240:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298642.093160:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8720 00010000:00010000:0.0:1780298642.093168:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298642.093176:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2397:x1866777609338368:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.093177:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63032bcb lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1ee0 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.093197:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.093207:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7fd0135/0xaf35132e63032bd2 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1ee7 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.093221:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548b9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604853376:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.093242:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481ff800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604853504:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.093271:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.093276:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4900 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.093897:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481ff800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604853504:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.093897:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548b9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604853376:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.093921:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63032bcb lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ee0 expref: 25 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.093927:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e7fd0135/0xaf35132e63032bd2 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ee7 expref: 19 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.093938:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63032bcb lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ee0 expref: 25 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.093946:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bafa00, rc 0) 00010000:00010000:3.0:1780298642.093948:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7fd0135/0xaf35132e63032bd2 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ee7 expref: 19 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.093957:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9be00, rc 0) 00000100:00100000:2.0:1780298642.093978:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+24:2398:x1866777609338240:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 821us (946us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.093983:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8719 00000100:00100000:3.0:1780298642.093996:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2397:x1866777609338368:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 824us (918us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.094005:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8720 00000100:00100000:3.0:1780298642.095023:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.095062:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609338496 00000100:00100000:1.0:1780298642.095069:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609338496 00000100:00100000:3.0:1780298642.095074:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.095110:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609338624 00000100:00100000:1.0:1780298642.095115:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609338624 00000100:00100000:0.0:1780298642.095117:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8721 00000100:00100000:0.0:1780298642.095153:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+24:2400:x1866777609338496:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.095163:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.095171:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007b6539a7/0xaf35132e63032bd9 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1eee expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.095180:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8722 00000100:00100000:0.0:1780298642.095196:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548b9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604853632:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.095216:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2399:x1866777609338624:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.095223:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.095231:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e63032be0 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1ef5 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.095238:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417d80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.095254:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604853760:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.095275:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.095946:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548b9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604853632:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.095961:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007b6539a7/0xaf35132e63032bd9 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1eee expref: 25 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.095974:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007b6539a7/0xaf35132e63032bd9 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1eee expref: 25 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.095982:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae800, rc 0) 00000100:00100000:2.0:1780298642.095983:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604853760:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.096001:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e63032be0 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ef5 expref: 19 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.096014:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+24:2400:x1866777609338496:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 886us (1029us total) trans 0 rc 301/301 00010000:00010000:2.0:1780298642.096019:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e63032be0 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ef5 expref: 19 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.096023:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8721 00010000:00010000:2.0:1780298642.096037:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9ac00, rc 0) 00000100:00100000:2.0:1780298642.096060:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2399:x1866777609338624:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 872us (988us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.096068:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8722 00000100:00100000:1.0:1780298642.097024:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.097070:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609338752 00000100:00100000:0.0:1780298642.097075:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609338752 00000100:00100000:1.0:1780298642.097080:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.097104:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609338880 00000100:00100000:0.0:1780298642.097107:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609338880 00000100:00100000:2.0:1780298642.097116:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8723 00000100:00100000:2.0:1780298642.097143:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b69880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+24:2400:x1866777609338752:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.097149:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.097156:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63032be7 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1efc expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.097163:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8724 00000100:00100000:0.0:1780298642.097173:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b6b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2400:x1866777609338880:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298642.097181:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604853888:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.097183:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.097194:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63032bee lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1f03 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.097222:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.097232:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548b9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604854016:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.097259:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417540 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.097837:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604853888:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.097849:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63032be7 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1efc expref: 25 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.097859:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63032be7 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1efc expref: 25 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.097866:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9aa00, rc 0) 00000100:00100000:2.0:1780298642.097887:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b69880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+24:2400:x1866777609338752:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 765us (866us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.097892:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8723 00000100:00100000:0.0:1780298642.097970:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548b9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604854016:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.097987:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63032bee lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f03 expref: 19 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.098001:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63032bee lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f03 expref: 19 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.098009:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae000, rc 0) 00000100:00100000:0.0:1780298642.098036:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b6b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2400:x1866777609338880:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 865us (956us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.098043:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8724 00000100:00100000:1.0:1780298642.098912:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.098942:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.098959:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609339008 00000100:00100000:3.0:1780298642.098964:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609339136 00000100:00100000:2.0:1780298642.098965:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609339008 00000100:00100000:3.0:1780298642.098970:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609339136 00000100:00100000:2.0:1780298642.099002:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8725 00000100:00100000:2.0:1780298642.099014:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b68a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+24:2400:x1866777609339008:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298642.099015:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8726 00010000:00010000:2.0:1780298642.099026:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298642.099031:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e60e850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2399:x1866777609339136:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.099036:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63032bf5 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1f0a expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.099040:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.099050:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63032bfc lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1f11 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.099067:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604854144:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.099081:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254888700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604854272:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.099109:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.099148:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348ce1840 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.099832:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604854144:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.099849:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63032bf5 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f0a expref: 25 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.099864:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63032bf5 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f0a expref: 25 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.099873:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9be00, rc 0) 00000100:00100000:2.0:1780298642.099906:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b68a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+24:2400:x1866777609339008:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 897us (994us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.099914:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8725 00000100:00100000:3.0:1780298642.100011:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254888700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604854272:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.100024:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63032bfc lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f11 expref: 19 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.100032:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63032bfc lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f11 expref: 19 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.100039:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2800, rc 0) 00000100:00100000:3.0:1780298642.100053:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e60e850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2399:x1866777609339136:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1029us (1112us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.100057:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8726 00000100:00100000:3.0:1780298642.101069:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.101114:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609339264 00000100:00100000:1.0:1780298642.101120:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609339264 00000100:00100000:1.0:1780298642.101190:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5917 00000100:00100000:1.0:1780298642.101200:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254888700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+24:2398:x1866777609339264:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298642.101424:0:19951:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:1.0:1780298642.101428:0:19951:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:1.0:1780298642.101430:0:19951:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:1.0:1780298642.101432:0:19951:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:1.0:1780298642.101434:0:19951:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 368 -> 376 00000100:00100000:1.0:1780298642.101468:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254888700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+25:2398:x1866777609339264:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 270us (398us total) trans 4294969603 rc 0/0 00000100:00100000:1.0:1780298642.101477:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5917 00000100:00100000:1.0:1780298642.102239:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.102271:0:6476:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609339392 00000100:00100000:2.0:1780298642.102275:0:6476:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609339392 00000100:00100000:2.0:1780298642.102315:0:6476:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61911 00000100:00100000:2.0:1780298642.102322:0:6476:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b69180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+89:29360:x1866777609339392:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.102336:0:6476:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.102348:0:6476:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9aa00/0xaf35132e63032c03 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 48 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1f18 expref: -99 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.102418:0:6476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ac00/0xaf35132e63032c0a lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 49 type: IBT flags: 0x40000000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.102429:0:6476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ac00/0xaf35132e63032c0a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 49 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.102437:0:6476:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ac00/0xaf35132e63032c0a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 49 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298642.102459:0:6476:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 7470171 00000004:00200000:2.0:1780298642.102463:0:6476:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298642.102478:0:6476:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ac00/0xaf35132e63032c0a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 49 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.102495:0:6476:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9aa00/0xaf35132e63032c03 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 49 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f18 expref: 91 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.102504:0:6476:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ac00/0xaf35132e63032c0a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 48 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f18 expref: 90 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.102512:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.102514:0:6476:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298642.102516:0:6476:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9ac00, rc 0) 00000100:00100000:2.0:1780298642.102543:0:6476:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b69180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+90:29360:x1866777609339392:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 222us (304us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.102550:0:6476:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61911 00000100:00100000:0.0:1780298642.103396:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.103434:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609339520 00000100:00100000:3.0:1780298642.103445:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609339520 00000100:00100000:3.0:1780298642.103488:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8727 00000100:00100000:3.0:1780298642.103496:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609339520:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.103510:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.103523:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e63032c11 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1f1f expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.103566:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25488aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604854400:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.103650:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348ce1840 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.104351:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25488aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604854400:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.104374:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e63032c11 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f1f expref: 15 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.104389:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e63032c11 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f1f expref: 15 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.104396:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2e00, rc 0) 00000100:00100000:2.0:1780298642.104423:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609339520:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 929us (1028us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.104428:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8727 00000100:00100000:1.0:1780298642.105325:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.105369:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609339648 00000100:00100000:0.0:1780298642.105374:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609339648 00000100:00100000:0.0:1780298642.105399:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8728 00000100:00100000:0.0:1780298642.105404:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b6a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609339648:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.105411:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.105419:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e63032c18 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1f26 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.105450:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548b9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604854528:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.105503:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417540 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.106201:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548b9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604854528:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.106215:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e63032c18 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f26 expref: 15 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.106226:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e63032c18 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f26 expref: 15 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.106238:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae600, rc 0) 00000100:00100000:1.0:1780298642.106262:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b6a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609339648:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 859us (939us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.106268:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8728 00000100:00100000:3.0:1780298642.107114:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.107182:0:9976:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609339904 00000100:00100000:2.0:1780298642.107188:0:9976:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609339904 00000100:00100000:2.0:1780298642.107233:0:9976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61912 00000100:00100000:2.0:1780298642.107241:0:9976:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254889880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+82:29360:x1866777609339904:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:2.0:1780298642.107275:0:9976:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298642.107291:0:9976:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37800/0xaf35132e63032b85 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 49 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e9a expref: 82 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.107322:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.107326:0:9976:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298642.107466:0:9976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b800/0xaf35132e63032c1f lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 48 type: IBT flags: 0x40000000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.107480:0:9976:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ac00/0xaf35132e63032c0a lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 48 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f18 expref: 89 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.107495:0:9976:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b800/0xaf35132e63032c1f lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 48 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.107517:0:9976:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ac00/0xaf35132e63032c0a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 48 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f18 expref: 89 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.107527:0:9976:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ac00/0xaf35132e63032c0a lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 48 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f18 expref: 89 pid: 6476 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.107535:0:9976:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fc380 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:9976:1866777604854656:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298642.107577:0:9976:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.108195:0:9976:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fc380 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:9976:1866777604854656:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298642.108210:0:9976:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b800/0xaf35132e63032c1f lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 48 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.108319:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.108363:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609340032 00000100:00100000:1.0:1780298642.108368:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609340032 00010000:00010000:1.0:1780298642.108386:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ac00/0xaf35132e63032c0a lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 48 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f18 expref: 90 pid: 6476 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.108416:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14950 00000100:00100000:1.0:1780298642.108426:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25488ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+90:2400:x1866777609340032:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298642.108437:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298642.108442:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ac00/0xaf35132e63032c0a lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 49 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f18 expref: 90 pid: 6476 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.108454:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ac00/0xaf35132e63032c0a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 49 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f18 expref: 90 pid: 6476 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.108465:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ac00/0xaf35132e63032c0a lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 49 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f18 expref: 90 pid: 6476 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.108474:0:7400:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298642.108477:0:7400:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b800/0xaf35132e63032c1f lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 48 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.108485:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b800/0xaf35132e63032c1f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 48 type: IBT flags: 0x40290000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.108490:0:7400:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b800/0xaf35132e63032c1f lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 48 type: IBT flags: 0x40290000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.108505:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298642.108529:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25488ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+89:2400:x1866777609340032:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 106us (213us total) trans 0 rc 0/0 00010000:00010000:2.0:1780298642.108532:0:9976:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b800/0xaf35132e63032c1f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 47 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.108536:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14950 00010000:00010000:2.0:1780298642.108545:0:9976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b800/0xaf35132e63032c1f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 47 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1780298642.108595:0:9976:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294990996 00000004:00010000:2.0:1780298642.108621:0:9976:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000d26e461f reply state 00000000f4d2a2ea transno 4294990997 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b800/0xaf35132e63032c1f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 47 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.108633:0:9976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b800/0xaf35132e63032c1f lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 47 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.108640:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.108643:0:9976:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1780298642.108645:0:9976:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1780298642.108710:0:9976:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254889880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+84:29360:x1866777609339904:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1470us (1601us total) trans 4294990997 rc 0/0 00000100:00100000:2.0:1780298642.108717:0:9976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61912 00000020:00080000:2.0:1780298642.108722:0:9976:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1780298642.109377:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.109416:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609340160 00000100:00100000:1.0:1780298642.109422:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609340160 00000100:00100000:1.0:1780298642.109458:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8729 00000100:00100000:1.0:1780298642.109467:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254889c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+25:29360:x1866777609340160:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.109477:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.109487:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e63032c26 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1f2d expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.109508:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000002ee949fd/0xaf35132e63032bb6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1622016->35176447] (req 1622016->1630207) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ecb expref: 26 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.109516:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(89 177) 00010000:00010000:1.0:1780298642.109521:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(90 177) 00010000:00010000:1.0:1780298642.109526:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e63032c26 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f2d expref: 26 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.109547:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000002ee949fd/0xaf35132e63032bb6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1622016->35176447] (req 1622016->1630207) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ecb expref: 26 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.109563:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000002ee949fd/0xaf35132e63032bb6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1622016->35176447] (req 1622016->1630207) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ecb expref: 26 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.109575:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b69880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604854784:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298642.109622:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.110307:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b69880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604854784:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298642.110327:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e63032c26 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f2d expref: 26 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.110342:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37000, rc 0) 00000100:00100000:1.0:1780298642.110384:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254889c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+26:29360:x1866777609340160:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 919us (1007us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.110392:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8729 00000100:00100000:3.0:1780298642.110561:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.110614:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609340288 00000100:00100000:1.0:1780298642.110618:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609340288 00010000:00010000:1.0:1780298642.110634:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000002ee949fd/0xaf35132e63032bb6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1622016->35176447] (req 1622016->1630207) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ecb expref: 26 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.110655:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14951 00000100:00100000:1.0:1780298642.110660:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b1d450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+26:2400:x1866777609340288:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298642.110667:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298642.110680:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000002ee949fd/0xaf35132e63032bb6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1622016->35176447] (req 1622016->1630207) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ecb expref: 26 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.110693:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000002ee949fd/0xaf35132e63032bb6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1622016->35176447] (req 1622016->1630207) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ecb expref: 26 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.110704:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002ee949fd/0xaf35132e63032bb6 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1622016->35176447] (req 1622016->1630207) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ecb expref: 26 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.110713:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(90 177) 00010000:00010000:1.0:1780298642.110715:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(90 177) 00010000:00010000:1.0:1780298642.110719:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e63032c26 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f2d expref: 25 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.110725:0:6463:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e63032c26 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f2d expref: 25 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.110739:0:6463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e63032c26 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f2d expref: 25 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.110745:0:6463:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b69880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6463:1866777604854912:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298642.110776:0:6463:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.111460:0:6463:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b69880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6463:1866777604854912:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298642.111469:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298642.111499:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b1d450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+25:2400:x1866777609340288:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 838us (941us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.111507:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14951 00000100:00100000:3.0:1780298642.111687:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.111720:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609340416 00000100:00100000:0.0:1780298642.111727:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609340416 00000100:00100000:0.0:1780298642.111770:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8730 00000100:00100000:0.0:1780298642.111777:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254889180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:29360:x1866777609340416:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.111790:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.111802:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007b6539a7/0xaf35132e63032c2d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1f34 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.111822:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e63032b77 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e93 expref: 17 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.111828:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(66 130) 00010000:00010000:0.0:1780298642.111831:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(67 130) 00010000:00010000:0.0:1780298642.111840:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007b6539a7/0xaf35132e63032c2d lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f34 expref: 19 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.111864:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e63032b77 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e93 expref: 17 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.111874:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e63032b77 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e93 expref: 17 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.111883:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548b8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604855040:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298642.111937:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484173c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.112599:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548b8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604855040:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298642.112617:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007b6539a7/0xaf35132e63032c2d lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f34 expref: 19 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.112628:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae800, rc 0) 00000100:00100000:3.0:1780298642.112654:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254889180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+19:29360:x1866777609340416:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 878us (969us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298642.112659:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8730 00000100:00100000:0.0:1780298642.112742:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.112774:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609340544 00000100:00100000:1.0:1780298642.112778:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609340544 00010000:00010000:1.0:1780298642.112794:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e63032b77 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e93 expref: 18 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.112815:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14952 00000100:00100000:1.0:1780298642.112821:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b8380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2399:x1866777609340544:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298642.112830:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298642.112844:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e63032b77 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e93 expref: 18 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.112857:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e63032b77 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e93 expref: 18 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.112872:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e63032b77 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1e93 expref: 18 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.112883:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(67 130) 00010000:00010000:1.0:1780298642.112894:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(67 130) 00010000:00010000:1.0:1780298642.112900:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000007b6539a7/0xaf35132e63032c2d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f34 expref: 18 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.112911:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007b6539a7/0xaf35132e63032c2d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f34 expref: 18 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.112930:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000007b6539a7/0xaf35132e63032c2d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f34 expref: 18 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.112938:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b69880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:8374:1866777604855168:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298642.112973:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b129c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.113648:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b69880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:8374:1866777604855168:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298642.113655:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298642.113676:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b8380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2399:x1866777609340544:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 856us (935us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.113682:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14952 00000100:00100000:1.0:1780298642.113749:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.113786:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609340672 00000100:00100000:2.0:1780298642.113792:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609340672 00002000:00010000:2.0:1780298642.113813:0:19928:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:2.0:1780298642.113817:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b69880 with x1866777609340672 ext(4194304->18446744073709551615) 00002000:00010000:2.0:1780298642.113825:0:19928:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf242b69880. 00000100:00100000:2.0:1780298642.113858:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5918 00000100:00100000:2.0:1780298642.113871:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b69880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+25:2399:x1866777609340672:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:0.0:1780298642.113888:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.113911:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609340800 00000100:00100000:3.0:1780298642.113916:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609340800 00002000:00010000:3.0:1780298642.113935:0:19950:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (3301156->18446744073709551615) 00002000:00100000:3.0:1780298642.113938:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2548ba300 with x1866777609340800 ext(3301156->18446744073709551615) 00002000:00010000:3.0:1780298642.113944:0:19950:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2548ba300. 00000100:00100000:3.0:1780298642.113964:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5919 00000100:00100000:3.0:1780298642.113974:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+19:2399:x1866777609340800:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:2.0:1780298642.114000:0:19928:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:2.0:1780298642.114003:0:19928:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.114005:0:19928:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.114006:0:19928:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.114008:0:19928:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 376 -> 376 00000100:00100000:2.0:1780298642.114056:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b69880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+26:2399:x1866777609340672:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 188us (304us total) trans 4294969604 rc 0/0 00000100:00100000:2.0:1780298642.114067:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5918 00002000:00010000:2.0:1780298642.114073:0:19928:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:2.0:1780298642.114075:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b69880 with x1866777609340672 ext(4194304->18446744073709551615) 00002000:00010000:2.0:1780298642.114080:0:19928:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf242b69880. 00002000:00010000:2.0:1780298642.114242:0:19950:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 3275867 -> 3301156 00002000:00010000:2.0:1780298642.114245:0:19950:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.114247:0:19950:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.114249:0:19950:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.114250:0:19950:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:2.0:1780298642.114284:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2399:x1866777609340800:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 315us (400us total) trans 4294969168 rc 0/0 00000100:00100000:2.0:1780298642.114289:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5919 00002000:00010000:2.0:1780298642.114303:0:19950:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (3301156->18446744073709551615) 00002000:00100000:2.0:1780298642.114306:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2548ba300 with x1866777609340800 ext(3301156->18446744073709551615) 00002000:00010000:2.0:1780298642.114312:0:19950:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2548ba300. 00000100:00100000:0.0:1780298642.115422:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.115453:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609340928 00000100:00100000:1.0:1780298642.115457:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609340928 00002000:00100000:1.0:1780298642.115477:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x12:0x0] (3297280->3301375) req@ffff9cf2548b9880 x1866777609340928/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298642.115485:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2548b9880 with x1866777609340928 ext(3297280->3301375) 00002000:00010000:1.0:1780298642.115488:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2548b9880 00000100:00100000:1.0:1780298642.115504:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5920 00000100:00100000:1.0:1780298642.115510:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2400:x1866777609340928:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298642.115648:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2400:x1866777609340928:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 137us (227us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298642.115656:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5920 00002000:00100000:1.0:1780298642.115663:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x12:0x0] (3297280->3301375) req@ffff9cf2548b9880 x1866777609340928/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/4536 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298642.115669:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2548b9880 with x1866777609340928 ext(3297280->3301375) 00002000:00010000:1.0:1780298642.115672:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2548b9880 00000100:00100000:1.0:1780298642.116744:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.116788:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609341056 00000100:00100000:2.0:1780298642.116795:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609341056 00002000:00100000:2.0:1780298642.116821:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (3297280->3301155) req@ffff9cf242b6b100 x1866777609341056/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 4368/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.116828:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b6b100 with x1866777609341056 ext(3297280->3301155) 00002000:00010000:2.0:1780298642.116832:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf242b6b100 00000100:00100000:2.0:1780298642.116857:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5921 00000100:00100000:2.0:1780298642.116864:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b6b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2400:x1866777609341056:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:2.0:1780298642.117311:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b6b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2400:x1866777609341056:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 448us (568us total) trans 4294969169 rc 0/0 00000100:00100000:2.0:1780298642.117317:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5921 00002000:00100000:2.0:1780298642.117322:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (3297280->3301155) req@ffff9cf242b6b100 x1866777609341056/t4294969169(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 4368/448 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.117328:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b6b100 with x1866777609341056 ext(3297280->3301155) 00002000:00010000:2.0:1780298642.117332:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf242b6b100 00000100:00100000:0.0:1780298642.118007:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.118047:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609341184 00000100:00100000:2.0:1780298642.118051:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609341184 00000100:00100000:2.0:1780298642.118079:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8731 00000100:00100000:2.0:1780298642.118084:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2399:x1866777609341184:12345-192.168.203.25@tcp:16:fsx.0 00000001:00080000:0.0:1780298642.118907:0:8351:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0000: transno 4294969169 is committed 00000001:00080000:0.0:1780298642.118924:0:8351:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0000: transno 4294969167 is committed 00000100:00100000:2.0:1780298642.119033:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609341184:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 947us (1027us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.119038:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8731 00000100:00100000:0.0:1780298642.119965:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.120017:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609341312 00000100:00100000:2.0:1780298642.120021:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609341312 00000100:00100000:2.0:1780298642.120049:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61913 00000100:00100000:2.0:1780298642.120054:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548ba680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+84:29360:x1866777609341312:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.120067:0:6477:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.120085:0:6477:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9aa00/0xaf35132e63032c34 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 48 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1f3b expref: -99 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.120171:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9be00/0xaf35132e63032c3b lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 49 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.120184:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9be00/0xaf35132e63032c3b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 49 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.120195:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9be00/0xaf35132e63032c3b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 49 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298642.120219:0:6477:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 7495460 00000004:00200000:2.0:1780298642.120222:0:6477:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298642.120236:0:6477:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9be00/0xaf35132e63032c3b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 49 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.120254:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9aa00/0xaf35132e63032c34 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 49 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f3b expref: 86 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.120264:0:6477:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9be00/0xaf35132e63032c3b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 48 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f3b expref: 85 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.120270:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.120272:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298642.120275:0:6477:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9be00, rc 0) 00000100:00100000:2.0:1780298642.120310:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548ba680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+85:29360:x1866777609341312:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 255us (347us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.120316:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61913 00000100:00100000:1.0:1780298642.121856:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.121898:0:7411:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609341568 00000100:00100000:2.0:1780298642.121903:0:7411:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609341568 00000100:00100000:2.0:1780298642.121937:0:7411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61914 00000100:00100000:2.0:1780298642.121945:0:7411:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b68a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+89:29360:x1866777609341568:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:2.0:1780298642.122071:0:7411:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a200/0xaf35132e63032c42 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 49 type: IBT flags: 0x40000000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.122085:0:7411:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9be00/0xaf35132e63032c3b lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 49 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f3b expref: 84 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.122101:0:7411:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a200/0xaf35132e63032c42 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 49 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.122158:0:7411:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9be00/0xaf35132e63032c3b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 49 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f3b expref: 84 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.122172:0:7411:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9be00/0xaf35132e63032c3b lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 49 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f3b expref: 84 pid: 6477 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.122187:0:7411:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fc380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7411:1866777604855296:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298642.122269:0:7411:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4900 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.122974:0:7411:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fc380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7411:1866777604855296:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298642.122994:0:7411:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a200/0xaf35132e63032c42 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 49 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.123178:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.123212:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609341696 00000100:00100000:0.0:1780298642.123219:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609341696 00010000:00010000:0.0:1780298642.123242:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9be00/0xaf35132e63032c3b lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 49 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f3b expref: 85 pid: 6477 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.123267:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14953 00000100:00100000:0.0:1780298642.123274:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+85:2400:x1866777609341696:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298642.123284:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298642.123289:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9be00/0xaf35132e63032c3b lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 50 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f3b expref: 85 pid: 6477 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.123311:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9be00/0xaf35132e63032c3b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 50 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f3b expref: 85 pid: 6477 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.123326:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9be00/0xaf35132e63032c3b lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 50 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f3b expref: 85 pid: 6477 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.123336:0:20253:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298642.123340:0:20253:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a200/0xaf35132e63032c42 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 49 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.123354:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a200/0xaf35132e63032c42 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 49 type: IBT flags: 0x40290000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.123359:0:20253:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a200/0xaf35132e63032c42 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 49 type: IBT flags: 0x40290000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.123377:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298642.123398:0:7411:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a200/0xaf35132e63032c42 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 48 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.123406:0:7411:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a200/0xaf35132e63032c42 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 48 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.123412:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+84:2400:x1866777609341696:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 139us (237us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.123418:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14953 00000001:00080000:2.0:1780298642.123452:0:7411:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294990998 00000004:00010000:2.0:1780298642.123480:0:7411:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000310d0ea5 reply state 000000004e736d8b transno 4294990999 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a200/0xaf35132e63032c42 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 48 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.123493:0:7411:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a200/0xaf35132e63032c42 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 48 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.123504:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.123507:0:7411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1780298642.123511:0:7411:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1780298642.123575:0:7411:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b68a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+92:29360:x1866777609341568:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1631us (1720us total) trans 4294990999 rc 0/0 00000100:00100000:2.0:1780298642.123582:0:7411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61914 00000020:00080000:2.0:1780298642.123587:0:7411:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1780298642.124309:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.124340:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609341824 00000100:00100000:0.0:1780298642.124344:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609341824 00000100:00100000:0.0:1780298642.124370:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8732 00000100:00100000:0.0:1780298642.124375:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255488380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:29360:x1866777609341824:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.124385:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.124396:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63032c49 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1f42 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.124413:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e63032c26 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f2d expref: 25 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.124419:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(90 177) 00010000:00010000:0.0:1780298642.124422:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(91 177) 00010000:00010000:0.0:1780298642.124425:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63032c49 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f42 expref: 15 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.124442:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e63032c26 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f2d expref: 25 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.124451:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e63032c26 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f2d expref: 25 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.124459:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548b8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604855424:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298642.124498:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417e40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.125108:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548b8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604855424:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298642.125145:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63032c49 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f42 expref: 15 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.125157:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bafa00, rc 0) 00000100:00100000:1.0:1780298642.125183:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255488380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:29360:x1866777609341824:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 808us (875us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.125188:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8732 00000100:00100000:3.0:1780298642.125349:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.125377:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609341952 00000100:00100000:0.0:1780298642.125381:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609341952 00010000:00010000:0.0:1780298642.125398:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e63032c26 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f2d expref: 26 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.125425:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14954 00000100:00100000:0.0:1780298642.125433:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25488a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+26:2400:x1866777609341952:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298642.125441:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298642.125456:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e63032c26 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f2d expref: 26 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.125463:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e63032c26 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f2d expref: 26 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.125475:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e63032c26 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f2d expref: 26 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.125485:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(91 177) 00010000:00010000:0.0:1780298642.125489:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(91 177) 00010000:00010000:0.0:1780298642.125494:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63032c49 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f42 expref: 14 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.125503:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63032c49 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f42 expref: 14 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.125526:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63032c49 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f42 expref: 14 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.125536:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548b8700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777604855552:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298642.125572:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417300 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.126272:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548b8700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777604855552:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298642.126283:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298642.126325:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25488a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+25:2400:x1866777609341952:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 892us (974us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298642.126334:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14954 00000100:00100000:3.0:1780298642.126522:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.126560:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609342080 00000100:00100000:1.0:1780298642.126580:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609342080 00000100:00100000:1.0:1780298642.126616:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8733 00000100:00100000:1.0:1780298642.126621:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254889f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609342080:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.126629:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.126638:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bf17d487/0xaf35132e63032c50 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1f49 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.126651:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000007b6539a7/0xaf35132e63032c2d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f34 expref: 6 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.126656:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(67 130) 00010000:00010000:1.0:1780298642.126659:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(68 130) 00010000:00010000:1.0:1780298642.126664:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bf17d487/0xaf35132e63032c50 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f49 expref: 10 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.126685:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000007b6539a7/0xaf35132e63032c2d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f34 expref: 6 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.126697:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000007b6539a7/0xaf35132e63032c2d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f34 expref: 6 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.126707:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604855680:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298642.126737:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12900 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.127435:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604855680:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298642.127446:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bf17d487/0xaf35132e63032c50 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f49 expref: 10 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.127455:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37400, rc 0) 00000100:00100000:1.0:1780298642.127475:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254889f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609342080:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 855us (955us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.127480:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8733 00000100:00100000:3.0:1780298642.127649:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.127678:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609342208 00000100:00100000:0.0:1780298642.127683:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609342208 00010000:00010000:0.0:1780298642.127701:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000007b6539a7/0xaf35132e63032c2d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f34 expref: 7 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.127729:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14955 00000100:00100000:0.0:1780298642.127739:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25488b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609342208:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298642.127749:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:0.0:1780298642.127759:0:6461:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 0 -> 8 00010000:00010000:0.0:1780298642.127765:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000007b6539a7/0xaf35132e63032c2d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f34 expref: 7 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.127776:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000007b6539a7/0xaf35132e63032c2d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f34 expref: 7 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.127796:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007b6539a7/0xaf35132e63032c2d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f34 expref: 7 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.127806:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(68 130) 00010000:00010000:0.0:1780298642.127810:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(68 130) 00010000:00010000:0.0:1780298642.127815:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000bf17d487/0xaf35132e63032c50 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f49 expref: 9 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.127823:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bf17d487/0xaf35132e63032c50 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f49 expref: 9 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.127849:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000bf17d487/0xaf35132e63032c50 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f49 expref: 9 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.127859:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548b8e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6461:1866777604855808:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298642.127904:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417a80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.128669:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548b8e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6461:1866777604855808:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298642.128679:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298642.128750:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25488b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609342208:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1012us (1142us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298642.128758:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14955 00000100:00100000:1.0:1780298642.128819:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.128841:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609342336 00000100:00100000:0.0:1780298642.128845:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609342336 00000100:00100000:1.0:1780298642.128849:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00002000:00010000:0.0:1780298642.128868:0:19951:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (5726782->18446744073709551615) 00002000:00100000:0.0:1780298642.128872:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25548b800 with x1866777609342336 ext(5726782->18446744073709551615) 00002000:00010000:0.0:1780298642.128877:0:19951:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf25548b800. 00000100:00100000:3.0:1780298642.128895:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609342464 00000100:00100000:0.0:1780298642.128900:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5922 00000100:00100000:3.0:1780298642.128901:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609342464 00000100:00100000:0.0:1780298642.128910:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2400:x1866777609342336:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:3.0:1780298642.128920:0:8363:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4194304->18446744073709551615) 00002000:00100000:3.0:1780298642.128925:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255489c00 with x1866777609342464 ext(4194304->18446744073709551615) 00002000:00010000:3.0:1780298642.128931:0:8363:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf255489c00. 00000100:00100000:3.0:1780298642.128955:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5923 00000100:00100000:3.0:1780298642.128964:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255489c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609342464:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:3.0:1780298642.129087:0:8363:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 3301156 -> 4194304 00002000:00010000:0.0:1780298642.129088:0:19951:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 5726782 00002000:00010000:3.0:1780298642.129091:0:8363:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298642.129092:0:19951:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:3.0:1780298642.129095:0:8363:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298642.129095:0:19951:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:3.0:1780298642.129099:0:8363:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298642.129100:0:19951:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:3.0:1780298642.129103:0:8363:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 8 -> 8 00002000:00010000:0.0:1780298642.129104:0:19951:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 376 -> 376 00000100:00100000:3.0:1780298642.129155:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255489c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609342464:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 191us (305us total) trans 4294969170 rc 0/0 00000100:00100000:0.0:1780298642.129161:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609342336:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 252us (339us total) trans 4294969605 rc 0/0 00000100:00100000:3.0:1780298642.129164:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5923 00002000:00010000:3.0:1780298642.129170:0:8363:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4194304->18446744073709551615) 00000100:00100000:0.0:1780298642.129174:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5922 00002000:00100000:3.0:1780298642.129175:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255489c00 with x1866777609342464 ext(4194304->18446744073709551615) 00002000:00010000:0.0:1780298642.129178:0:19951:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (5726782->18446744073709551615) 00002000:00010000:3.0:1780298642.129180:0:8363:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf255489c00. 00002000:00100000:0.0:1780298642.129184:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25548b800 with x1866777609342336 ext(5726782->18446744073709551615) 00002000:00010000:0.0:1780298642.129190:0:19951:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf25548b800. 00000100:00100000:0.0:1780298642.129880:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.129924:0:6476:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609342592 00000100:00100000:2.0:1780298642.129935:0:6476:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609342592 00000100:00100000:2.0:1780298642.129984:0:6476:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61915 00000100:00100000:2.0:1780298642.129995:0:6476:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+92:29360:x1866777609342592:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.130016:0:6476:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.130033:0:6476:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b000/0xaf35132e63032c57 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 49 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1f50 expref: -99 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.130117:0:6476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ac00/0xaf35132e63032c5e lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 50 type: IBT flags: 0x40000000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.130158:0:6476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ac00/0xaf35132e63032c5e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 50 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.130170:0:6476:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ac00/0xaf35132e63032c5e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 50 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298642.130190:0:6476:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 9921086 00000004:00200000:2.0:1780298642.130194:0:6476:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298642.130208:0:6476:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ac00/0xaf35132e63032c5e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 50 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.130229:0:6476:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b000/0xaf35132e63032c57 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 50 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f50 expref: 94 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.130245:0:6476:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ac00/0xaf35132e63032c5e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 49 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f50 expref: 93 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.130253:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.130256:0:6476:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298642.130259:0:6476:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9ac00, rc 0) 00000100:00100000:2.0:1780298642.130316:0:6476:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+93:29360:x1866777609342592:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 324us (436us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.130321:0:6476:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61915 00000100:00100000:1.0:1780298642.131790:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.131826:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609342720 00000100:00100000:2.0:1780298642.131831:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609342720 00000100:00100000:2.0:1780298642.131864:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5924 00000100:00100000:2.0:1780298642.131868:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255489500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609342720:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298642.132003:0:19950:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 5726782 -> 5726782 00002000:00010000:2.0:1780298642.132006:0:19950:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.132008:0:19950:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.132009:0:19950:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.132011:0:19950:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 376 -> 384 00000100:00100000:2.0:1780298642.132042:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255489500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609342720:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 173us (252us total) trans 4294969606 rc 0/0 00000100:00100000:2.0:1780298642.132050:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5924 00000100:00100000:1.0:1780298642.133259:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.133306:0:9976:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609342848 00000100:00100000:2.0:1780298642.133312:0:9976:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609342848 00000100:00100000:2.0:1780298642.133350:0:9976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61916 00000100:00100000:2.0:1780298642.133358:0:9976:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255488700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+84:29360:x1866777609342848:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.133370:0:9976:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.133381:0:9976:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9aa00/0xaf35132e63032c65 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 50 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1f57 expref: -99 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.133435:0:9976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693e00/0xaf35132e63032c6c lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 51 type: IBT flags: 0x40000000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.133443:0:9976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693e00/0xaf35132e63032c6c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 51 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.133450:0:9976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693e00/0xaf35132e63032c6c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 51 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298642.133463:0:9976:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 9921086 00000004:00200000:2.0:1780298642.133465:0:9976:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298642.133475:0:9976:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693e00/0xaf35132e63032c6c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 51 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.133488:0:9976:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9aa00/0xaf35132e63032c65 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 51 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f57 expref: 86 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.133497:0:9976:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693e00/0xaf35132e63032c6c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 50 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f57 expref: 85 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.133502:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.133504:0:9976:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298642.133506:0:9976:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341693e00, rc 0) 00000100:00100000:2.0:1780298642.133532:0:9976:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255488700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+85:29360:x1866777609342848:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 177us (275us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.133537:0:9976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61916 00000100:00100000:0.0:1780298642.134610:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.134696:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609342976 00000100:00100000:3.0:1780298642.134713:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609342976 00000100:00100000:1.0:1780298642.134747:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.134750:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8734 00000100:00100000:3.0:1780298642.134758:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+25:2398:x1866777609342976:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298642.134767:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609343104 00010000:00010000:3.0:1780298642.134771:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298642.134773:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609343104 00010000:00010000:3.0:1780298642.134786:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63032c73 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1f5e expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.134814:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8735 00000100:00100000:0.0:1780298642.134826:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255489f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609343104:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.134836:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.134845:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e63032c7a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1f65 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.134883:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254889500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604855936:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.134885:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548b8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604856064:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.134956:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348ce1840 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.134956:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.135683:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254889500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604855936:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.135705:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63032c73 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f5e expref: 26 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.135776:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63032c73 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f5e expref: 26 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.135787:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3000, rc 0) 00000100:00100000:3.0:1780298642.135821:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+25:2398:x1866777609342976:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1064us (1213us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.135831:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8734 00000100:00100000:2.0:1780298642.135947:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548b8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604856064:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.135971:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e63032c7a lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f65 expref: 7 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.135987:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e63032c7a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f65 expref: 7 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.135995:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baea00, rc 0) 00000100:00100000:2.0:1780298642.136030:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255489f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609343104:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1209us (1284us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.136035:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8735 00000100:00100000:0.0:1780298642.136916:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.136966:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609343232 00000100:00100000:1.0:1780298642.136972:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609343232 00000100:00100000:1.0:1780298642.137012:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8736 00000100:00100000:3.0:1780298642.137019:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.137026:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+25:2397:x1866777609343232:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.137035:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298642.137042:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609343360 00010000:00010000:1.0:1780298642.137047:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63032c81 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1f6c expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.137048:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609343360 00000100:00100000:1.0:1780298642.137079:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604856192:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.137089:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8737 00000100:00100000:0.0:1780298642.137099:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254889500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609343360:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.137108:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.137117:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63032c88 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1f73 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.137174:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12900 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.137178:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243063800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604856320:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.137222:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417c00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.137883:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604856192:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.137907:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63032c81 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f6c expref: 26 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.137925:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63032c81 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f6c expref: 26 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.137937:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36a00, rc 0) 00000100:00100000:0.0:1780298642.137953:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243063800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604856320:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.137966:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+25:2397:x1866777609343232:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 944us (1051us total) trans 0 rc 301/301 00010000:00010000:0.0:1780298642.137968:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63032c88 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f73 expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.137975:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8736 00010000:00010000:0.0:1780298642.137988:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63032c88 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f73 expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.137998:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae400, rc 0) 00000100:00100000:0.0:1780298642.138023:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254889500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609343360:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 926us (1008us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.138030:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8737 00000100:00100000:0.0:1780298642.139252:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.139282:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609343488 00000100:00100000:1.0:1780298642.139286:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609343488 00000100:00100000:1.0:1780298642.139326:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5925 00000100:00100000:1.0:1780298642.139331:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243063800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609343488:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298642.139523:0:19943:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 5726782 -> 5726782 00002000:00010000:1.0:1780298642.139527:0:19943:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:1.0:1780298642.139530:0:19943:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:1.0:1780298642.139532:0:19943:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:1.0:1780298642.139534:0:19943:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 384 -> 400 00000100:00100000:1.0:1780298642.139565:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243063800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609343488:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 232us (311us total) trans 4294969607 rc 0/0 00000100:00100000:1.0:1780298642.139572:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5925 00000100:00100000:3.0:1780298642.140364:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.140410:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609343616 00000100:00100000:2.0:1780298642.140414:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609343616 00000100:00100000:2.0:1780298642.140436:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8738 00000100:00100000:2.0:1780298642.140442:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254888000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+25:2397:x1866777609343616:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.140448:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.140455:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000036bac5fe/0xaf35132e63032c8f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1f7a expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.140485:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604856448:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.140521:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.140524:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.140565:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609343744 00000100:00100000:1.0:1780298642.140570:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609343744 00000100:00100000:1.0:1780298642.140602:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8739 00000100:00100000:1.0:1780298642.140610:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243060000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609343744:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.140618:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.140626:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003c8dab73/0xaf35132e63032c96 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1f81 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.140653:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604856576:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.140689:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12900 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.141328:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604856448:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.141328:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604856576:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.141345:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000036bac5fe/0xaf35132e63032c8f lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f7a expref: 26 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.141345:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003c8dab73/0xaf35132e63032c96 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f81 expref: 7 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.141359:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000036bac5fe/0xaf35132e63032c8f lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f7a expref: 26 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.141359:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003c8dab73/0xaf35132e63032c96 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f81 expref: 7 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.141369:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36800, rc 0) 00010000:00010000:2.0:1780298642.141371:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341693400, rc 0) 00000100:00100000:2.0:1780298642.141398:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254888000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+25:2397:x1866777609343616:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 955us (1033us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.141398:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243060000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609343744:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 788us (879us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.141405:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8738 00000100:00100000:1.0:1780298642.141410:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8739 00000100:00100000:3.0:1780298642.142354:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.142374:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.142391:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609343872 00000100:00100000:2.0:1780298642.142395:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609343872 00000100:00100000:1.0:1780298642.142407:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609344000 00000100:00100000:1.0:1780298642.142412:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609344000 00000100:00100000:2.0:1780298642.142426:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8740 00000100:00100000:2.0:1780298642.142435:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25488bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+25:2399:x1866777609343872:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.142444:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298642.142450:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8741 00010000:00010000:2.0:1780298642.142453:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d7062f72/0xaf35132e63032c9d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1f88 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.142463:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254888a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609344000:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.142472:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.142482:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032ca4 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1f8f expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.142485:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604856704:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.142520:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604856832:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.142529:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.142552:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12900 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.143181:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604856704:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.143192:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d7062f72/0xaf35132e63032c9d lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f88 expref: 26 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.143202:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d7062f72/0xaf35132e63032c9d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f88 expref: 26 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.143209:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341692e00, rc 0) 00000100:00100000:2.0:1780298642.143231:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25488bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+25:2399:x1866777609343872:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 800us (878us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.143238:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8740 00000100:00100000:1.0:1780298642.143435:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604856832:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.143451:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032ca4 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f8f expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.143466:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032ca4 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f8f expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.143475:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37000, rc 0) 00000100:00100000:1.0:1780298642.143507:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254888a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609344000:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1050us (1131us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.143513:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8741 00000100:00100000:0.0:1780298642.144476:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.144511:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609344128 00000100:00100000:2.0:1780298642.144516:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609344128 00002000:00100000:2.0:1780298642.144542:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (446464->450559) req@ffff9cf243063b80 x1866777609344128/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.144553:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243063b80 with x1866777609344128 ext(446464->450559) 00002000:00010000:2.0:1780298642.144558:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf243063b80 00000100:00100000:2.0:1780298642.144581:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5926 00000100:00100000:2.0:1780298642.144589:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243063b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609344128:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298642.144746:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243063b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609344128:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 157us (269us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298642.144754:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5926 00002000:00100000:2.0:1780298642.144760:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (446464->450559) req@ffff9cf243063b80 x1866777609344128/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/4536 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.144769:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243063b80 with x1866777609344128 ext(446464->450559) 00002000:00010000:2.0:1780298642.144773:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf243063b80 00000100:00100000:0.0:1780298642.145662:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.145704:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609344256 00000100:00100000:2.0:1780298642.145709:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609344256 00002000:00100000:2.0:1780298642.145730:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (454656->458751) req@ffff9cf243060a80 x1866777609344256/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.145746:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243060a80 with x1866777609344256 ext(454656->458751) 00002000:00010000:2.0:1780298642.145750:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf243060a80 00000100:00100000:2.0:1780298642.145770:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5927 00000100:00100000:2.0:1780298642.145778:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243060a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2399:x1866777609344256:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298642.145883:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243060a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2399:x1866777609344256:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 106us (221us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298642.145890:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5927 00002000:00100000:2.0:1780298642.145896:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (454656->458751) req@ffff9cf243060a80 x1866777609344256/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/4536 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.145904:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243060a80 with x1866777609344256 ext(454656->458751) 00002000:00010000:2.0:1780298642.145907:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf243060a80 00000100:00100000:3.0:1780298642.146987:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.147037:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609344384 00000100:00100000:2.0:1780298642.147042:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609344384 00000100:00100000:3.0:1780298642.147068:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.147085:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609344512 00000100:00100000:2.0:1780298642.147089:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609344512 00000100:00100000:0.0:1780298642.147109:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8742 00000100:00100000:2.0:1780298642.147121:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8743 00000100:00100000:0.0:1780298642.147122:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254888380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+25:2399:x1866777609344384:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.147148:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298642.147154:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25488a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609344512:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.147158:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c2d588e1/0xaf35132e63032cab lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1f96 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.147168:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.147178:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d1b192d6/0xaf35132e63032cb2 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1f9d expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.147198:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243062300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604856960:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.147215:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604857088:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.147239:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417c00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.147247:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.147917:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243062300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604856960:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.147937:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c2d588e1/0xaf35132e63032cab lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f96 expref: 26 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.147952:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c2d588e1/0xaf35132e63032cab lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f96 expref: 26 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.147963:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf800, rc 0) 00000100:00100000:2.0:1780298642.147997:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254888380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+25:2399:x1866777609344384:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 879us (1010us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.148004:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8742 00000100:00100000:1.0:1780298642.148013:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604857088:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.148034:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d1b192d6/0xaf35132e63032cb2 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f9d expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.148048:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d1b192d6/0xaf35132e63032cb2 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f9d expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.148058:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341692400, rc 0) 00000100:00100000:1.0:1780298642.148087:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25488a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609344512:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 940us (1020us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.148095:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8743 00000100:00100000:3.0:1780298642.148999:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.149034:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.149048:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609344640 00000100:00100000:0.0:1780298642.149053:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609344640 00000100:00100000:2.0:1780298642.149071:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609344768 00000100:00100000:2.0:1780298642.149076:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609344768 00000100:00100000:0.0:1780298642.149084:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8744 00000100:00100000:0.0:1780298642.149092:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25488ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+25:2400:x1866777609344640:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.149100:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.149107:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63032cb9 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1fa4 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.149116:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8745 00000100:00100000:1.0:1780298642.149148:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25488aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609344768:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298642.149154:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243061f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604857216:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.149156:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.149168:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a10929f0/0xaf35132e63032cc0 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1fab expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.149192:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.149201:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604857344:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.149234:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12900 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.149973:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243061f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604857216:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.149991:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63032cb9 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fa4 expref: 26 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.150008:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63032cb9 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fa4 expref: 26 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.150020:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf200, rc 0) 00000100:00100000:1.0:1780298642.150043:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604857344:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.150062:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a10929f0/0xaf35132e63032cc0 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fab expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.150078:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a10929f0/0xaf35132e63032cc0 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fab expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.150095:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37200, rc 0) 00000100:00100000:3.0:1780298642.150100:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25488ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+25:2400:x1866777609344640:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1006us (1099us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.150109:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8744 00000100:00100000:1.0:1780298642.150145:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25488aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609344768:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1021us (1111us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.150151:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8745 00000100:00100000:0.0:1780298642.151589:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.151631:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609345024 00000100:00100000:2.0:1780298642.151636:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609345024 00000100:00100000:2.0:1780298642.151682:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61917 00000100:00100000:2.0:1780298642.151691:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243060380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+85:29360:x1866777609345024:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:2.0:1780298642.151737:0:6477:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298642.151756:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693e00/0xaf35132e63032c6c lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 51 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f57 expref: 85 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.151771:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.151776:0:6477:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298642.151963:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692e00/0xaf35132e63032cc7 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 50 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.151978:0:6477:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ac00/0xaf35132e63032c5e lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 50 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f50 expref: 92 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.152001:0:6477:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692e00/0xaf35132e63032cc7 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 50 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.152026:0:6477:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ac00/0xaf35132e63032c5e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 50 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f50 expref: 92 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.152039:0:6477:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ac00/0xaf35132e63032c5e lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 50 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f50 expref: 92 pid: 6476 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.152052:0:6477:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fd180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6477:1866777604857472:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298642.152106:0:6477:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.152762:0:6477:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fd180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6477:1866777604857472:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298642.152773:0:6477:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692e00/0xaf35132e63032cc7 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 50 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.152959:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.153002:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609345152 00000100:00100000:1.0:1780298642.153007:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609345152 00010000:00010000:1.0:1780298642.153031:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ac00/0xaf35132e63032c5e lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 50 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f50 expref: 93 pid: 6476 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.153053:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14956 00000100:00100000:1.0:1780298642.153059:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243062680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+93:2400:x1866777609345152:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298642.153066:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298642.153070:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ac00/0xaf35132e63032c5e lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 51 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f50 expref: 93 pid: 6476 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.153080:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ac00/0xaf35132e63032c5e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 51 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f50 expref: 93 pid: 6476 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.153091:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ac00/0xaf35132e63032c5e lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 51 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f50 expref: 93 pid: 6476 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.153098:0:7400:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298642.153101:0:7400:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692e00/0xaf35132e63032cc7 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 50 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.153108:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692e00/0xaf35132e63032cc7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 50 type: IBT flags: 0x40290000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.153114:0:7400:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692e00/0xaf35132e63032cc7 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 50 type: IBT flags: 0x40290000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.153154:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298642.153173:0:6477:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692e00/0xaf35132e63032cc7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 49 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.153182:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243062680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+92:2400:x1866777609345152:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 121us (222us total) trans 0 rc 0/0 00010000:00010000:2.0:1780298642.153184:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692e00/0xaf35132e63032cc7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 49 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.153197:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14956 00000001:00080000:2.0:1780298642.153235:0:6477:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991000 00000004:00010000:2.0:1780298642.153259:0:6477:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000a21a74d5 reply state 000000005d96f96f transno 4294991001 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692e00/0xaf35132e63032cc7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 49 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.153269:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692e00/0xaf35132e63032cc7 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 49 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.153280:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.153282:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1780298642.153285:0:6477:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1780298642.153355:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243060380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+87:29360:x1866777609345024:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1666us (1766us total) trans 4294991001 rc 0/0 00000100:00100000:2.0:1780298642.153368:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61917 00000020:00080000:2.0:1780298642.153372:0:6477:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1780298642.154060:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.154088:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609345280 00000100:00100000:3.0:1780298642.154092:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609345280 00000100:00100000:3.0:1780298642.154121:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8746 00000100:00100000:3.0:1780298642.154148:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243063480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+25:29360:x1866777609345280:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.154158:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.154168:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63032cce lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1fb2 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.154187:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63032c49 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f42 expref: 17 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.154195:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(91 177) 00010000:00010000:3.0:1780298642.154198:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(92 177) 00010000:00010000:3.0:1780298642.154211:0:20252:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63032cce lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fb2 expref: 26 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.154227:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63032c49 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f42 expref: 17 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.154235:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63032c49 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f42 expref: 17 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.154242:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604857600:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298642.154275:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348ce1840 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.154859:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24310ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604857600:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298642.154872:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63032cce lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fb2 expref: 26 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.154880:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3000, rc 0) 00000100:00100000:3.0:1780298642.154902:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243063480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+26:29360:x1866777609345280:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 758us (844us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298642.154908:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8746 00000100:00100000:0.0:1780298642.155267:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.155320:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609345408 00000100:00100000:2.0:1780298642.155325:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609345408 00002000:00100000:2.0:1780298642.155353:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (446464->458751) req@ffff9cf243061f80 x1866777609345408/t0(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 12776/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.155365:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243061f80 with x1866777609345408 ext(446464->458751) 00010000:00010000:2.0:1780298642.155372:0:8363:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63032c49 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f42 expref: 18 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.155386:0:8363:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63032c49 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f42 expref: 18 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1780298642.155395:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf243061f80 00000100:00100000:2.0:1780298642.155417:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 182 00000100:00100000:2.0:1780298642.155426:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243061f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2399:x1866777609345408:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:2.0:1780298642.155948:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243061f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2399:x1866777609345408:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 523us (680us total) trans 4294969608 rc 0/0 00000100:00100000:2.0:1780298642.155956:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 182 00002000:00100000:2.0:1780298642.155963:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (446464->458751) req@ffff9cf243061f80 x1866777609345408/t4294969608(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 12776/448 e 0 to 0 dl 1780298653 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.155972:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243061f80 with x1866777609345408 ext(446464->458751) 00010000:00010000:2.0:1780298642.155978:0:8363:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63032c49 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f42 expref: 20 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.155989:0:8363:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63032c49 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f42 expref: 20 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1780298642.155996:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf243061f80 00000100:00100000:0.0:1780298642.156716:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.156753:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609345536 00000100:00100000:1.0:1780298642.156757:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609345536 00010000:00010000:1.0:1780298642.156772:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63032c49 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f42 expref: 20 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.156796:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14957 00000100:00100000:1.0:1780298642.156802:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243062300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2398:x1866777609345536:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298642.156809:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:1.0:1780298642.156817:0:6463:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 400 -> 424 00010000:00010000:1.0:1780298642.156821:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63032c49 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f42 expref: 20 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.156829:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63032c49 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f42 expref: 20 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.156842:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63032c49 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f42 expref: 20 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.156850:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(92 177) 00010000:00010000:1.0:1780298642.156853:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(92 177) 00010000:00010000:1.0:1780298642.156857:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63032cce lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fb2 expref: 25 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.156863:0:6463:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63032cce lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fb2 expref: 25 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.156879:0:6463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63032cce lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fb2 expref: 25 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.156885:0:6463:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548ad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6463:1866777604857728:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298642.156916:0:6463:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12900 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.157651:0:6463:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548ad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6463:1866777604857728:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298642.157662:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298642.157689:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243062300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2398:x1866777609345536:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 887us (973us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.157697:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14957 00000100:00100000:3.0:1780298642.157826:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.157856:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609345664 00000100:00100000:1.0:1780298642.157860:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609345664 00000100:00100000:1.0:1780298642.157879:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8747 00000100:00100000:1.0:1780298642.157884:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:29360:x1866777609345664:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.157892:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.157901:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032cd5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1fb9 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.157915:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000bf17d487/0xaf35132e63032c50 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f49 expref: 10 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.157923:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(68 130) 00010000:00010000:1.0:1780298642.157926:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(69 130) 00010000:00010000:1.0:1780298642.157931:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032cd5 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fb9 expref: 7 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.157950:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000bf17d487/0xaf35132e63032c50 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f49 expref: 10 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.157967:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000bf17d487/0xaf35132e63032c50 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f49 expref: 10 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.157979:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604857856:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298642.158017:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12900 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.158696:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604857856:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298642.158707:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032cd5 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fb9 expref: 7 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.158717:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37000, rc 0) 00000100:00100000:1.0:1780298642.158739:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609345664:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 855us (915us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.158744:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8747 00000100:00100000:3.0:1780298642.158860:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.158895:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609345792 00000100:00100000:1.0:1780298642.158901:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609345792 00010000:00010000:1.0:1780298642.158919:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000bf17d487/0xaf35132e63032c50 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f49 expref: 11 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.158944:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14958 00000100:00100000:1.0:1780298642.158950:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b1f450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609345792:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298642.158959:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298642.158982:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000bf17d487/0xaf35132e63032c50 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f49 expref: 11 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.158994:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000bf17d487/0xaf35132e63032c50 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f49 expref: 11 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.159009:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bf17d487/0xaf35132e63032c50 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1f49 expref: 11 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.159021:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(69 130) 00010000:00010000:1.0:1780298642.159024:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(69 130) 00010000:00010000:1.0:1780298642.159029:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032cd5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fb9 expref: 6 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.159037:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032cd5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fb9 expref: 6 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.159058:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032cd5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fb9 expref: 6 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.159075:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548ad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:8374:1866777604857984:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298642.159107:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12900 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.159786:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548ad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:8374:1866777604857984:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298642.159794:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298642.159814:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b1f450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609345792:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 866us (958us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.159820:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14958 00000100:00100000:3.0:1780298642.159897:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.160016:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609345920 00000100:00100000:2.0:1780298642.160021:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609345920 00000100:00100000:3.0:1780298642.160030:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00002000:00010000:2.0:1780298642.160042:0:19950:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (1798145->18446744073709551615) 00002000:00100000:2.0:1780298642.160047:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24310c000 with x1866777609345920 ext(1798145->18446744073709551615) 00002000:00010000:2.0:1780298642.160055:0:19950:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24310c000. 00000100:00100000:2.0:1780298642.160077:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609346048 00000100:00100000:2.0:1780298642.160081:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609346048 00000100:00100000:0.0:1780298642.160087:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5928 00002000:00010000:2.0:1780298642.160096:0:19950:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00000100:00100000:0.0:1780298642.160100:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+26:2400:x1866777609345920:12345-192.168.203.25@tcp:10:fsx.0 00002000:00100000:2.0:1780298642.160101:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f754700 with x1866777609346048 ext(0->18446744073709551615) 00002000:00010000:2.0:1780298642.160109:0:19950:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f754700. 00000100:00100000:2.0:1780298642.160151:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5929 00000100:00100000:2.0:1780298642.160162:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f754700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609346048:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:2.0:1780298642.160383:0:19950:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4194304 -> 0 00002000:00010000:2.0:1780298642.160387:0:19950:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.160390:0:19950:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.160393:0:19950:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.160396:0:19950:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 8 -> 0 00002000:00010000:0.0:1780298642.160407:0:16972:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 5726782 -> 1798145 00002000:00010000:0.0:1780298642.160412:0:16972:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298642.160415:0:16972:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298642.160418:0:16972:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298642.160420:0:16972:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 424 -> 384 00000100:00100000:2.0:1780298642.160432:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f754700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609346048:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 272us (402us total) trans 4294969171 rc 0/0 00000100:00100000:2.0:1780298642.160441:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5929 00002000:00010000:2.0:1780298642.160445:0:19950:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00000100:00100000:0.0:1780298642.160450:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+27:2400:x1866777609345920:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 353us (552us total) trans 4294969609 rc 0/0 00002000:00100000:2.0:1780298642.160451:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f754700 with x1866777609346048 ext(0->18446744073709551615) 00002000:00010000:2.0:1780298642.160459:0:19950:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f754700. 00000100:00100000:0.0:1780298642.160464:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5928 00002000:00010000:0.0:1780298642.160468:0:16972:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (1798145->18446744073709551615) 00002000:00100000:0.0:1780298642.160472:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24310c000 with x1866777609345920 ext(1798145->18446744073709551615) 00002000:00010000:0.0:1780298642.160477:0:16972:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24310c000. 00000100:00100000:0.0:1780298642.161185:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.161219:0:7411:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609346176 00000100:00100000:2.0:1780298642.161224:0:7411:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609346176 00000100:00100000:2.0:1780298642.161257:0:7411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61918 00000100:00100000:2.0:1780298642.161265:0:7411:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243062680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+87:29360:x1866777609346176:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.161280:0:7411:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.161304:0:7411:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693e00/0xaf35132e63032cdc lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 50 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1fc0 expref: -99 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.161379:0:7411:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693400/0xaf35132e63032ce3 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 51 type: IBT flags: 0x40000000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.161391:0:7411:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693400/0xaf35132e63032ce3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 51 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.161401:0:7411:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693400/0xaf35132e63032ce3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 51 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298642.161426:0:7411:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 1798145 00000004:00200000:2.0:1780298642.161431:0:7411:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298642.161449:0:7411:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693400/0xaf35132e63032ce3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 51 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.161474:0:7411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693e00/0xaf35132e63032cdc lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 51 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fc0 expref: 89 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.161492:0:7411:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693400/0xaf35132e63032ce3 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 50 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fc0 expref: 88 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.161499:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.161502:0:7411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298642.161504:0:7411:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341693400, rc 0) 00000100:00100000:2.0:1780298642.161534:0:7411:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243062680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+88:29360:x1866777609346176:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 270us (348us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.161541:0:7411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61918 00000100:00100000:3.0:1780298642.162650:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.162683:0:6476:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609346304 00000100:00100000:2.0:1780298642.162687:0:6476:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609346304 00000100:00100000:2.0:1780298642.162711:0:6476:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61919 00000100:00100000:2.0:1780298642.162718:0:6476:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f757480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+92:29360:x1866777609346304:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.162726:0:6476:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.162734:0:6476:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692400/0xaf35132e63032cea lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 51 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1fc7 expref: -99 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.162766:0:6476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be600/0xaf35132e63032cf1 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 52 type: IBT flags: 0x40000000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.162774:0:6476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be600/0xaf35132e63032cf1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 52 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.162779:0:6476:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be600/0xaf35132e63032cf1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 52 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298642.162789:0:6476:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 1798145 00000004:00200000:2.0:1780298642.162792:0:6476:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298642.162799:0:6476:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be600/0xaf35132e63032cf1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 52 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.162810:0:6476:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692400/0xaf35132e63032cea lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 52 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fc7 expref: 94 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.162818:0:6476:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be600/0xaf35132e63032cf1 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 51 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fc7 expref: 93 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.162823:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.162824:0:6476:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298642.162826:0:6476:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf2445be600, rc 0) 00000100:00100000:2.0:1780298642.162848:0:6476:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f757480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+93:29360:x1866777609346304:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 132us (200us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.162852:0:6476:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61919 00000100:00100000:3.0:1780298642.163827:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.163889:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609346432 00000100:00100000:0.0:1780298642.163894:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609346432 00000100:00100000:3.0:1780298642.163919:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.163942:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609346560 00000100:00100000:0.0:1780298642.163946:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609346560 00000100:00100000:1.0:1780298642.163976:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8748 00000100:00100000:0.0:1780298642.163986:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8749 00000100:00100000:1.0:1780298642.163991:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f754380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2398:x1866777609346432:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298642.163998:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243809c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609346560:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.164002:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.164005:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.164014:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e63032cf8 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1fce expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.164015:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c37b15a2/0xaf35132e63032cff lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1fd5 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.164059:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243062300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604858112:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.164065:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604858240:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.164104:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417f00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.164107:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12f00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.165097:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604858240:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.165098:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243062300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604858112:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.165150:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e63032cf8 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fce expref: 20 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.165168:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e63032cf8 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fce expref: 20 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.165184:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37200, rc 0) 00010000:00010000:3.0:1780298642.165209:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c37b15a2/0xaf35132e63032cff lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fd5 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.165229:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f754380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2398:x1866777609346432:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1238us (1397us total) trans 0 rc 301/301 00010000:00010000:3.0:1780298642.165232:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c37b15a2/0xaf35132e63032cff lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fd5 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.165245:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8748 00010000:00010000:3.0:1780298642.165249:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baee00, rc 0) 00000100:00100000:3.0:1780298642.165276:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243809c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609346560:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1282us (1358us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.165283:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8749 00000100:00100000:3.0:1780298642.166312:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.166352:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609346816 00000100:00100000:3.0:1780298642.166357:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.166357:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609346816 00000100:00100000:0.0:1780298642.166389:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609346688 00000100:00100000:0.0:1780298642.166391:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609346688 00000100:00100000:1.0:1780298642.166412:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8750 00000100:00100000:0.0:1780298642.166422:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8751 00000100:00100000:1.0:1780298642.166427:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243808380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609346816:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.166436:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298642.166437:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24380a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2400:x1866777609346688:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.166448:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bf17d487/0xaf35132e63032d06 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1fe3 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.166448:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.166460:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005d10d3aa/0xaf35132e63032d0d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1fdc expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.166481:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604858368:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.166495:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243062300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604858496:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.166530:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12f00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.166532:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417f00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.167258:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604858368:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.167278:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bf17d487/0xaf35132e63032d06 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fe3 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.167283:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243062300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604858496:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.167314:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bf17d487/0xaf35132e63032d06 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fe3 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.167317:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005d10d3aa/0xaf35132e63032d0d lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fdc expref: 20 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.167327:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37400, rc 0) 00010000:00010000:3.0:1780298642.167341:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005d10d3aa/0xaf35132e63032d0d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fdc expref: 20 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.167352:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baec00, rc 0) 00000100:00100000:1.0:1780298642.167360:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243808380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609346816:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 939us (1046us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.167367:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8750 00000100:00100000:3.0:1780298642.167380:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24380a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2400:x1866777609346688:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 947us (1023us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.167388:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8751 00000100:00100000:3.0:1780298642.171859:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.171910:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609346944 00000100:00100000:2.0:1780298642.171917:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609346944 00002000:00100000:2.0:1780298642.171946:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (0->1802239) req@ffff9cf24380b100 x1866777609346944/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.171954:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24380b100 with x1866777609346944 ext(0->1802239) 00002000:00010000:2.0:1780298642.171958:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24380b100 00000100:00100000:2.0:1780298642.171976:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5930 00000100:00100000:2.0:1780298642.171983:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24380b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+27:2400:x1866777609346944:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298642.177157:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24380b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+27:2400:x1866777609346944:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 5165us (5293us total) trans 0 rc 1802240/1802240 00000100:00100000:2.0:1780298642.177167:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5930 00002000:00100000:2.0:1780298642.177178:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (0->1802239) req@ffff9cf24380b100 x1866777609346944/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/440 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 1802240/1802240 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.177186:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24380b100 with x1866777609346944 ext(0->1802239) 00002000:00010000:2.0:1780298642.177191:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24380b100 00000100:00100000:0.0:1780298642.178583:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.178606:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.178629:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609347072 00000100:00100000:3.0:1780298642.178637:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609347072 00000100:00100000:1.0:1780298642.178647:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609347200 00000100:00100000:1.0:1780298642.178653:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609347200 00000100:00100000:3.0:1780298642.178695:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8752 00000100:00100000:1.0:1780298642.178714:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8753 00000100:00100000:3.0:1780298642.178716:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243062300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2400:x1866777609347072:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298642.178731:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242860000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609347200:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.178735:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.178743:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.178757:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a10929f0/0xaf35132e63032d14 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1ff1 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.178789:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ad6f98a8/0xaf35132e63032d1b lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1fea expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.178799:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604858624:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.178822:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fd180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604858752:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.178862:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.178862:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12f00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.179683:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604858624:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.179707:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a10929f0/0xaf35132e63032d14 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ff1 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.179726:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a10929f0/0xaf35132e63032d14 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ff1 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.179736:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37200, rc 0) 00000100:00100000:2.0:1780298642.179742:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fd180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604858752:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.179758:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ad6f98a8/0xaf35132e63032d1b lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fea expref: 20 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.179767:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242860000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609347200:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1044us (1160us total) trans 0 rc 301/301 00010000:00010000:2.0:1780298642.179772:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ad6f98a8/0xaf35132e63032d1b lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fea expref: 20 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.179776:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8753 00010000:00010000:2.0:1780298642.179790:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf2445be800, rc 0) 00000100:00100000:2.0:1780298642.179816:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243062300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2400:x1866777609347072:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1113us (1237us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.179821:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8752 00000100:00100000:2.0:1780298642.180793:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.180826:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609347328 00000100:00100000:1.0:1780298642.180830:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609347328 00000100:00100000:1.0:1780298642.180859:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8754 00000100:00100000:1.0:1780298642.180868:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2399:x1866777609347328:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.180876:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.180884:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bf17d487/0xaf35132e63032d22 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1ff8 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.180908:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.180912:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604858880:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.180959:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12f00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.180984:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609347456 00000100:00100000:3.0:1780298642.180991:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609347456 00000100:00100000:0.0:1780298642.181037:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8755 00000100:00100000:0.0:1780298642.181048:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609347456:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.181059:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.181071:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008361787d/0xaf35132e63032d29 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d1fff expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.181153:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604859008:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.181215:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417c00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.181604:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604858880:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.181619:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bf17d487/0xaf35132e63032d22 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ff8 expref: 20 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.181633:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bf17d487/0xaf35132e63032d22 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1ff8 expref: 20 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.181643:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37400, rc 0) 00000100:00100000:1.0:1780298642.181678:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2399:x1866777609347328:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 813us (885us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.181685:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8754 00000100:00100000:0.0:1780298642.181805:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604859008:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.181829:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008361787d/0xaf35132e63032d29 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fff expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.181847:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008361787d/0xaf35132e63032d29 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fff expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.181858:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2800, rc 0) 00000100:00100000:0.0:1780298642.181889:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609347456:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 844us (982us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.181897:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8755 00000100:00100000:0.0:1780298642.182692:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.182723:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609347584 00000100:00100000:2.0:1780298642.182728:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609347584 00000100:00100000:2.0:1780298642.182753:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8756 00000100:00100000:2.0:1780298642.182758:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:29360:x1866777609347584:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.182766:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.182775:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000408198ca/0xaf35132e63032d30 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2006 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.182789:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63032cce lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fb2 expref: 26 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.182795:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(92 177) 00010000:00010000:2.0:1780298642.182799:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(93 177) 00010000:00010000:2.0:1780298642.182802:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000408198ca/0xaf35132e63032d30 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1691648->1699839] (req 1691648->1699839) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2006 expref: 20 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.182817:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63032cce lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fb2 expref: 26 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.182826:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63032cce lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fb2 expref: 26 pid: 20252 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.182833:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604859136:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298642.182863:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.183495:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604859136:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298642.183508:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000408198ca/0xaf35132e63032d30 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1691648->1699839] (req 1691648->1699839) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2006 expref: 20 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.183521:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf2445bec00, rc 0) 00000100:00100000:2.0:1780298642.183563:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:29360:x1866777609347584:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 803us (870us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.183570:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8756 00000100:00100000:0.0:1780298642.184864:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.184917:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609347712 00000100:00100000:1.0:1780298642.184925:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609347712 00010000:00010000:1.0:1780298642.184950:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63032cce lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fb2 expref: 27 pid: 20252 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.184985:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14959 00000100:00100000:1.0:1780298642.184994:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fa300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+27:2400:x1866777609347712:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298642.185006:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298642.185019:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63032cce lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fb2 expref: 27 pid: 20252 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.185031:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63032cce lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fb2 expref: 27 pid: 20252 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.185052:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63032cce lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fb2 expref: 27 pid: 20252 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.185065:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(93 177) 00010000:00010000:1.0:1780298642.185069:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(93 177) 00010000:00010000:1.0:1780298642.185076:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000408198ca/0xaf35132e63032d30 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1691648->35246079] (req 1691648->1699839) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2006 expref: 19 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.185086:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000408198ca/0xaf35132e63032d30 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1691648->35246079] (req 1691648->1699839) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2006 expref: 19 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.185112:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000408198ca/0xaf35132e63032d30 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1691648->35246079] (req 1691648->1699839) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2006 expref: 19 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.185157:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777604859264:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298642.185211:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12f00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.185889:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777604859264:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298642.185906:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298642.185944:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fa300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+26:2400:x1866777609347712:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 951us (1078us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.185955:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.185970:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14959 00000100:00100000:2.0:1780298642.186008:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609347840 00000100:00100000:2.0:1780298642.186013:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609347840 00000100:00100000:2.0:1780298642.186047:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5931 00000100:00100000:2.0:1780298642.186054:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2397:x1866777609347840:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298642.186228:0:19928:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 1798145 -> 1798145 00002000:00010000:2.0:1780298642.186231:0:19928:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.186233:0:19928:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.186234:0:19928:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.186236:0:19928:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 384 -> 400 00000100:00100000:2.0:1780298642.186267:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2397:x1866777609347840:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 213us (317us total) trans 4294969610 rc 0/0 00000100:00100000:2.0:1780298642.186275:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5931 00000100:00100000:3.0:1780298642.187019:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.187054:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609347968 00000100:00100000:2.0:1780298642.187058:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609347968 00000100:00100000:2.0:1780298642.187078:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8757 00000100:00100000:2.0:1780298642.187082:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243808a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+26:2398:x1866777609347968:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.187089:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.187096:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ad6f98a8/0xaf35132e63032d37 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d200d expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.187121:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604859392:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.187174:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.187745:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604859392:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.187759:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ad6f98a8/0xaf35132e63032d37 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d200d expref: 27 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.187769:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ad6f98a8/0xaf35132e63032d37 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d200d expref: 27 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.187776:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf2445be800, rc 0) 00000100:00100000:2.0:1780298642.187798:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243808a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+26:2398:x1866777609347968:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 716us (781us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.187803:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8757 00000100:00100000:0.0:1780298642.188544:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.188579:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609348096 00000100:00100000:3.0:1780298642.188583:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609348096 00000100:00100000:3.0:1780298642.188608:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8758 00000100:00100000:3.0:1780298642.188613:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+26:2397:x1866777609348096:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.188619:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.188625:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e63032d3e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2014 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.188645:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24380ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604859520:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.188680:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348ce1840 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.189110:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24380ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604859520:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.189141:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e63032d3e lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2014 expref: 27 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.189152:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e63032d3e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2014 expref: 27 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.189160:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2e00, rc 0) 00000100:00100000:0.0:1780298642.189184:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+26:2397:x1866777609348096:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 571us (641us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.189189:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8758 00000100:00100000:0.0:1780298642.189960:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.189994:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609348224 00000100:00100000:2.0:1780298642.189999:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609348224 00000100:00100000:2.0:1780298642.190027:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5932 00000100:00100000:2.0:1780298642.190033:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2398:x1866777609348224:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298642.190089:0:8363:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 1798145 -> 1798145 00002000:00010000:2.0:1780298642.190092:0:8363:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.190093:0:8363:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.190095:0:8363:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.190096:0:8363:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 400 -> 400 00000100:00100000:2.0:1780298642.190120:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:2398:x1866777609348224:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 88us (160us total) trans 4294969611 rc 0/0 00000100:00100000:2.0:1780298642.190143:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5932 00000100:00100000:2.0:1780298642.190797:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.190827:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609348352 00000100:00100000:0.0:1780298642.190830:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609348352 00000100:00100000:0.0:1780298642.190854:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8759 00000100:00100000:0.0:1780298642.190859:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+26:2397:x1866777609348352:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.190865:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.190871:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008361787d/0xaf35132e63032d45 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d201b expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.190895:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604859648:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.190925:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417c00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.191509:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604859648:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.191522:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008361787d/0xaf35132e63032d45 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d201b expref: 27 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.191531:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008361787d/0xaf35132e63032d45 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d201b expref: 27 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.191537:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2800, rc 0) 00000100:00100000:0.0:1780298642.191558:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+26:2397:x1866777609348352:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 700us (762us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.191563:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8759 00000100:00100000:3.0:1780298642.192185:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.192209:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609348480 00000100:00100000:2.0:1780298642.192213:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609348480 00000100:00100000:2.0:1780298642.192234:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8760 00000100:00100000:2.0:1780298642.192239:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24380a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+26:2398:x1866777609348480:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.192246:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.192254:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002de18a35/0xaf35132e63032d4c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2022 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.192283:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604859776:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.192328:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4a80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.192895:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604859776:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.192910:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002de18a35/0xaf35132e63032d4c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2022 expref: 27 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.192924:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002de18a35/0xaf35132e63032d4c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2022 expref: 27 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.192948:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf2445bfa00, rc 0) 00000100:00100000:3.0:1780298642.192976:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24380a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+26:2398:x1866777609348480:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 736us (792us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.192983:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8760 00000100:00100000:3.0:1780298642.194665:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.194699:0:9976:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609348736 00000100:00100000:2.0:1780298642.194704:0:9976:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609348736 00000100:00100000:2.0:1780298642.194745:0:9976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61920 00000100:00100000:2.0:1780298642.194751:0:9976:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24380bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+93:29360:x1866777609348736:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:2.0:1780298642.194786:0:9976:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298642.194801:0:9976:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be600/0xaf35132e63032cf1 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 52 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fc7 expref: 93 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.194811:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.194814:0:9976:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298642.194937:0:9976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be800/0xaf35132e63032d53 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 51 type: IBT flags: 0x40000000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.194951:0:9976:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693400/0xaf35132e63032ce3 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 51 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fc0 expref: 87 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.194967:0:9976:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be800/0xaf35132e63032d53 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 51 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.194992:0:9976:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693400/0xaf35132e63032ce3 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 51 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fc0 expref: 87 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.195005:0:9976:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693400/0xaf35132e63032ce3 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 51 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fc0 expref: 87 pid: 7411 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.195014:0:9976:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fce00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:9976:1866777604859904:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298642.195051:0:9976:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.195667:0:9976:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fce00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:9976:1866777604859904:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298642.195677:0:9976:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be800/0xaf35132e63032d53 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 51 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.195809:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.195841:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609348864 00000100:00100000:0.0:1780298642.195845:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609348864 00010000:00010000:0.0:1780298642.195861:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693400/0xaf35132e63032ce3 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 51 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fc0 expref: 88 pid: 7411 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.195882:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14960 00000100:00100000:0.0:1780298642.195887:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243808000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+88:2399:x1866777609348864:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298642.195894:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298642.195898:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693400/0xaf35132e63032ce3 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 52 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fc0 expref: 88 pid: 7411 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.195910:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693400/0xaf35132e63032ce3 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 52 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fc0 expref: 88 pid: 7411 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.195921:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693400/0xaf35132e63032ce3 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 52 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fc0 expref: 88 pid: 7411 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.195929:0:6462:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298642.195932:0:6462:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be800/0xaf35132e63032d53 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 51 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.195940:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be800/0xaf35132e63032d53 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 51 type: IBT flags: 0x40290000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.195945:0:6462:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be800/0xaf35132e63032d53 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 51 type: IBT flags: 0x40290000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.195958:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298642.195976:0:9976:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be800/0xaf35132e63032d53 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 50 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.195983:0:9976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be800/0xaf35132e63032d53 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 50 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.195989:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243808000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+87:2399:x1866777609348864:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 98us (178us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.195997:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14960 00000001:00080000:2.0:1780298642.196029:0:9976:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991002 00000004:00010000:2.0:1780298642.196055:0:9976:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 0000000093641547 reply state 0000000039f8082d transno 4294991003 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be800/0xaf35132e63032d53 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 50 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.196068:0:9976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be800/0xaf35132e63032d53 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 50 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.196072:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.196074:0:9976:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1780298642.196076:0:9976:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1780298642.196142:0:9976:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24380bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+95:29360:x1866777609348736:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1373us (1460us total) trans 4294991003 rc 0/0 00000100:00100000:2.0:1780298642.196148:0:9976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61920 00000020:00080000:2.0:1780298642.196152:0:9976:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1780298642.196658:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.196685:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609348992 00000100:00100000:2.0:1780298642.196689:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609348992 00000100:00100000:2.0:1780298642.196715:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8761 00000100:00100000:2.0:1780298642.196719:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243809880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:29360:x1866777609348992:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.196725:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.196732:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000000d6c33e/0xaf35132e63032d5a lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2029 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.196746:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000408198ca/0xaf35132e63032d30 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1691648->35246079] (req 1691648->1699839) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2006 expref: 23 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.196752:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(93 177) 00010000:00010000:2.0:1780298642.196755:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(94 177) 00010000:00010000:2.0:1780298642.196758:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000000d6c33e/0xaf35132e63032d5a lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2029 expref: 23 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.196769:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000408198ca/0xaf35132e63032d30 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1691648->35246079] (req 1691648->1699839) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2006 expref: 23 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.196775:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000408198ca/0xaf35132e63032d30 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1691648->35246079] (req 1691648->1699839) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2006 expref: 23 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.196782:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604860032:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298642.196810:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.197258:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604860032:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298642.197279:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000000d6c33e/0xaf35132e63032d5a lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2029 expref: 23 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.197286:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf2445bfe00, rc 0) 00000100:00100000:2.0:1780298642.197318:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243809880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:29360:x1866777609348992:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 599us (661us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.197323:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8761 00000100:00100000:1.0:1780298642.197493:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.197531:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609349120 00000100:00100000:0.0:1780298642.197535:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609349120 00010000:00010000:0.0:1780298642.197546:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000408198ca/0xaf35132e63032d30 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1691648->35246079] (req 1691648->1699839) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2006 expref: 23 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.197564:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14961 00000100:00100000:0.0:1780298642.197568:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2400:x1866777609349120:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298642.197576:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298642.197584:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000408198ca/0xaf35132e63032d30 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1691648->35246079] (req 1691648->1699839) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2006 expref: 23 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.197592:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000408198ca/0xaf35132e63032d30 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1691648->35246079] (req 1691648->1699839) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2006 expref: 23 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.197602:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000408198ca/0xaf35132e63032d30 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1691648->35246079] (req 1691648->1699839) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2006 expref: 23 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.197609:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(94 177) 00010000:00010000:0.0:1780298642.197612:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(94 177) 00010000:00010000:0.0:1780298642.197615:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000000d6c33e/0xaf35132e63032d5a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2029 expref: 22 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.197621:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000000d6c33e/0xaf35132e63032d5a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2029 expref: 22 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.197634:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000000d6c33e/0xaf35132e63032d5a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2029 expref: 22 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.197640:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fb480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777604860160:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298642.197667:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417c00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.198161:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fb480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777604860160:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298642.198174:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298642.198196:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:2400:x1866777609349120:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 627us (705us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.198203:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14961 00000100:00100000:1.0:1780298642.198233:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.198259:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609349248 00000100:00100000:0.0:1780298642.198263:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609349248 00000100:00100000:0.0:1780298642.198282:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8762 00000100:00100000:0.0:1780298642.198287:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609349248:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.198305:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.198313:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bde78cfa/0xaf35132e63032d61 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2030 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.198327:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032cd5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fb9 expref: 7 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.198332:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(69 130) 00010000:00010000:0.0:1780298642.198334:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(70 130) 00010000:00010000:0.0:1780298642.198337:0:20252:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bde78cfa/0xaf35132e63032d61 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2030 expref: 11 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.198348:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032cd5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fb9 expref: 7 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.198356:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032cd5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fb9 expref: 7 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.198363:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604860288:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298642.198391:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417c00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.198776:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604860288:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298642.198786:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bde78cfa/0xaf35132e63032d61 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2030 expref: 11 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.198793:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3a00, rc 0) 00000100:00100000:0.0:1780298642.198813:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:29360:x1866777609349248:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 526us (581us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.198819:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8762 00000100:00100000:3.0:1780298642.198935:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.198962:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609349376 00000100:00100000:2.0:1780298642.198967:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609349376 00010000:00010000:2.0:1780298642.198981:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032cd5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fb9 expref: 8 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.199002:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14962 00000100:00100000:2.0:1780298642.199007:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243809180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609349376:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298642.199015:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298642.199033:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032cd5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fb9 expref: 8 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.199044:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032cd5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fb9 expref: 8 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.199058:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032cd5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d1fb9 expref: 8 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.199069:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(70 130) 00010000:00010000:2.0:1780298642.199073:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(70 130) 00010000:00010000:2.0:1780298642.199078:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000bde78cfa/0xaf35132e63032d61 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2030 expref: 10 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.199088:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bde78cfa/0xaf35132e63032d61 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2030 expref: 10 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.199109:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000bde78cfa/0xaf35132e63032d61 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2030 expref: 10 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.199116:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fce00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:7400:1866777604860416:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298642.199163:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.199649:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fce00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:7400:1866777604860416:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298642.199656:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298642.199675:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243809180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609349376:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 669us (741us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.199680:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14962 00000100:00100000:3.0:1780298642.199784:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.199828:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609349504 00000100:00100000:1.0:1780298642.199832:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609349504 00002000:00010000:1.0:1780298642.199850:0:19949:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00000100:00100000:3.0:1780298642.199856:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00002000:00100000:1.0:1780298642.199858:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24380ad80 with x1866777609349504 ext(4194304->18446744073709551615) 00002000:00010000:1.0:1780298642.199866:0:19949:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24380ad80. 00000100:00100000:2.0:1780298642.199886:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609349632 00000100:00100000:2.0:1780298642.199890:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609349632 00000100:00100000:1.0:1780298642.199894:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5933 00002000:00010000:2.0:1780298642.199901:0:19950:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (2302427->18446744073709551615) 00000100:00100000:1.0:1780298642.199905:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24380ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:2397:x1866777609349504:12345-192.168.203.25@tcp:10:fsx.0 00002000:00100000:2.0:1780298642.199906:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24304c380 with x1866777609349632 ext(2302427->18446744073709551615) 00002000:00010000:2.0:1780298642.199912:0:19950:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24304c380. 00000100:00100000:2.0:1780298642.199930:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5934 00000100:00100000:2.0:1780298642.199937:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609349632:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:1.0:1780298642.199992:0:19949:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 1798145 -> 4194304 00002000:00010000:1.0:1780298642.199995:0:19949:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:1.0:1780298642.199997:0:19949:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:1.0:1780298642.199999:0:19949:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:1.0:1780298642.200000:0:19949:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 400 -> 400 00000100:00100000:1.0:1780298642.200025:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24380ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2397:x1866777609349504:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 125us (245us total) trans 4294969612 rc 0/0 00000100:00100000:1.0:1780298642.200031:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5933 00002000:00010000:1.0:1780298642.200034:0:19949:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:1.0:1780298642.200036:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24380ad80 with x1866777609349504 ext(4194304->18446744073709551615) 00002000:00010000:1.0:1780298642.200041:0:19949:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24380ad80. 00002000:00010000:2.0:1780298642.200042:0:19950:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 0 -> 2302427 00002000:00010000:2.0:1780298642.200046:0:19950:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.200048:0:19950:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.200050:0:19950:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.200052:0:19950:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:2.0:1780298642.200071:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609349632:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 135us (218us total) trans 4294969172 rc 0/0 00000100:00100000:2.0:1780298642.200075:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5934 00002000:00010000:2.0:1780298642.200078:0:19950:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (2302427->18446744073709551615) 00002000:00100000:2.0:1780298642.200080:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24304c380 with x1866777609349632 ext(2302427->18446744073709551615) 00002000:00010000:2.0:1780298642.200084:0:19950:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24304c380. 00000100:00100000:1.0:1780298642.200973:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.201007:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609349760 00000100:00100000:2.0:1780298642.201011:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609349760 00002000:00100000:2.0:1780298642.201038:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (2297856->2301951) req@ffff9cf255489180 x1866777609349760/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.201044:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255489180 with x1866777609349760 ext(2297856->2301951) 00002000:00010000:2.0:1780298642.201047:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf255489180 00000100:00100000:2.0:1780298642.201061:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5935 00000100:00100000:2.0:1780298642.201066:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255489180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609349760:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298642.201199:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255489180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609349760:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 132us (225us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298642.201205:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5935 00002000:00100000:2.0:1780298642.201210:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (2297856->2301951) req@ffff9cf255489180 x1866777609349760/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/4536 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.201215:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255489180 with x1866777609349760 ext(2297856->2301951) 00002000:00010000:2.0:1780298642.201218:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf255489180 00000100:00100000:3.0:1780298642.202076:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.202104:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609349888 00000100:00100000:2.0:1780298642.202108:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609349888 00002000:00100000:2.0:1780298642.202137:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x12:0x0] (2301952->2306047) req@ffff9cf24304e680 x1866777609349888/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.202144:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24304e680 with x1866777609349888 ext(2301952->2306047) 00002000:00010000:2.0:1780298642.202147:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24304e680 00000100:00100000:2.0:1780298642.202165:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5936 00000100:00100000:2.0:1780298642.202172:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609349888:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298642.202243:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609349888:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 71us (166us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298642.202250:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5936 00002000:00100000:2.0:1780298642.202257:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x12:0x0] (2301952->2306047) req@ffff9cf24304e680 x1866777609349888/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/4536 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.202263:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24304e680 with x1866777609349888 ext(2301952->2306047) 00002000:00010000:2.0:1780298642.202266:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24304e680 00000100:00100000:3.0:1780298642.203100:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.203152:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609350016 00000100:00100000:2.0:1780298642.203157:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609350016 00002000:00100000:2.0:1780298642.203178:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x12:0x0] (2297856->2302426) req@ffff9cf24304f800 x1866777609350016/t0(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 5064/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.203187:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24304f800 with x1866777609350016 ext(2297856->2302426) 00002000:00010000:2.0:1780298642.203190:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24304f800 00000100:00100000:2.0:1780298642.203207:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5937 00000100:00100000:2.0:1780298642.203213:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609350016:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:2.0:1780298642.203577:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2397:x1866777609350016:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 364us (476us total) trans 4294969173 rc 0/0 00000100:00100000:2.0:1780298642.203582:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5937 00002000:00100000:2.0:1780298642.203586:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x12:0x0] (2297856->2302426) req@ffff9cf24304f800 x1866777609350016/t4294969173(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 5064/448 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.203591:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24304f800 with x1866777609350016 ext(2297856->2302426) 00002000:00010000:2.0:1780298642.203593:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24304f800 00000100:00100000:1.0:1780298642.204255:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.204281:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609350144 00000100:00100000:0.0:1780298642.204285:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609350144 00000100:00100000:0.0:1780298642.204315:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8763 00000100:00100000:0.0:1780298642.204320:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255488a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2398:x1866777609350144:12345-192.168.203.25@tcp:16:fsx.0 00000001:00080000:2.0:1780298642.204903:0:8351:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0000: transno 4294969173 is committed 00000001:00080000:2.0:1780298642.204911:0:8351:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0000: transno 4294969171 is committed 00000100:00100000:0.0:1780298642.204965:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255488a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609350144:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 645us (712us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.204970:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8763 00000100:00100000:3.0:1780298642.205627:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.205654:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609350272 00000100:00100000:2.0:1780298642.205657:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609350272 00000100:00100000:2.0:1780298642.205683:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61921 00000100:00100000:2.0:1780298642.205688:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+87:29360:x1866777609350272:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.205695:0:6477:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.205704:0:6477:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfa00/0xaf35132e63032d68 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 51 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2037 expref: -99 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.205749:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf400/0xaf35132e63032d6f lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 52 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.205757:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf400/0xaf35132e63032d6f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 52 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.205763:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf400/0xaf35132e63032d6f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 52 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298642.205776:0:6477:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 6496731 00000004:00200000:2.0:1780298642.205779:0:6477:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298642.205789:0:6477:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf400/0xaf35132e63032d6f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 52 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.205804:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfa00/0xaf35132e63032d68 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 52 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2037 expref: 89 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.205813:0:6477:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf400/0xaf35132e63032d6f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 51 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2037 expref: 88 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.205818:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.205820:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298642.205822:0:6477:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf2445bf400, rc 0) 00000100:00100000:2.0:1780298642.205843:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+88:29360:x1866777609350272:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 155us (217us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.205847:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61921 00000100:00100000:3.0:1780298642.206610:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.206641:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609350400 00000100:00100000:2.0:1780298642.206646:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609350400 00000100:00100000:2.0:1780298642.206675:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8764 00000100:00100000:2.0:1780298642.206682:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+26:2398:x1866777609350400:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.206690:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.206698:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fedd83c9/0xaf35132e63032d76 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d203e expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.206727:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604860544:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.206763:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.206813:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.206842:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609350528 00000100:00100000:2.0:1780298642.206846:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609350528 00000100:00100000:2.0:1780298642.206870:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8765 00000100:00100000:2.0:1780298642.206875:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609350528:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.206881:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.206887:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000a323d4b/0xaf35132e63032d7d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2045 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.206905:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604860672:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.206926:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.207350:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604860544:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.207362:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000fedd83c9/0xaf35132e63032d76 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d203e expref: 27 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.207370:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fedd83c9/0xaf35132e63032d76 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d203e expref: 27 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.207376:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf2445be000, rc 0) 00000100:00100000:2.0:1780298642.207396:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+26:2398:x1866777609350400:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 716us (786us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.207400:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8764 00002000:00010000:2.0:1780298642.207506:0:9128:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks: 0 -> 16 00000100:00100000:2.0:1780298642.207515:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604860672:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.207543:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000a323d4b/0xaf35132e63032d7d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2045 expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.207556:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000a323d4b/0xaf35132e63032d7d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2045 expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.207572:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf2445bf000, rc 0) 00000100:00100000:2.0:1780298642.207596:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609350528:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 721us (784us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.207600:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8765 00000100:00100000:3.0:1780298642.208253:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.208281:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609350656 00000100:00100000:0.0:1780298642.208285:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609350656 00000100:00100000:0.0:1780298642.208314:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8766 00000100:00100000:0.0:1780298642.208320:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+26:2397:x1866777609350656:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.208325:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.208331:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008361787d/0xaf35132e63032d84 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d204c expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.208352:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604860800:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.208389:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417c00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.208467:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.208498:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609350784 00000100:00100000:0.0:1780298642.208502:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609350784 00000100:00100000:0.0:1780298642.208522:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8767 00000100:00100000:0.0:1780298642.208527:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609350784:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.208532:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.208538:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a269a887/0xaf35132e63032d8b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2053 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.208556:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604860928:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.208578:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.208897:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604860800:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.208909:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008361787d/0xaf35132e63032d84 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d204c expref: 27 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.208917:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008361787d/0xaf35132e63032d84 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d204c expref: 27 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.208923:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2800, rc 0) 00000100:00100000:0.0:1780298642.208943:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+26:2397:x1866777609350656:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 625us (691us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.208948:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8766 00000100:00100000:0.0:1780298642.209106:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604860928:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.209116:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a269a887/0xaf35132e63032d8b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2053 expref: 7 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.209146:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a269a887/0xaf35132e63032d8b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2053 expref: 7 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.209153:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3e00, rc 0) 00000100:00100000:0.0:1780298642.209173:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609350784:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 646us (707us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.209177:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8767 00000100:00100000:1.0:1780298642.209891:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.209911:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.209923:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609350912 00000100:00100000:0.0:1780298642.209927:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609350912 00000100:00100000:2.0:1780298642.209941:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609351040 00000100:00100000:2.0:1780298642.209945:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609351040 00000100:00100000:0.0:1780298642.209955:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8768 00000100:00100000:0.0:1780298642.209962:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25488a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+26:2398:x1866777609350912:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.209967:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.209974:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e63032d92 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d205a expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.209977:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8769 00000100:00100000:1.0:1780298642.209989:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25488b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609351040:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.209996:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298642.210000:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604861056:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.210004:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003c8dab73/0xaf35132e63032d99 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2061 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.210031:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f746300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604861184:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.210037:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.210058:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12f00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.210646:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604861056:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.210660:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e63032d92 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d205a expref: 27 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.210670:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e63032d92 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d205a expref: 27 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.210680:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2e00, rc 0) 00000100:00100000:0.0:1780298642.210710:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25488a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+26:2398:x1866777609350912:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 750us (819us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.210717:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8768 00000100:00100000:2.0:1780298642.210731:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f746300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604861184:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.210750:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003c8dab73/0xaf35132e63032d99 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2061 expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.210759:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003c8dab73/0xaf35132e63032d99 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2061 expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.210765:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36800, rc 0) 00000100:00100000:2.0:1780298642.210790:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25488b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609351040:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 803us (877us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.210798:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8769 00010000:00010000:0.0:1780298642.211221:0:6464:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1780298642.211526:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.211568:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609351168 00000100:00100000:2.0:1780298642.211573:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609351168 00000100:00100000:2.0:1780298642.211606:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8770 00000100:00100000:2.0:1780298642.211614:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f746a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+26:29360:x1866777609351168:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.211621:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.211629:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000408198ca/0xaf35132e63032da0 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2068 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.211651:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000000d6c33e/0xaf35132e63032d5a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2029 expref: 22 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.211660:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(94 177) 00010000:00010000:2.0:1780298642.211664:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(95 177) 00010000:00010000:2.0:1780298642.211668:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000408198ca/0xaf35132e63032da0 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1941504->1949695] (req 1941504->1949695) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2068 expref: 27 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.211689:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000000d6c33e/0xaf35132e63032d5a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2029 expref: 22 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.211701:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000000d6c33e/0xaf35132e63032d5a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2029 expref: 22 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.211712:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604861312:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298642.211751:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.212271:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604861312:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298642.212287:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000408198ca/0xaf35132e63032da0 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1941504->1949695] (req 1941504->1949695) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2068 expref: 27 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.212311:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf2445bec00, rc 0) 00000100:00100000:2.0:1780298642.212339:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f746a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+27:29360:x1866777609351168:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 726us (812us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.212349:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8770 00000100:00100000:1.0:1780298642.212546:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.212578:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609351296 00000100:00100000:2.0:1780298642.212583:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609351296 00010000:00010000:2.0:1780298642.212601:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000000d6c33e/0xaf35132e63032d5a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2029 expref: 23 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.212626:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14963 00000100:00100000:2.0:1780298642.212634:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f746680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2400:x1866777609351296:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298642.212642:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298642.212649:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000000d6c33e/0xaf35132e63032d5a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2029 expref: 23 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.212656:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000000d6c33e/0xaf35132e63032d5a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2029 expref: 23 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.212666:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000000d6c33e/0xaf35132e63032d5a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2029 expref: 23 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.212673:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(95 177) 00010000:00010000:2.0:1780298642.212675:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(95 177) 00010000:00010000:2.0:1780298642.212681:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000408198ca/0xaf35132e63032da0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1941504->35495935] (req 1941504->1949695) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2068 expref: 26 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.212686:0:6463:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000408198ca/0xaf35132e63032da0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1941504->35495935] (req 1941504->1949695) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2068 expref: 26 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.212703:0:6463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000408198ca/0xaf35132e63032da0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1941504->35495935] (req 1941504->1949695) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2068 expref: 26 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.212710:0:6463:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fed80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6463:1866777604861440:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298642.212736:0:6463:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.213370:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.213398:0:6463:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fed80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6463:1866777604861440:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298642.213405:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609351424 00000100:00100000:0.0:1780298642.213409:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609351424 00010000:00010000:2.0:1780298642.213411:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298642.213440:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5938 00000100:00100000:2.0:1780298642.213446:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f746680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:2400:x1866777609351296:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 812us (899us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.213452:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f744700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+27:2397:x1866777609351424:12345-192.168.203.25@tcp:22:fsx.0 00000100:00100000:2.0:1780298642.213459:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14963 00002000:00010000:0.0:1780298642.213605:0:19951:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:0.0:1780298642.213609:0:19951:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298642.213611:0:19951:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298642.213613:0:19951:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298642.213615:0:19951:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 400 -> 416 00000100:00100000:0.0:1780298642.213644:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f744700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+28:2397:x1866777609351424:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 197us (273us total) trans 4294969613 rc 0/0 00000100:00100000:0.0:1780298642.213653:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5938 00000100:00100000:1.0:1780298642.214352:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.214381:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609351552 00000100:00100000:0.0:1780298642.214385:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609351552 00000100:00100000:0.0:1780298642.214407:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8771 00000100:00100000:0.0:1780298642.214416:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f747100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+28:2397:x1866777609351552:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.214421:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.214428:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000658b0d9c/0xaf35132e63032da7 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d206f expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.214454:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604861568:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.214461:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.214496:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609351680 00000100:00100000:0.0:1780298642.214499:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.214500:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609351680 00000100:00100000:1.0:1780298642.214528:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8772 00000100:00100000:1.0:1780298642.214534:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609351680:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.214540:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.214547:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032dae lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2076 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.214568:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f745c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604861696:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.214595:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b129c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.215036:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604861568:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.215048:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000658b0d9c/0xaf35132e63032da7 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d206f expref: 29 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.215057:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000658b0d9c/0xaf35132e63032da7 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d206f expref: 29 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.215064:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2c00, rc 0) 00000100:00100000:0.0:1780298642.215086:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f747100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+28:2397:x1866777609351552:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 674us (735us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.215091:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8771 00000100:00100000:0.0:1780298642.215195:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f745c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604861696:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.215212:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032dae lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2076 expref: 7 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.215222:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032dae lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2076 expref: 7 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.215229:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37000, rc 0) 00000100:00100000:0.0:1780298642.215249:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609351680:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 716us (793us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.215257:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8772 00000100:00100000:3.0:1780298642.215976:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.216017:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609351808 00000100:00100000:2.0:1780298642.216020:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609351808 00000100:00100000:3.0:1780298642.216025:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.216049:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609351936 00000100:00100000:2.0:1780298642.216052:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609351936 00000100:00100000:2.0:1780298642.216065:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8773 00000100:00100000:2.0:1780298642.216071:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e60fc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+28:2398:x1866777609351808:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298642.216075:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8774 00010000:00010000:2.0:1780298642.216079:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.216086:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000019a622b7/0xaf35132e63032db5 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d207d expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.216086:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609351936:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.216096:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.216105:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e9b43ec/0xaf35132e63032dbc lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2084 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.216111:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604861824:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.216153:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604861952:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.216161:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.216180:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.216776:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604861824:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.216781:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604861952:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.216796:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000019a622b7/0xaf35132e63032db5 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d207d expref: 29 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.216802:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004e9b43ec/0xaf35132e63032dbc lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2084 expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.216829:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000019a622b7/0xaf35132e63032db5 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d207d expref: 29 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.216834:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e9b43ec/0xaf35132e63032dbc lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2084 expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.216841:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf2445be600, rc 0) 00010000:00010000:0.0:1780298642.216851:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2a00, rc 0) 00000100:00100000:1.0:1780298642.216868:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e60fc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+28:2398:x1866777609351808:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 796us (893us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.216873:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609351936:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 789us (850us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.216879:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8773 00000100:00100000:0.0:1780298642.216890:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8774 00000100:00100000:1.0:1780298642.217647:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.217676:0:7411:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609352064 00000100:00100000:2.0:1780298642.217680:0:7411:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609352064 00000100:00100000:2.0:1780298642.217705:0:7411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61922 00000100:00100000:2.0:1780298642.217709:0:7411:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f746a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+95:29360:x1866777609352064:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.217719:0:7411:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.217727:0:7411:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693400/0xaf35132e63032dc3 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 52 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d208b expref: -99 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.217778:0:7411:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692400/0xaf35132e63032dca lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 53 type: IBT flags: 0x40000000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.217786:0:7411:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692400/0xaf35132e63032dca lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 53 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.217793:0:7411:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692400/0xaf35132e63032dca lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 53 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298642.217807:0:7411:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 6496731 00000004:00200000:2.0:1780298642.217809:0:7411:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298642.217819:0:7411:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692400/0xaf35132e63032dca lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 53 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.217833:0:7411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693400/0xaf35132e63032dc3 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 53 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d208b expref: 97 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.217842:0:7411:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692400/0xaf35132e63032dca lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 52 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d208b expref: 96 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.217847:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.217848:0:7411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298642.217850:0:7411:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341692400, rc 0) 00000100:00100000:2.0:1780298642.217871:0:7411:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f746a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+96:29360:x1866777609352064:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 162us (226us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.217875:0:7411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61922 00000100:00100000:3.0:1780298642.218669:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.218701:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609352192 00000100:00100000:0.0:1780298642.218705:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609352192 00000100:00100000:0.0:1780298642.218734:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8775 00000100:00100000:0.0:1780298642.218743:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:2397:x1866777609352192:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.218752:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.218760:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e63032dd1 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2092 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.218784:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604862080:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.218814:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.219400:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604862080:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.219415:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e63032dd1 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2092 expref: 23 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.219426:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e63032dd1 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2092 expref: 23 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.219435:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2e00, rc 0) 00000100:00100000:0.0:1780298642.219462:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:2397:x1866777609352192:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 721us (793us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.219469:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8775 00000100:00100000:1.0:1780298642.220188:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.220212:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609352320 00000100:00100000:2.0:1780298642.220216:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609352320 00000100:00100000:2.0:1780298642.220244:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5939 00000100:00100000:2.0:1780298642.220250:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f747100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609352320:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298642.220384:0:19950:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 2302427 -> 2302427 00002000:00010000:2.0:1780298642.220387:0:19950:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.220388:0:19950:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.220390:0:19950:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.220391:0:19950:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 16 -> 32 00000100:00100000:2.0:1780298642.220415:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f747100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609352320:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 166us (227us total) trans 4294969174 rc 0/0 00000100:00100000:2.0:1780298642.220419:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5939 00000100:00100000:2.0:1780298642.221216:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.221243:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609352448 00000100:00100000:0.0:1780298642.221247:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609352448 00000100:00100000:0.0:1780298642.221268:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8776 00000100:00100000:0.0:1780298642.221275:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:2397:x1866777609352448:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.221283:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.221301:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a269a887/0xaf35132e63032dd8 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2099 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.221332:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604862208:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.221361:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.222002:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604862208:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.222019:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a269a887/0xaf35132e63032dd8 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2099 expref: 23 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.222033:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a269a887/0xaf35132e63032dd8 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2099 expref: 23 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.222043:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3e00, rc 0) 00000100:00100000:0.0:1780298642.222071:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:2397:x1866777609352448:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 796us (855us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.222078:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8776 00000100:00100000:1.0:1780298642.222926:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.222966:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609352576 00000100:00100000:3.0:1780298642.222970:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609352576 00000100:00100000:3.0:1780298642.223001:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8777 00000100:00100000:3.0:1780298642.223014:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f744000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:2398:x1866777609352576:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.223023:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.223037:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63032ddf lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d20a0 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.223061:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604862336:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.223095:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348ce1840 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.223748:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604862336:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.223767:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63032ddf lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d20a0 expref: 23 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.223782:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63032ddf lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d20a0 expref: 23 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.223790:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2600, rc 0) 00000100:00100000:3.0:1780298642.223816:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f744000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:2398:x1866777609352576:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 805us (890us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.223823:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8777 00000100:00100000:2.0:1780298642.224790:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.224826:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609352704 00000100:00100000:1.0:1780298642.224830:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609352704 00000100:00100000:1.0:1780298642.224857:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8778 00000100:00100000:1.0:1780298642.224862:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:2397:x1866777609352704:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.224871:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.224879:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f68a0cb8/0xaf35132e63032de6 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d20a7 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.224905:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f747800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604862464:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.224938:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b129c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.225664:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f747800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604862464:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.225686:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f68a0cb8/0xaf35132e63032de6 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d20a7 expref: 23 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.225703:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f68a0cb8/0xaf35132e63032de6 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d20a7 expref: 23 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.225713:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37800, rc 0) 00000100:00100000:2.0:1780298642.225747:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:2397:x1866777609352704:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 884us (960us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.225756:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8778 00000100:00100000:2.0:1780298642.226860:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.226898:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609352832 00000100:00100000:0.0:1780298642.226903:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609352832 00000100:00100000:0.0:1780298642.226933:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8779 00000100:00100000:0.0:1780298642.226939:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:29360:x1866777609352832:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.226947:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.226955:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e63032ded lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d20ae expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.226971:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000408198ca/0xaf35132e63032da0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1941504->35495935] (req 1941504->1949695) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2068 expref: 27 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.226981:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(95 177) 00010000:00010000:0.0:1780298642.226984:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(96 177) 00010000:00010000:0.0:1780298642.226987:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e63032ded lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2113536->2117631] (req 2113536->2117631) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d20ae expref: 23 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.227002:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000408198ca/0xaf35132e63032da0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1941504->35495935] (req 1941504->1949695) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2068 expref: 27 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.227012:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000408198ca/0xaf35132e63032da0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1941504->35495935] (req 1941504->1949695) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2068 expref: 27 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.227020:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604862592:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298642.227058:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.227779:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604862592:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298642.227798:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e63032ded lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2113536->2117631] (req 2113536->2117631) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d20ae expref: 23 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.227807:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2a00, rc 0) 00000100:00100000:0.0:1780298642.227840:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:29360:x1866777609352832:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 900us (983us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.227847:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8779 00000100:00100000:1.0:1780298642.227995:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.228019:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609352960 00000100:00100000:3.0:1780298642.228024:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609352960 00010000:00010000:3.0:1780298642.228046:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000408198ca/0xaf35132e63032da0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1941504->35495935] (req 1941504->1949695) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2068 expref: 28 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.228075:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14964 00000100:00100000:3.0:1780298642.228082:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+28:2399:x1866777609352960:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298642.228092:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298642.228107:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000408198ca/0xaf35132e63032da0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1941504->35495935] (req 1941504->1949695) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2068 expref: 28 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.228154:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000408198ca/0xaf35132e63032da0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1941504->35495935] (req 1941504->1949695) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2068 expref: 28 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.228175:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000408198ca/0xaf35132e63032da0 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1941504->35495935] (req 1941504->1949695) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2068 expref: 28 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.228188:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(96 177) 00010000:00010000:3.0:1780298642.228191:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(96 177) 00010000:00010000:3.0:1780298642.228199:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e63032ded lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2113536->2117631) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d20ae expref: 22 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.228209:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e63032ded lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2113536->2117631) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d20ae expref: 22 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.228269:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e63032ded lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2113536->2117631) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d20ae expref: 22 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.228283:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fd880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777604862720:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298642.228336:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.229068:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fd880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777604862720:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298642.229083:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298642.229117:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+27:2399:x1866777609352960:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1031us (1117us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.229160:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14964 00000100:00100000:2.0:1780298642.229312:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.229339:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609353088 00000100:00100000:0.0:1780298642.229343:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609353088 00002000:00100000:0.0:1780298642.229365:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (2113536->2117631) req@ffff9cf3481fd880 x1866777609353088/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.229372:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fd880 with x1866777609353088 ext(2113536->2117631) 00002000:00010000:0.0:1780298642.229375:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481fd880 00000100:00100000:0.0:1780298642.229392:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5940 00000100:00100000:0.0:1780298642.229397:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2398:x1866777609353088:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298642.229534:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2398:x1866777609353088:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 135us (221us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298642.229541:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5940 00002000:00100000:0.0:1780298642.229547:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (2113536->2117631) req@ffff9cf3481fd880 x1866777609353088/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/4536 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.229556:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fd880 with x1866777609353088 ext(2113536->2117631) 00002000:00010000:0.0:1780298642.229559:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481fd880 00000100:00100000:0.0:1780298642.230510:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.230543:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609353216 00000100:00100000:2.0:1780298642.230547:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609353216 00002000:00100000:2.0:1780298642.230566:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (2117632->2121727) req@ffff9cf3481f9880 x1866777609353216/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.230572:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481f9880 with x1866777609353216 ext(2117632->2121727) 00002000:00010000:2.0:1780298642.230574:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481f9880 00000100:00100000:2.0:1780298642.230590:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5941 00000100:00100000:2.0:1780298642.230595:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2398:x1866777609353216:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298642.230672:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2398:x1866777609353216:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 77us (163us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298642.230676:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5941 00002000:00100000:2.0:1780298642.230680:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (2117632->2121727) req@ffff9cf3481f9880 x1866777609353216/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/4536 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.230686:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481f9880 with x1866777609353216 ext(2117632->2121727) 00002000:00010000:2.0:1780298642.230688:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481f9880 00000100:00100000:2.0:1780298642.231596:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.231615:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.231642:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609353344 00000100:00100000:0.0:1780298642.231647:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609353344 00000100:00100000:3.0:1780298642.231658:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609353472 00000100:00100000:3.0:1780298642.231665:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609353472 00000100:00100000:0.0:1780298642.231683:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8780 00000100:00100000:0.0:1780298642.231694:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+27:2397:x1866777609353344:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298642.231696:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8781 00010000:00010000:0.0:1780298642.231707:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298642.231708:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609353472:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.231716:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.231722:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e63032df4 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d20b5 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.231725:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000036bac5fe/0xaf35132e63032dfb lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d20bc expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.231755:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(96 177) 00010000:00010000:0.0:1780298642.231759:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(96 177) 00010000:00010000:0.0:1780298642.231767:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e63032df4 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d20b5 expref: 28 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.231774:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604862848:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.231794:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e63032df4 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d20b5 expref: 28 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.231804:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2e00, rc 0) 00000100:00100000:2.0:1780298642.231821:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.231830:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+28:2397:x1866777609353344:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 139us (233us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.231845:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8780 00000100:00100000:2.0:1780298642.232553:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604862848:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.232577:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000036bac5fe/0xaf35132e63032dfb lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d20bc expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.232592:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000036bac5fe/0xaf35132e63032dfb lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d20bc expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.232604:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341693400, rc 0) 00000100:00100000:2.0:1780298642.232637:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609353472:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 929us (1016us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.232644:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8781 00000100:00100000:1.0:1780298642.233520:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.233555:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609353600 00000100:00100000:2.0:1780298642.233560:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609353600 00000100:00100000:2.0:1780298642.233596:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8782 00000100:00100000:2.0:1780298642.233605:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609353600:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.233616:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.233628:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e06fa3fe/0xaf35132e63032e02 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d20c3 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.233662:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604862976:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.233705:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.234438:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604862976:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.234459:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e06fa3fe/0xaf35132e63032e02 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d20c3 expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.234488:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e06fa3fe/0xaf35132e63032e02 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d20c3 expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.234497:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341693e00, rc 0) 00000100:00100000:2.0:1780298642.234523:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609353600:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 921us (1004us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.234528:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8782 00000100:00100000:0.0:1780298642.235952:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.235985:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609353728 00000100:00100000:2.0:1780298642.235991:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609353728 00000100:00100000:2.0:1780298642.236030:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5942 00000100:00100000:2.0:1780298642.236039:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609353728:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298642.236209:0:8363:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 2302427 -> 4115648 00002000:00010000:2.0:1780298642.236212:0:8363:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.236214:0:8363:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.236215:0:8363:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.236217:0:8363:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 32 -> 40 00000100:00100000:2.0:1780298642.236250:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609353728:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 206us (293us total) trans 4294969175 rc 0/0 00000100:00100000:2.0:1780298642.236255:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5942 00000100:00100000:1.0:1780298642.237058:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.237097:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609353856 00000100:00100000:0.0:1780298642.237101:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609353856 00000100:00100000:0.0:1780298642.237139:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8783 00000100:00100000:0.0:1780298642.237146:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609353856:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.237155:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.237168:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c0d26bbc/0xaf35132e63032e09 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d20ca expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.237209:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604863104:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.237252:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417c00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.237910:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604863104:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.237925:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c0d26bbc/0xaf35132e63032e09 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d20ca expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.237937:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c0d26bbc/0xaf35132e63032e09 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d20ca expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.237946:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3c00, rc 0) 00000100:00100000:0.0:1780298642.237966:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609353856:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 822us (911us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.237970:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8783 00000100:00100000:2.0:1780298642.238815:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.238861:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609353984 00000100:00100000:1.0:1780298642.238866:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609353984 00000100:00100000:1.0:1780298642.238901:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8784 00000100:00100000:1.0:1780298642.238910:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609353984:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.238919:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.238928:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e63032e10 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d20d1 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.238958:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f745c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604863232:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.239002:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12f00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.239671:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f745c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604863232:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.239685:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e63032e10 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d20d1 expref: 7 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.239696:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e63032e10 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d20d1 expref: 7 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.239705:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36a00, rc 0) 00000100:00100000:3.0:1780298642.239748:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609353984:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 840us (932us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.239755:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8784 00000100:00100000:2.0:1780298642.240801:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.240836:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609354112 00000100:00100000:0.0:1780298642.240840:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609354112 00002000:00100000:0.0:1780298642.240863:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (167936->176127) req@ffff9cf3481fdc00 x1866777609354112/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.240870:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fdc00 with x1866777609354112 ext(167936->176127) 00002000:00010000:0.0:1780298642.240874:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf3481fdc00 00000100:00100000:0.0:1780298642.240890:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5943 00000100:00100000:0.0:1780298642.240896:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609354112:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298642.241029:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609354112:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 134us (229us total) trans 0 rc 8192/8192 00000100:00100000:0.0:1780298642.241035:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5943 00002000:00100000:0.0:1780298642.241038:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (167936->176127) req@ffff9cf3481fdc00 x1866777609354112/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/8632 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 8192/8192 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.241044:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fdc00 with x1866777609354112 ext(167936->176127) 00002000:00010000:0.0:1780298642.241047:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf3481fdc00 00000100:00100000:2.0:1780298642.242007:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.242055:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609354240 00000100:00100000:3.0:1780298642.242059:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609354240 00000100:00100000:3.0:1780298642.242085:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8785 00000100:00100000:3.0:1780298642.242089:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609354240:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.242096:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.242106:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e63032e17 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d20d8 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.242161:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604863360:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.242204:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348ce1840 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.242836:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604863360:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.242851:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e63032e17 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d20d8 expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.242863:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e63032e17 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d20d8 expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.242873:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2600, rc 0) 00000100:00100000:3.0:1780298642.242895:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609354240:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 806us (890us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.242899:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8785 00000100:00100000:1.0:1780298642.243768:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.243814:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609354368 00000100:00100000:2.0:1780298642.243819:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609354368 00000100:00100000:2.0:1780298642.243851:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8786 00000100:00100000:2.0:1780298642.243858:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609354368:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.243867:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.243876:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002de18a35/0xaf35132e63032e1e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d20df expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.243905:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604863488:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.243945:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.244731:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604863488:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.244753:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002de18a35/0xaf35132e63032e1e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d20df expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.244766:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002de18a35/0xaf35132e63032e1e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d20df expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.244775:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf2445bfa00, rc 0) 00000100:00100000:1.0:1780298642.244800:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609354368:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 943us (1034us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.244807:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8786 00000100:00100000:1.0:1780298642.246062:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.246107:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609354496 00000100:00100000:2.0:1780298642.246112:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609354496 00000100:00100000:2.0:1780298642.246180:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5944 00000100:00100000:2.0:1780298642.246189:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609354496:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298642.246371:0:19950:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4115648 -> 4115648 00002000:00010000:2.0:1780298642.246373:0:19950:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.246375:0:19950:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.246377:0:19950:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.246378:0:19950:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 40 -> 48 00000100:00100000:2.0:1780298642.246405:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609354496:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 219us (346us total) trans 4294969176 rc 0/0 00000100:00100000:2.0:1780298642.246410:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5944 00000100:00100000:2.0:1780298642.247360:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.247395:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609354624 00000100:00100000:0.0:1780298642.247399:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609354624 00000100:00100000:0.0:1780298642.247423:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8787 00000100:00100000:0.0:1780298642.247429:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481ffb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609354624:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.247437:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.247447:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d4afe0b/0xaf35132e63032e25 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d20e6 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.247487:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604863616:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.247527:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.248180:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604863616:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.248200:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001d4afe0b/0xaf35132e63032e25 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d20e6 expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.248212:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d4afe0b/0xaf35132e63032e25 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d20e6 expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.248219:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3000, rc 0) 00000100:00100000:1.0:1780298642.248242:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481ffb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609354624:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 814us (883us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.248247:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8787 00000100:00100000:1.0:1780298642.249073:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.249108:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609354752 00000100:00100000:3.0:1780298642.249142:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609354752 00000100:00100000:3.0:1780298642.249174:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8788 00000100:00100000:3.0:1780298642.249181:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b6a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609354752:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.249189:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.249196:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63032e2c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d20ed expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.249227:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604863744:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.249271:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348ce1840 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.249932:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604863744:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.249945:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63032e2c lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d20ed expref: 7 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.249955:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63032e2c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d20ed expref: 7 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.249963:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2800, rc 0) 00000100:00100000:3.0:1780298642.249984:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b6a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609354752:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 805us (927us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.249989:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8788 00000100:00100000:0.0:1780298642.251226:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.251272:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609354880 00000100:00100000:2.0:1780298642.251278:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609354880 00002000:00100000:2.0:1780298642.251320:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (1966080->1978367) req@ffff9cf3481fad80 x1866777609354880/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.251331:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fad80 with x1866777609354880 ext(1966080->1978367) 00002000:00010000:2.0:1780298642.251336:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481fad80 00000100:00100000:2.0:1780298642.251356:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5945 00000100:00100000:2.0:1780298642.251365:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+28:2398:x1866777609354880:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298642.251546:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+28:2398:x1866777609354880:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 183us (319us total) trans 0 rc 12288/12288 00000100:00100000:2.0:1780298642.251554:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5945 00002000:00100000:2.0:1780298642.251561:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (1966080->1978367) req@ffff9cf3481fad80 x1866777609354880/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/12728 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 12288/12288 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.251569:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fad80 with x1866777609354880 ext(1966080->1978367) 00002000:00010000:2.0:1780298642.251573:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481fad80 00000100:00100000:1.0:1780298642.252772:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.252889:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609355008 00000100:00100000:3.0:1780298642.252895:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609355008 00000100:00100000:3.0:1780298642.252931:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8789 00000100:00100000:3.0:1780298642.252939:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609355008:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.252950:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.252974:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e63032e33 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d20f4 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.253015:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604863872:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.253068:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348ce1840 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.253823:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604863872:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.253846:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e63032e33 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d20f4 expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.253866:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e63032e33 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d20f4 expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.253878:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2600, rc 0) 00000100:00100000:3.0:1780298642.253907:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609355008:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 969us (1136us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.253914:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8789 00000100:00100000:0.0:1780298642.254943:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.254979:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609355136 00000100:00100000:1.0:1780298642.254984:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609355136 00000100:00100000:1.0:1780298642.255011:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8790 00000100:00100000:1.0:1780298642.255016:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609355136:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.255025:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.255034:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003c8dab73/0xaf35132e63032e3a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d20fb expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.255061:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f754380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604864000:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.255097:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12f00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.255864:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f754380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604864000:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.255884:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003c8dab73/0xaf35132e63032e3a lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d20fb expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.255899:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003c8dab73/0xaf35132e63032e3a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d20fb expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.255909:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36800, rc 0) 00000100:00100000:2.0:1780298642.255941:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609355136:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 924us (1000us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.255948:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8790 00000100:00100000:2.0:1780298642.257016:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.257069:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609355264 00000100:00100000:1.0:1780298642.257075:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609355264 00000100:00100000:1.0:1780298642.257111:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8791 00000100:00100000:1.0:1780298642.257121:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481ff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:29360:x1866777609355264:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.257154:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.257164:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032e41 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2102 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.257182:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000bde78cfa/0xaf35132e63032d61 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2030 expref: 12 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.257189:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(70 130) 00010000:00010000:1.0:1780298642.257191:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(71 130) 00010000:00010000:1.0:1780298642.257195:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032e41 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3829760->3837951] (req 3829760->3837951) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2102 expref: 7 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.257212:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000bde78cfa/0xaf35132e63032d61 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2030 expref: 12 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.257222:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000bde78cfa/0xaf35132e63032d61 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2030 expref: 12 pid: 20252 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.257231:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f754380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604864128:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298642.257277:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b129c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.257980:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f754380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604864128:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298642.257995:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032e41 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3829760->3837951] (req 3829760->3837951) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2102 expref: 7 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.258007:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37000, rc 0) 00000100:00100000:1.0:1780298642.258042:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481ff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609355264:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 923us (1027us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.258048:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8791 00000100:00100000:0.0:1780298642.258281:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.258324:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609355392 00000100:00100000:1.0:1780298642.258329:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609355392 00010000:00010000:1.0:1780298642.258349:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000bde78cfa/0xaf35132e63032d61 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2030 expref: 13 pid: 20252 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.258377:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14965 00000100:00100000:1.0:1780298642.258385:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b01450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609355392:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298642.258396:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298642.258411:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000bde78cfa/0xaf35132e63032d61 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2030 expref: 13 pid: 20252 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.258423:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000bde78cfa/0xaf35132e63032d61 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2030 expref: 13 pid: 20252 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.258437:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bde78cfa/0xaf35132e63032d61 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2030 expref: 13 pid: 20252 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.258444:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(71 130) 00010000:00010000:1.0:1780298642.258447:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(71 130) 00010000:00010000:1.0:1780298642.258452:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032e41 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3829760->3837951) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2102 expref: 6 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.258458:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032e41 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3829760->3837951) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2102 expref: 6 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.258474:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032e41 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3829760->3837951) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2102 expref: 6 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.258481:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f754380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:20253:1866777604864256:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298642.258510:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b129c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.259216:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f754380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:20253:1866777604864256:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298642.259224:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298642.259245:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b01450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609355392:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 862us (965us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.259252:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14965 00000100:00100000:0.0:1780298642.259637:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.259680:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609355520 00000100:00100000:2.0:1780298642.259686:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609355520 00002000:00100000:2.0:1780298642.259715:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x12:0x0] (3829760->3837951) req@ffff9cf3481faa00 x1866777609355520/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.259725:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481faa00 with x1866777609355520 ext(3829760->3837951) 00002000:00010000:2.0:1780298642.259729:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf3481faa00 00000100:00100000:2.0:1780298642.259752:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5946 00000100:00100000:2.0:1780298642.259761:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481faa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609355520:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298642.259931:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481faa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609355520:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 171us (293us total) trans 0 rc 8192/8192 00000100:00100000:2.0:1780298642.259940:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5946 00002000:00100000:2.0:1780298642.259946:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x12:0x0] (3829760->3837951) req@ffff9cf3481faa00 x1866777609355520/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/8632 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 8192/8192 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.259955:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481faa00 with x1866777609355520 ext(3829760->3837951) 00002000:00010000:2.0:1780298642.259958:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf3481faa00 00000100:00100000:0.0:1780298642.261412:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.261460:0:6476:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609355776 00000100:00100000:2.0:1780298642.261466:0:6476:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609355776 00000100:00100000:2.0:1780298642.261506:0:6476:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61923 00000100:00100000:2.0:1780298642.261526:0:6476:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+96:29360:x1866777609355776:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:2.0:1780298642.261569:0:6476:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298642.261590:0:6476:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692400/0xaf35132e63032dca lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 53 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d208b expref: 96 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.261606:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.261611:0:6476:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298642.261772:0:6476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfe00/0xaf35132e63032e48 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 52 type: IBT flags: 0x40000000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.261786:0:6476:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf400/0xaf35132e63032d6f lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 52 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2037 expref: 87 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.261801:0:6476:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfe00/0xaf35132e63032e48 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 52 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.261828:0:6476:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf400/0xaf35132e63032d6f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 52 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2037 expref: 87 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.261841:0:6476:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf400/0xaf35132e63032d6f lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 52 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2037 expref: 87 pid: 6477 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.261852:0:6476:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fc700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6476:1866777604864384:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298642.261908:0:6476:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4900 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.262798:0:6476:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fc700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6476:1866777604864384:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298642.262815:0:6476:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfe00/0xaf35132e63032e48 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 52 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.262937:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.262962:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609355904 00000100:00100000:3.0:1780298642.262967:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609355904 00010000:00010000:3.0:1780298642.262986:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf400/0xaf35132e63032d6f lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 52 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2037 expref: 88 pid: 6477 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.263005:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14966 00000100:00100000:3.0:1780298642.263010:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+88:2400:x1866777609355904:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298642.263017:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298642.263021:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf400/0xaf35132e63032d6f lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 53 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2037 expref: 88 pid: 6477 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.263045:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf400/0xaf35132e63032d6f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 53 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2037 expref: 88 pid: 6477 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.263055:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf400/0xaf35132e63032d6f lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 53 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2037 expref: 88 pid: 6477 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.263063:0:6462:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298642.263065:0:6462:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfe00/0xaf35132e63032e48 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 52 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.263073:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfe00/0xaf35132e63032e48 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 52 type: IBT flags: 0x40290000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.263077:0:6462:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfe00/0xaf35132e63032e48 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 52 type: IBT flags: 0x40290000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.263093:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298642.263116:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+87:2400:x1866777609355904:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 105us (181us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298642.263121:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14966 00010000:00010000:2.0:1780298642.263121:0:6476:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfe00/0xaf35132e63032e48 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 51 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.263164:0:6476:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfe00/0xaf35132e63032e48 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 51 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1780298642.263219:0:6476:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991004 00000004:00010000:2.0:1780298642.263240:0:6476:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000b10c0396 reply state 000000005f511268 transno 4294991005 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfe00/0xaf35132e63032e48 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 51 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.263251:0:6476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfe00/0xaf35132e63032e48 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 51 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.263257:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.263260:0:6476:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1780298642.263263:0:6476:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1780298642.263328:0:6476:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+98:29360:x1866777609355776:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1815us (1919us total) trans 4294991005 rc 0/0 00000100:00100000:2.0:1780298642.263334:0:6476:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61923 00000020:00080000:2.0:1780298642.263336:0:6476:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1780298642.264070:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.264107:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609356032 00000100:00100000:3.0:1780298642.264112:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609356032 00000100:00100000:3.0:1780298642.264182:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8792 00000100:00100000:3.0:1780298642.264191:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:29360:x1866777609356032:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.264202:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.264215:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63032e4f lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2109 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.264250:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e63032ded lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2113536->2117631) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d20ae expref: 24 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.264260:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e63032df4 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d20b5 expref: 27 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.264267:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(96 177) 00010000:00010000:3.0:1780298642.264272:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(96 177) 00010000:00010000:3.0:1780298642.264277:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63032e4f lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2109 expref: 24 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.264312:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e63032df4 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d20b5 expref: 27 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.264323:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e63032df4 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d20b5 expref: 27 pid: 19992 timeout: 931 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.264334:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604864512:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298642.264386:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e63032ded lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2113536->2117631) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d20ae expref: 24 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.264396:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e63032ded lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2113536->2117631) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d20ae expref: 24 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.264417:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604864640:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298642.264441:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348ce1840 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.265064:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604864512:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298642.265406:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604864640:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298642.265430:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63032e4f lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2109 expref: 24 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.265442:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:3.0:1780298642.265451:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2600, rc 0) 00000100:00100000:3.0:1780298642.265489:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+24:29360:x1866777609356032:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1299us (1419us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298642.265497:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8792 00000100:00100000:1.0:1780298642.265500:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609356160 00000100:00100000:1.0:1780298642.265506:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609356160 00010000:00010000:1.0:1780298642.265532:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e63032df4 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d20b5 expref: 28 pid: 19992 timeout: 931 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.265569:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14967 00000100:00100000:1.0:1780298642.265584:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+28:2400:x1866777609356160:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298642.265596:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298642.265612:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e63032df4 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d20b5 expref: 28 pid: 19992 timeout: 931 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.265624:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e63032df4 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d20b5 expref: 28 pid: 19992 timeout: 931 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.265640:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e63032df4 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d20b5 expref: 28 pid: 19992 timeout: 931 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.265655:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298642.265684:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+27:2400:x1866777609356160:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 107us (246us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.265694:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14967 00000100:00100000:3.0:1780298642.265830:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.265874:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609356288 00000100:00100000:2.0:1780298642.265880:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609356288 00010000:00010000:2.0:1780298642.265898:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e63032ded lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2113536->2117631) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d20ae expref: 24 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.265927:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14968 00000100:00100000:2.0:1780298642.265936:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+24:2399:x1866777609356288:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298642.265945:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298642.265956:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e63032ded lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2113536->2117631) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d20ae expref: 24 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.265967:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e63032ded lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2113536->2117631) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d20ae expref: 24 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.265981:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e63032ded lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2113536->2117631) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d20ae expref: 24 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.265992:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(96 177) 00010000:00010000:2.0:1780298642.265996:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(96 177) 00010000:00010000:2.0:1780298642.266002:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63032e4f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2109 expref: 23 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.266012:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63032e4f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2109 expref: 23 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.266046:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63032e4f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2109 expref: 23 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.266055:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fc700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777604864768:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298642.266092:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.266817:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fc700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777604864768:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298642.266826:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298642.266854:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2399:x1866777609356288:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 920us (1025us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.266866:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14968 00000100:00100000:3.0:1780298642.266875:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.266926:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609356416 00000100:00100000:0.0:1780298642.266931:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609356416 00000100:00100000:0.0:1780298642.266969:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8793 00000100:00100000:0.0:1780298642.266979:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:29360:x1866777609356416:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.266990:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.267002:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c0d26bbc/0xaf35132e63032e56 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2110 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.267030:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032e41 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3829760->3837951) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2102 expref: 6 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.267038:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(71 130) 00010000:00010000:0.0:1780298642.267046:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(71 130) 00010000:00010000:0.0:1780298642.267051:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c0d26bbc/0xaf35132e63032e56 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2110 expref: 13 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.267074:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032e41 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3829760->3837951) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2102 expref: 6 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.267087:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032e41 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3829760->3837951) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2102 expref: 6 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.267098:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fb800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604864896:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298642.267166:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417300 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.267725:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fb800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604864896:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298642.267737:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c0d26bbc/0xaf35132e63032e56 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2110 expref: 13 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.267745:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3c00, rc 0) 00000100:00100000:0.0:1780298642.267765:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:29360:x1866777609356416:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 790us (894us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.267770:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8793 00000100:00100000:1.0:1780298642.267960:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.267993:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609356544 00000100:00100000:2.0:1780298642.267998:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609356544 00010000:00010000:2.0:1780298642.268010:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032e41 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3829760->3837951) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2102 expref: 7 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.268030:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14969 00000100:00100000:2.0:1780298642.268036:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f754380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609356544:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298642.268043:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298642.268053:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032e41 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3829760->3837951) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2102 expref: 7 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.268061:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032e41 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3829760->3837951) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2102 expref: 7 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.268071:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032e41 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3829760->3837951) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2102 expref: 7 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.268079:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(71 130) 00010000:00010000:2.0:1780298642.268081:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(71 130) 00010000:00010000:2.0:1780298642.268085:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c0d26bbc/0xaf35132e63032e56 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2110 expref: 12 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.268090:0:6463:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c0d26bbc/0xaf35132e63032e56 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2110 expref: 12 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.268107:0:6463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000c0d26bbc/0xaf35132e63032e56 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2110 expref: 12 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.268114:0:6463:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fc700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777604865024:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298642.268166:0:6463:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.268755:0:6463:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fc700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777604865024:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298642.268762:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298642.268786:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f754380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609356544:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 751us (827us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.268790:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14969 00000100:00100000:1.0:1780298642.268954:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.268999:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609356672 00000100:00100000:2.0:1780298642.269003:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609356672 00002000:00010000:2.0:1780298642.269020:0:8363:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:2.0:1780298642.269024:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f746680 with x1866777609356672 ext(4194304->18446744073709551615) 00002000:00010000:2.0:1780298642.269031:0:8363:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f746680. 00000100:00100000:3.0:1780298642.269038:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.269093:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609356800 00000100:00100000:2.0:1780298642.269096:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609356800 00002000:00010000:2.0:1780298642.269106:0:8363:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (400109->18446744073709551615) 00000100:00100000:0.0:1780298642.269107:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5947 00002000:00100000:2.0:1780298642.269111:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24304d500 with x1866777609356800 ext(400109->18446744073709551615) 00002000:00010000:2.0:1780298642.269116:0:8363:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24304d500. 00000100:00100000:0.0:1780298642.269144:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f746680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2398:x1866777609356672:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:2.0:1780298642.269157:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5948 00000100:00100000:2.0:1780298642.269168:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609356800:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:0.0:1780298642.269261:0:8364:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:0.0:1780298642.269264:0:8364:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298642.269266:0:8364:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298642.269268:0:8364:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298642.269270:0:8364:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 416 -> 416 00000100:00100000:0.0:1780298642.269308:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f746680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+24:2398:x1866777609356672:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 192us (355us total) trans 4294969614 rc 0/0 00000100:00100000:0.0:1780298642.269316:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5947 00002000:00010000:0.0:1780298642.269320:0:8364:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:0.0:1780298642.269323:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f746680 with x1866777609356672 ext(4194304->18446744073709551615) 00002000:00010000:0.0:1780298642.269328:0:8364:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f746680. 00002000:00010000:2.0:1780298642.269405:0:8363:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4115648 -> 400109 00002000:00010000:2.0:1780298642.269409:0:8363:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.269413:0:8363:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.269415:0:8363:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.269417:0:8363:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 48 -> 0 00000100:00100000:2.0:1780298642.269445:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2398:x1866777609356800:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 280us (407us total) trans 4294969177 rc 0/0 00000100:00100000:2.0:1780298642.269452:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5948 00002000:00010000:2.0:1780298642.269456:0:8363:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (400109->18446744073709551615) 00002000:00100000:2.0:1780298642.269459:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24304d500 with x1866777609356800 ext(400109->18446744073709551615) 00002000:00010000:2.0:1780298642.269463:0:8363:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24304d500. 00000100:00100000:3.0:1780298642.270195:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.270225:0:9976:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609356928 00000100:00100000:2.0:1780298642.270230:0:9976:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609356928 00000100:00100000:2.0:1780298642.270259:0:9976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61924 00000100:00100000:2.0:1780298642.270267:0:9976:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+98:29360:x1866777609356928:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.270280:0:9976:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.270301:0:9976:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be600/0xaf35132e63032e5d lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 52 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2117 expref: -99 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.270360:0:9976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfa00/0xaf35132e63032e64 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 53 type: IBT flags: 0x40000000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.270371:0:9976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfa00/0xaf35132e63032e64 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 53 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.270378:0:9976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfa00/0xaf35132e63032e64 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 53 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298642.270394:0:9976:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 4594413 00000004:00200000:2.0:1780298642.270397:0:9976:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298642.270409:0:9976:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfa00/0xaf35132e63032e64 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 53 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.270424:0:9976:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be600/0xaf35132e63032e5d lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 53 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2117 expref: 100 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.270432:0:9976:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfa00/0xaf35132e63032e64 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 52 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2117 expref: 99 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.270438:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.270440:0:9976:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298642.270442:0:9976:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf2445bfa00, rc 0) 00000100:00100000:2.0:1780298642.270463:0:9976:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+99:29360:x1866777609356928:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 198us (269us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.270467:0:9976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61924 00000100:00100000:1.0:1780298642.271836:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.271865:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609357056 00000100:00100000:2.0:1780298642.271870:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609357056 00000100:00100000:2.0:1780298642.271895:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8794 00000100:00100000:2.0:1780298642.271901:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+27:29360:x1866777609357056:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.271908:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.271916:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000a323d4b/0xaf35132e63032e6b lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d211e expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.271932:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63032e4f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2109 expref: 23 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.271938:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(96 177) 00010000:00010000:2.0:1780298642.271941:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(97 177) 00010000:00010000:2.0:1780298642.271945:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000a323d4b/0xaf35132e63032e6b lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2514944->2523135] (req 2514944->2523135) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d211e expref: 28 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.271966:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63032e4f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2109 expref: 23 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.271977:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63032e4f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2109 expref: 23 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.271985:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604865152:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298642.272027:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.272836:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604865152:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298642.272851:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000a323d4b/0xaf35132e63032e6b lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2514944->2523135] (req 2514944->2523135) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d211e expref: 28 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.272865:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf2445bf000, rc 0) 00000100:00100000:2.0:1780298642.272887:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+28:29360:x1866777609357056:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 987us (1054us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.272895:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8794 00000100:00100000:1.0:1780298642.272966:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.273013:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609357184 00000100:00100000:0.0:1780298642.273018:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609357184 00010000:00010000:0.0:1780298642.273037:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63032e4f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2109 expref: 24 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.273069:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14970 00000100:00100000:0.0:1780298642.273079:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255092300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+24:2399:x1866777609357184:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298642.273089:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298642.273102:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63032e4f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2109 expref: 24 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.273112:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63032e4f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2109 expref: 24 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.273142:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63032e4f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2109 expref: 24 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.273155:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(97 177) 00010000:00010000:0.0:1780298642.273159:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(97 177) 00010000:00010000:0.0:1780298642.273166:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000000a323d4b/0xaf35132e63032e6b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2514944->36069375] (req 2514944->2523135) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d211e expref: 27 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.273174:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000a323d4b/0xaf35132e63032e6b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2514944->36069375] (req 2514944->2523135) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d211e expref: 27 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.273198:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000000a323d4b/0xaf35132e63032e6b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2514944->36069375] (req 2514944->2523135) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d211e expref: 27 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.273209:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fb800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777604865280:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298642.273242:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417300 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.273887:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fb800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777604865280:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298642.273899:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298642.273938:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255092300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2399:x1866777609357184:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 856us (969us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.273947:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14970 00000100:00100000:1.0:1780298642.274008:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.274042:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609357312 00000100:00100000:2.0:1780298642.274046:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609357312 00000100:00100000:2.0:1780298642.274077:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5949 00000100:00100000:2.0:1780298642.274085:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255091f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+28:2398:x1866777609357312:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298642.274239:0:19950:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:2.0:1780298642.274243:0:19950:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.274246:0:19950:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.274248:0:19950:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.274251:0:19950:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 416 -> 416 00000100:00100000:2.0:1780298642.274282:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255091f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+29:2398:x1866777609357312:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 199us (275us total) trans 4294969615 rc 0/0 00000100:00100000:2.0:1780298642.274290:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5949 00000100:00100000:0.0:1780298642.275219:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.275240:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609357440 00000100:00100000:1.0:1780298642.275245:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609357440 00000100:00100000:1.0:1780298642.275282:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8795 00000100:00100000:1.0:1780298642.275301:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2398:x1866777609357440:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.275309:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.275316:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009cb6f854/0xaf35132e63032e72 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2125 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.275343:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255093b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604865408:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.275378:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12f00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.276049:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255093b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604865408:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.276070:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009cb6f854/0xaf35132e63032e72 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2125 expref: 24 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.276083:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009cb6f854/0xaf35132e63032e72 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2125 expref: 24 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.276089:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36e00, rc 0) 00000100:00100000:1.0:1780298642.276110:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2398:x1866777609357440:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 823us (893us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.276115:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8795 00000100:00100000:2.0:1780298642.276961:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.276990:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609357568 00000100:00100000:3.0:1780298642.276994:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609357568 00000100:00100000:2.0:1780298642.277050:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8796 00000100:00100000:2.0:1780298642.277056:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2398:x1866777609357568:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.277064:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.277071:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fedd83c9/0xaf35132e63032e79 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d212c expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.277093:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481ff800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604865536:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.277147:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.277812:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481ff800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604865536:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.277824:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000fedd83c9/0xaf35132e63032e79 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d212c expref: 24 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.277835:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fedd83c9/0xaf35132e63032e79 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d212c expref: 24 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.277842:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf2445be000, rc 0) 00000100:00100000:2.0:1780298642.277865:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2398:x1866777609357568:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 810us (906us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.277873:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8796 00000100:00100000:0.0:1780298642.278691:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.278776:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609357696 00000100:00100000:3.0:1780298642.278780:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609357696 00000100:00100000:0.0:1780298642.278803:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.278821:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609357824 00000100:00100000:1.0:1780298642.278825:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609357824 00000100:00100000:3.0:1780298642.278835:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8797 00000100:00100000:3.0:1780298642.278843:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+29:2398:x1866777609357696:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298642.278848:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8798 00010000:00010000:3.0:1780298642.278852:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298642.278859:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609357824:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.278861:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e63032e80 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2133 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.278866:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.278875:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63032e87 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d213a expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.278906:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604865664:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.278912:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481ff800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604865792:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.278948:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348ce1840 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.278949:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4a80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.279580:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604865664:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.279596:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e63032e80 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2133 expref: 30 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.279607:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481ff800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604865792:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.279608:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e63032e80 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2133 expref: 30 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.279620:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2800, rc 0) 00010000:00010000:2.0:1780298642.279636:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63032e87 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d213a expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.279644:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+29:2398:x1866777609357696:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 803us (953us total) trans 0 rc 301/301 00010000:00010000:2.0:1780298642.279649:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63032e87 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d213a expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.279653:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8797 00010000:00010000:2.0:1780298642.279665:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf2445be600, rc 0) 00000100:00100000:2.0:1780298642.279684:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609357824:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 829us (883us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.279690:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8798 00000100:00100000:1.0:1780298642.280662:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.280696:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609357952 00000100:00100000:0.0:1780298642.280701:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609357952 00000100:00100000:0.0:1780298642.280726:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8799 00000100:00100000:0.0:1780298642.280731:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255093b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+29:29360:x1866777609357952:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.280737:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.280744:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e63032e8e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2141 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.280756:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(97 177) 00010000:00010000:0.0:1780298642.280759:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(97 177) 00010000:00010000:0.0:1780298642.280765:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e63032e8e lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [856064->2514943] (req 856064->860159) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2141 expref: 30 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.280774:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e63032e8e lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [856064->2514943] (req 856064->860159) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2141 expref: 30 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.280780:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2000, rc 0) 00000100:00100000:0.0:1780298642.280802:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255093b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+30:29360:x1866777609357952:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 71us (141us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.280806:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8799 00000100:00100000:0.0:1780298642.281554:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.281584:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609358080 00000100:00100000:2.0:1780298642.281588:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609358080 00002000:00100000:2.0:1780298642.281610:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (856064->860159) req@ffff9cf3481f8000 x1866777609358080/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.281622:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481f8000 with x1866777609358080 ext(856064->860159) 00002000:00010000:2.0:1780298642.281625:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481f8000 00000100:00100000:2.0:1780298642.281641:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5950 00000100:00100000:2.0:1780298642.281646:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+30:2398:x1866777609358080:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298642.281782:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+30:2398:x1866777609358080:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 135us (227us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298642.281792:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5950 00002000:00100000:2.0:1780298642.281798:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (856064->860159) req@ffff9cf3481f8000 x1866777609358080/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/4536 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.281807:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481f8000 with x1866777609358080 ext(856064->860159) 00002000:00010000:2.0:1780298642.281811:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481f8000 00000100:00100000:2.0:1780298642.282754:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.282783:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609358208 00000100:00100000:1.0:1780298642.282787:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609358208 00000100:00100000:1.0:1780298642.282811:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8800 00000100:00100000:1.0:1780298642.282816:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481ff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2397:x1866777609358208:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.282823:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.282830:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e63032e95 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2148 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.282856:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255091180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604865920:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.282891:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12f00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.283564:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255091180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604865920:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.283579:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e63032e95 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2148 expref: 24 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.283591:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e63032e95 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2148 expref: 24 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.283598:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37200, rc 0) 00000100:00100000:2.0:1780298642.283621:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481ff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2397:x1866777609358208:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 805us (869us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.283626:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8800 00000100:00100000:0.0:1780298642.284364:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.284401:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609358336 00000100:00100000:3.0:1780298642.284406:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609358336 00000100:00100000:3.0:1780298642.284430:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8801 00000100:00100000:3.0:1780298642.284435:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2397:x1866777609358336:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.284442:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.284449:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e63032e9c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d214f expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.284469:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604866048:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.284498:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348ce1840 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.285078:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604866048:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.285094:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e63032e9c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d214f expref: 24 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.285107:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e63032e9c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d214f expref: 24 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.285116:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2a00, rc 0) 00000100:00100000:3.0:1780298642.285155:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2397:x1866777609358336:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 720us (792us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.285160:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8801 00000100:00100000:1.0:1780298642.285909:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.285938:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609358464 00000100:00100000:2.0:1780298642.285942:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609358464 00000100:00100000:2.0:1780298642.285965:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8802 00000100:00100000:2.0:1780298642.285972:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255092d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+30:29360:x1866777609358464:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.285978:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.285985:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000408198ca/0xaf35132e63032ea3 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2156 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.285999:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e63032e8e lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [856064->2514943] (req 856064->860159) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2141 expref: 31 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.286006:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(97 177) 00010000:00010000:2.0:1780298642.286008:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(98 177) 00010000:00010000:2.0:1780298642.286011:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000408198ca/0xaf35132e63032ea3 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1974272->1978367] (req 1974272->1978367) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2156 expref: 31 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.286026:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e63032e8e lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [856064->2514943] (req 856064->860159) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2141 expref: 31 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.286035:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e63032e8e lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [856064->2514943] (req 856064->860159) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2141 expref: 31 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.286041:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604866176:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298642.286079:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.286704:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604866176:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298642.286720:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000408198ca/0xaf35132e63032ea3 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1974272->1978367] (req 1974272->1978367) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2156 expref: 31 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.286731:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf2445bec00, rc 0) 00000100:00100000:2.0:1780298642.286758:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255092d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+31:29360:x1866777609358464:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 788us (850us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.286763:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8802 00000100:00100000:0.0:1780298642.286969:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.287010:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609358592 00000100:00100000:1.0:1780298642.287014:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609358592 00010000:00010000:1.0:1780298642.287033:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e63032e8e lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [856064->2514943] (req 856064->860159) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2141 expref: 31 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.287059:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14971 00000100:00100000:1.0:1780298642.287075:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+31:2400:x1866777609358592:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298642.287084:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298642.287095:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e63032e8e lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [856064->2514943] (req 856064->860159) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2141 expref: 31 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.287110:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e63032e8e lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [856064->2514943] (req 856064->860159) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2141 expref: 31 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.287139:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e63032e8e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [856064->2514943] (req 856064->860159) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2141 expref: 31 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.287148:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(98 177) 00010000:00010000:1.0:1780298642.287151:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(98 177) 00010000:00010000:1.0:1780298642.287156:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000408198ca/0xaf35132e63032ea3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1974272->2514943] (req 1974272->1978367) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2156 expref: 30 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.287161:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000408198ca/0xaf35132e63032ea3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1974272->2514943] (req 1974272->1978367) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2156 expref: 30 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.287176:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000408198ca/0xaf35132e63032ea3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1974272->2514943] (req 1974272->1978367) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2156 expref: 30 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.287183:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777604866304:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298642.287212:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12f00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.287887:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777604866304:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298642.287895:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298642.287918:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+30:2400:x1866777609358592:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 846us (950us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.287924:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14971 00000100:00100000:0.0:1780298642.287988:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.288033:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609358720 00000100:00100000:3.0:1780298642.288037:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609358720 00000100:00100000:3.0:1780298642.288100:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5951 00000100:00100000:3.0:1780298642.288109:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+30:2397:x1866777609358720:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:3.0:1780298642.288256:0:19928:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:3.0:1780298642.288259:0:19928:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:3.0:1780298642.288260:0:19928:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:3.0:1780298642.288262:0:19928:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:3.0:1780298642.288263:0:19928:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 416 -> 416 00000100:00100000:3.0:1780298642.288285:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+31:2397:x1866777609358720:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 179us (297us total) trans 4294969616 rc 0/0 00000100:00100000:3.0:1780298642.288290:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5951 00000100:00100000:2.0:1780298642.288972:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.289017:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609358848 00000100:00100000:0.0:1780298642.289020:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609358848 00000100:00100000:0.0:1780298642.289047:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61925 00000100:00100000:0.0:1780298642.289052:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+87:29360:x1866777609358848:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.289061:0:6477:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.289072:0:6477:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3a00/0xaf35132e63032eaa lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 53 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d215d expref: -99 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.289166:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3800/0xaf35132e63032eb1 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 54 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.289180:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3800/0xaf35132e63032eb1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 54 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.289188:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3800/0xaf35132e63032eb1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 54 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1780298642.289208:0:6477:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 4594413 00000004:00200000:0.0:1780298642.289211:0:6477:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1780298642.289223:0:6477:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3800/0xaf35132e63032eb1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 54 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.289239:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3a00/0xaf35132e63032eaa lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 54 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d215d expref: 89 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.289249:0:6477:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3800/0xaf35132e63032eb1 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 53 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d215d expref: 88 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.289254:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298642.289256:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1780298642.289259:0:6477:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3800, rc 0) 00000100:00100000:0.0:1780298642.289290:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+88:29360:x1866777609358848:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 237us (318us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.289307:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61925 00000100:00100000:1.0:1780298642.290073:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.290102:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609358976 00000100:00100000:0.0:1780298642.290107:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609358976 00000100:00100000:0.0:1780298642.290165:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8803 00000100:00100000:0.0:1780298642.290173:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255091500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+31:2397:x1866777609358976:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.290180:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.290188:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000488a7b88/0xaf35132e63032eb8 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2164 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.290215:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254919f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604866432:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.290222:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.290259:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417300 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.290260:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609359104 00000100:00100000:1.0:1780298642.290265:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609359104 00000100:00100000:1.0:1780298642.290311:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8804 00000100:00100000:1.0:1780298642.290321:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348007100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609359104:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.290329:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.290338:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e63032ebf lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d216b expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.290365:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255092a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604866560:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.290395:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12f00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.290971:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254919f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604866432:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.290995:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000488a7b88/0xaf35132e63032eb8 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2164 expref: 32 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.290996:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255092a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604866560:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.291018:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e63032ebf lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d216b expref: 7 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.291018:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000488a7b88/0xaf35132e63032eb8 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2164 expref: 32 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.291040:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2200, rc 0) 00010000:00010000:1.0:1780298642.291044:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e63032ebf lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d216b expref: 7 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.291052:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36a00, rc 0) 00000100:00100000:0.0:1780298642.291068:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255091500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+31:2397:x1866777609358976:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 896us (1005us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.291074:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348007100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609359104:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 755us (852us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.291077:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8803 00000100:00100000:1.0:1780298642.291088:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8804 00000100:00100000:2.0:1780298642.291944:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.291998:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609359232 00000100:00100000:3.0:1780298642.292003:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609359232 00000100:00100000:2.0:1780298642.292030:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.292059:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609359360 00000100:00100000:1.0:1780298642.292065:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609359360 00000100:00100000:0.0:1780298642.292075:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8805 00000100:00100000:0.0:1780298642.292084:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+31:2397:x1866777609359232:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298642.292089:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8806 00010000:00010000:0.0:1780298642.292093:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298642.292099:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609359360:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.292101:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001d4afe0b/0xaf35132e63032ec6 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2172 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.292107:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.292116:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63032ecd lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2179 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.292140:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254919f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604866688:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.292176:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417300 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.292203:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604866816:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.292242:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348ce1840 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.292769:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254919f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604866688:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.292783:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001d4afe0b/0xaf35132e63032ec6 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2172 expref: 32 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.292792:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001d4afe0b/0xaf35132e63032ec6 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2172 expref: 32 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.292800:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3000, rc 0) 00000100:00100000:0.0:1780298642.292827:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+31:2397:x1866777609359232:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 746us (886us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.292832:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8805 00000100:00100000:3.0:1780298642.292879:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604866816:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.292899:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63032ecd lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2179 expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.292913:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63032ecd lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2179 expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.292954:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2800, rc 0) 00000100:00100000:3.0:1780298642.292977:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609359360:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 882us (949us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.292981:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8806 00000100:00100000:1.0:1780298642.294634:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.294687:0:7411:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609359616 00000100:00100000:2.0:1780298642.294692:0:7411:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609359616 00000100:00100000:2.0:1780298642.294728:0:7411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61926 00000100:00100000:2.0:1780298642.294737:0:7411:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255092a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+88:29360:x1866777609359616:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:2.0:1780298642.294775:0:7411:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298642.294793:0:7411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3800/0xaf35132e63032eb1 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 54 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d215d expref: 88 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.294809:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.294813:0:7411:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298642.294962:0:7411:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf400/0xaf35132e63032ed4 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 53 type: IBT flags: 0x40000000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.294976:0:7411:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfa00/0xaf35132e63032e64 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 53 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2117 expref: 98 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.294992:0:7411:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf400/0xaf35132e63032ed4 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 53 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.295028:0:7411:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfa00/0xaf35132e63032e64 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 53 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2117 expref: 98 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.295042:0:7411:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfa00/0xaf35132e63032e64 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 53 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2117 expref: 98 pid: 9976 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.295054:0:7411:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd7b80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7411:1866777604866944:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298642.295102:0:7411:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.295668:0:7411:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd7b80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7411:1866777604866944:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298642.295682:0:7411:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf400/0xaf35132e63032ed4 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 53 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.295844:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.295911:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609359744 00000100:00100000:3.0:1780298642.295915:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609359744 00010000:00010000:3.0:1780298642.295930:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfa00/0xaf35132e63032e64 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 53 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2117 expref: 99 pid: 9976 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.295949:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14972 00000100:00100000:3.0:1780298642.295954:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b03450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+99:2399:x1866777609359744:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298642.295968:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298642.295972:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfa00/0xaf35132e63032e64 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 54 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2117 expref: 99 pid: 9976 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.295979:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfa00/0xaf35132e63032e64 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 54 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2117 expref: 99 pid: 9976 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.295988:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfa00/0xaf35132e63032e64 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 54 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2117 expref: 99 pid: 9976 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.295996:0:6462:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298642.295998:0:6462:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf400/0xaf35132e63032ed4 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 53 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.296005:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf400/0xaf35132e63032ed4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 53 type: IBT flags: 0x40290000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.296010:0:6462:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf400/0xaf35132e63032ed4 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 53 type: IBT flags: 0x40290000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.296023:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298642.296043:0:7411:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf400/0xaf35132e63032ed4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 52 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.296045:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b03450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+98:2399:x1866777609359744:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 91us (202us total) trans 0 rc 0/0 00010000:00010000:1.0:1780298642.296058:0:7411:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf400/0xaf35132e63032ed4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 52 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.296061:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14972 00000001:00080000:1.0:1780298642.296105:0:7411:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991006 00000004:00010000:1.0:1780298642.296146:0:7411:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 0000000004b65273 reply state 0000000029b9523e transno 4294991007 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf400/0xaf35132e63032ed4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 52 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.296156:0:7411:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf400/0xaf35132e63032ed4 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 52 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.296161:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298642.296164:0:7411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1780298642.296167:0:7411:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1780298642.296223:0:7411:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255092a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+90:29360:x1866777609359616:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1487us (1590us total) trans 4294991007 rc 0/0 00000100:00100000:1.0:1780298642.296231:0:7411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61926 00000020:00080000:1.0:1780298642.296235:0:7411:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1780298642.296754:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.296782:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609359872 00000100:00100000:0.0:1780298642.296786:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609359872 00000100:00100000:0.0:1780298642.296809:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8807 00000100:00100000:0.0:1780298642.296815:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255090700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+31:29360:x1866777609359872:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.296821:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.296828:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000488a7b88/0xaf35132e63032edb lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2180 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.296842:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000408198ca/0xaf35132e63032ea3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1974272->2514943] (req 1974272->1978367) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2156 expref: 32 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.296851:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000000a323d4b/0xaf35132e63032e6b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2514944->36069375] (req 2514944->2523135) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d211e expref: 32 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.296856:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 2 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(98 177) 00010000:00010000:0.0:1780298642.296859:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(100 177) 00010000:00010000:0.0:1780298642.296861:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000488a7b88/0xaf35132e63032edb lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2180 expref: 32 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.296874:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000000a323d4b/0xaf35132e63032e6b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2514944->36069375] (req 2514944->2523135) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d211e expref: 32 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.296880:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000000a323d4b/0xaf35132e63032e6b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2514944->36069375] (req 2514944->2523135) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d211e expref: 32 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.296887:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254919f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604867072:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298642.296921:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000408198ca/0xaf35132e63032ea3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1974272->2514943] (req 1974272->1978367) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2156 expref: 32 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.296929:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000408198ca/0xaf35132e63032ea3 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1974272->2514943] (req 1974272->1978367) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2156 expref: 32 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.296935:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25491a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604867200:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298642.296951:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417300 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.297479:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254919f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604867072:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298642.297581:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25491a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604867200:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298642.297601:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000488a7b88/0xaf35132e63032edb lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2180 expref: 32 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.297614:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2200, rc 0) 00000100:00100000:0.0:1780298642.297645:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255090700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+32:29360:x1866777609359872:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 829us (890us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.297653:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8807 00000100:00100000:3.0:1780298642.297716:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.297752:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609360000 00000100:00100000:1.0:1780298642.297756:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609360000 00010000:00010000:1.0:1780298642.297773:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000000a323d4b/0xaf35132e63032e6b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2514944->36069375] (req 2514944->2523135) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d211e expref: 32 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.297799:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14973 00000100:00100000:1.0:1780298642.297806:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+32:2400:x1866777609360000:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298642.297815:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298642.297829:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000000a323d4b/0xaf35132e63032e6b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2514944->36069375] (req 2514944->2523135) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d211e expref: 32 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.297840:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000000a323d4b/0xaf35132e63032e6b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2514944->36069375] (req 2514944->2523135) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d211e expref: 32 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.297855:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000a323d4b/0xaf35132e63032e6b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2514944->36069375] (req 2514944->2523135) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d211e expref: 32 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.297865:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298642.297890:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+31:2400:x1866777609360000:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 85us (174us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.297897:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14973 00000100:00100000:3.0:1780298642.297955:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.297973:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609360128 00000100:00100000:2.0:1780298642.297977:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609360128 00010000:00010000:2.0:1780298642.297991:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000408198ca/0xaf35132e63032ea3 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1974272->2514943] (req 1974272->1978367) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2156 expref: 31 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.298010:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14974 00000100:00100000:2.0:1780298642.298014:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+31:2399:x1866777609360128:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298642.298020:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298642.298030:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000408198ca/0xaf35132e63032ea3 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1974272->2514943] (req 1974272->1978367) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2156 expref: 31 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.298040:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000408198ca/0xaf35132e63032ea3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1974272->2514943] (req 1974272->1978367) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2156 expref: 31 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.298053:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000408198ca/0xaf35132e63032ea3 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1974272->2514943] (req 1974272->1978367) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2156 expref: 31 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.298069:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(100 177) 00010000:00010000:2.0:1780298642.298072:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(100 177) 00010000:00010000:2.0:1780298642.298076:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000488a7b88/0xaf35132e63032edb lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2180 expref: 30 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.298082:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000488a7b88/0xaf35132e63032edb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2180 expref: 30 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.298098:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000488a7b88/0xaf35132e63032edb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2180 expref: 30 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.298104:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd6a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777604867328:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298642.298153:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.298723:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd6a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777604867328:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298642.298733:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298642.298757:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+30:2399:x1866777609360128:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 741us (801us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.298768:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14974 00000100:00100000:3.0:1780298642.298818:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.298850:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609360256 00000100:00100000:1.0:1780298642.298854:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609360256 00000100:00100000:1.0:1780298642.298883:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8808 00000100:00100000:1.0:1780298642.298892:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:29360:x1866777609360256:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.298900:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.298909:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e63032ee2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2187 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.298927:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000c0d26bbc/0xaf35132e63032e56 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2110 expref: 13 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.298935:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(71 130) 00010000:00010000:1.0:1780298642.298938:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(72 130) 00010000:00010000:1.0:1780298642.298942:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e63032ee2 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2187 expref: 7 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.298959:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000c0d26bbc/0xaf35132e63032e56 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2110 expref: 13 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.298970:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000c0d26bbc/0xaf35132e63032e56 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2110 expref: 13 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.298981:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255090380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604867456:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298642.299009:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4f00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.299575:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255090380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604867456:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298642.299591:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e63032ee2 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2187 expref: 7 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.299601:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36a00, rc 0) 00000100:00100000:1.0:1780298642.299628:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609360256:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 737us (808us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.299641:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8808 00000100:00100000:3.0:1780298642.299754:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.299768:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609360384 00000100:00100000:2.0:1780298642.299773:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609360384 00010000:00010000:2.0:1780298642.299786:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000c0d26bbc/0xaf35132e63032e56 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2110 expref: 14 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.299806:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14975 00000100:00100000:2.0:1780298642.299811:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609360384:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298642.299817:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298642.299826:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000c0d26bbc/0xaf35132e63032e56 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2110 expref: 14 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.299837:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000c0d26bbc/0xaf35132e63032e56 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2110 expref: 14 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.299847:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c0d26bbc/0xaf35132e63032e56 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2110 expref: 14 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.299854:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(72 130) 00010000:00010000:2.0:1780298642.299856:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(72 130) 00010000:00010000:2.0:1780298642.299860:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e63032ee2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2187 expref: 6 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.299865:0:6463:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e63032ee2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2187 expref: 6 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.299878:0:6463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e63032ee2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2187 expref: 6 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.299884:0:6463:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd4000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777604867584:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298642.299910:0:6463:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.300635:0:6463:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd4000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777604867584:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298642.300644:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298642.300669:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609360384:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 857us (916us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.300674:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14975 00000100:00100000:3.0:1780298642.300853:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.300886:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.300889:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609360512 00000100:00100000:1.0:1780298642.300892:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609360512 00002000:00010000:1.0:1780298642.300908:0:19943:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4928525->18446744073709551615) 00002000:00100000:1.0:1780298642.300911:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24304fb80 with x1866777609360512 ext(4928525->18446744073709551615) 00002000:00010000:1.0:1780298642.300917:0:19943:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24304fb80. 00000100:00100000:1.0:1780298642.300930:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609360640 00000100:00100000:1.0:1780298642.300932:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609360640 00002000:00010000:1.0:1780298642.300939:0:19943:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4194304->18446744073709551615) 00002000:00100000:1.0:1780298642.300941:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24304f100 with x1866777609360640 ext(4194304->18446744073709551615) 00002000:00010000:1.0:1780298642.300944:0:19943:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24304f100. 00000100:00100000:0.0:1780298642.300945:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5952 00000100:00100000:0.0:1780298642.300958:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+30:2397:x1866777609360512:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:1.0:1780298642.300964:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5953 00000100:00100000:1.0:1780298642.300971:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609360640:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:1.0:1780298642.301068:0:19943:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 400109 -> 4194304 00002000:00010000:1.0:1780298642.301073:0:19943:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:1.0:1780298642.301075:0:19943:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:1.0:1780298642.301077:0:19943:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:1.0:1780298642.301078:0:19943:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 0 -> 0 00002000:00010000:0.0:1780298642.301091:0:8364:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4928525 00002000:00010000:0.0:1780298642.301096:0:8364:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298642.301099:0:8364:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00000100:00100000:1.0:1780298642.301102:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609360640:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 132us (216us total) trans 4294969178 rc 0/0 00002000:00010000:0.0:1780298642.301104:0:8364:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298642.301110:0:8364:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 416 -> 416 00000100:00100000:1.0:1780298642.301113:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5953 00002000:00010000:1.0:1780298642.301117:0:19943:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4194304->18446744073709551615) 00002000:00100000:1.0:1780298642.301119:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24304f100 with x1866777609360640 ext(4194304->18446744073709551615) 00002000:00010000:1.0:1780298642.301136:0:19943:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24304f100. 00000100:00100000:0.0:1780298642.301184:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+31:2397:x1866777609360512:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 227us (328us total) trans 4294969617 rc 0/0 00000100:00100000:0.0:1780298642.301192:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5952 00002000:00010000:0.0:1780298642.301196:0:8364:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4928525->18446744073709551615) 00002000:00100000:0.0:1780298642.301199:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24304fb80 with x1866777609360512 ext(4928525->18446744073709551615) 00002000:00010000:0.0:1780298642.301205:0:8364:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24304fb80. 00000100:00100000:0.0:1780298642.302230:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.302256:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609360768 00000100:00100000:2.0:1780298642.302260:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609360768 00002000:00100000:2.0:1780298642.302278:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (4923392->4927487) req@ffff9cf25491a300 x1866777609360768/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.302285:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25491a300 with x1866777609360768 ext(4923392->4927487) 00002000:00010000:2.0:1780298642.302289:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf25491a300 00000100:00100000:2.0:1780298642.302314:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5954 00000100:00100000:2.0:1780298642.302320:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+31:2398:x1866777609360768:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298642.302435:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+31:2398:x1866777609360768:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 115us (204us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298642.302443:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5954 00002000:00100000:2.0:1780298642.302449:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (4923392->4927487) req@ffff9cf25491a300 x1866777609360768/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/4536 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.302458:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25491a300 with x1866777609360768 ext(4923392->4927487) 00002000:00010000:2.0:1780298642.302462:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf25491a300 00000100:00100000:2.0:1780298642.303490:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.303525:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609360896 00000100:00100000:0.0:1780298642.303529:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609360896 00002000:00100000:0.0:1780298642.303547:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (4927488->4931583) req@ffff9cf347cd5c00 x1866777609360896/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.303553:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf347cd5c00 with x1866777609360896 ext(4927488->4931583) 00002000:00010000:0.0:1780298642.303556:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf347cd5c00 00000100:00100000:0.0:1780298642.303570:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5955 00000100:00100000:0.0:1780298642.303575:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+31:2398:x1866777609360896:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298642.303633:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+31:2398:x1866777609360896:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 58us (144us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298642.303639:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5955 00002000:00100000:0.0:1780298642.303643:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (4927488->4931583) req@ffff9cf347cd5c00 x1866777609360896/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/4536 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.303649:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf347cd5c00 with x1866777609360896 ext(4927488->4931583) 00002000:00010000:0.0:1780298642.303651:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf347cd5c00 00000100:00100000:3.0:1780298642.304622:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.304658:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609361024 00000100:00100000:2.0:1780298642.304662:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609361024 00002000:00100000:2.0:1780298642.304682:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (4923392->4928524) req@ffff9cf37f754380 x1866777609361024/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 5624/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.304689:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f754380 with x1866777609361024 ext(4923392->4928524) 00002000:00010000:2.0:1780298642.304691:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f754380 00000100:00100000:2.0:1780298642.304708:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5956 00000100:00100000:2.0:1780298642.304713:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f754380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+31:2398:x1866777609361024:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:2.0:1780298642.305086:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f754380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+33:2398:x1866777609361024:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 373us (463us total) trans 4294969618 rc 0/0 00000100:00100000:2.0:1780298642.305091:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5956 00002000:00100000:2.0:1780298642.305096:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (4923392->4928524) req@ffff9cf37f754380 x1866777609361024/t4294969618(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 5624/448 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.305107:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f754380 with x1866777609361024 ext(4923392->4928524) 00002000:00010000:2.0:1780298642.305109:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f754380 00000100:00100000:3.0:1780298642.305729:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.305758:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609361152 00000100:00100000:2.0:1780298642.305762:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609361152 00000100:00100000:2.0:1780298642.305783:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8809 00000100:00100000:2.0:1780298642.305788:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+33:2398:x1866777609361152:12345-192.168.203.25@tcp:16:fsx.0 00000001:00080000:1.0:1780298642.306557:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969618 is committed 00000001:00080000:1.0:1780298642.306573:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969614 is committed 00000100:00100000:2.0:1780298642.306668:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609361152:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 879us (938us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.306673:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8809 00000100:00100000:1.0:1780298642.307469:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.307499:0:6476:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609361280 00000100:00100000:2.0:1780298642.307503:0:6476:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609361280 00000100:00100000:2.0:1780298642.307528:0:6476:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61927 00000100:00100000:2.0:1780298642.307533:0:6476:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255090380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+98:29360:x1866777609361280:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.307542:0:6476:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.307554:0:6476:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfa00/0xaf35132e63032ee9 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 53 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d218e expref: -99 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.307613:0:6476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be000/0xaf35132e63032ef0 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 54 type: IBT flags: 0x40000000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.307625:0:6476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be000/0xaf35132e63032ef0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 54 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.307633:0:6476:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be000/0xaf35132e63032ef0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 54 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298642.307650:0:6476:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 9122829 00000004:00200000:2.0:1780298642.307653:0:6476:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298642.307666:0:6476:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be000/0xaf35132e63032ef0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 54 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.307683:0:6476:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfa00/0xaf35132e63032ee9 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 54 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d218e expref: 100 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.307693:0:6476:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be000/0xaf35132e63032ef0 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 53 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d218e expref: 99 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.307699:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.307701:0:6476:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298642.307703:0:6476:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf2445be000, rc 0) 00000100:00100000:2.0:1780298642.307726:0:6476:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255090380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+99:29360:x1866777609361280:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 193us (259us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.307730:0:6476:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61927 00000100:00100000:3.0:1780298642.308587:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.308619:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609361408 00000100:00100000:0.0:1780298642.308623:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609361408 00000100:00100000:0.0:1780298642.308652:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8810 00000100:00100000:0.0:1780298642.308658:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2397:x1866777609361408:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.308664:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.308673:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a5a492f0/0xaf35132e63032ef7 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2195 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.308709:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254919f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604867712:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.308735:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.308763:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417e40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.308776:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609361536 00000100:00100000:2.0:1780298642.308780:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609361536 00000100:00100000:2.0:1780298642.308808:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8811 00000100:00100000:2.0:1780298642.308814:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255091c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609361536:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.308821:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.308829:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000a323d4b/0xaf35132e63032efe lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d219c expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.308859:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604867840:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.308891:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4a80 going to sleep for 11 seconds 00002000:00010000:0.0:1780298642.309426:0:20252:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks: 416 -> 432 00000100:00100000:0.0:1780298642.309437:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254919f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604867712:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.309457:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a5a492f0/0xaf35132e63032ef7 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2195 expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.309475:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a5a492f0/0xaf35132e63032ef7 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2195 expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.309484:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3600, rc 0) 00000100:00100000:0.0:1780298642.309513:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2397:x1866777609361408:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 854us (925us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.309521:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8810 00000100:00100000:2.0:1780298642.309646:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604867840:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.309666:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000a323d4b/0xaf35132e63032efe lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d219c expref: 14 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.309681:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000a323d4b/0xaf35132e63032efe lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d219c expref: 14 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.309692:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf2445bf000, rc 0) 00000100:00100000:2.0:1780298642.309715:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255091c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609361536:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 902us (982us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.309722:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8811 00000100:00100000:3.0:1780298642.310576:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.310598:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.310617:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609361664 00000100:00100000:1.0:1780298642.310622:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609361664 00000100:00100000:0.0:1780298642.310625:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609361792 00000100:00100000:0.0:1780298642.310630:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609361792 00000100:00100000:1.0:1780298642.310656:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8812 00000100:00100000:1.0:1780298642.310664:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2398:x1866777609361664:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.310671:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298642.310675:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8813 00010000:00010000:1.0:1780298642.310679:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003c8dab73/0xaf35132e63032f05 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d21a3 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.310689:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609361792:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.310697:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298642.310707:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255091880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604867968:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.310708:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0cae770/0xaf35132e63032f0c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d21aa expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.310740:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254919f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604868096:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.310754:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4f00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.310764:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417e40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.311423:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255091880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604867968:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.311439:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254919f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604868096:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.311444:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003c8dab73/0xaf35132e63032f05 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d21a3 expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.311456:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e0cae770/0xaf35132e63032f0c lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d21aa expref: 14 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.311462:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003c8dab73/0xaf35132e63032f05 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d21a3 expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.311482:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36800, rc 0) 00010000:00010000:0.0:1780298642.311486:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0cae770/0xaf35132e63032f0c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d21aa expref: 14 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.311494:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2600, rc 0) 00000100:00100000:1.0:1780298642.311504:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2398:x1866777609361664:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 843us (928us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.311510:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8812 00000100:00100000:0.0:1780298642.311514:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609361792:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 831us (915us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.311520:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8813 00000100:00100000:2.0:1780298642.312616:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.312642:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609361920 00000100:00100000:3.0:1780298642.312647:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609361920 00002000:00100000:3.0:1780298642.312668:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x12:0x0] (2793472->2797567) req@ffff9cf347cd5880 x1866777609361920/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298642.312675:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf347cd5880 with x1866777609361920 ext(2793472->2797567) 00002000:00010000:3.0:1780298642.312678:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf347cd5880 00000100:00100000:3.0:1780298642.312692:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5957 00000100:00100000:3.0:1780298642.312697:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609361920:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298642.312822:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609361920:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 124us (207us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1780298642.312827:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5957 00002000:00100000:3.0:1780298642.312832:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x12:0x0] (2793472->2797567) req@ffff9cf347cd5880 x1866777609361920/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/4536 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298642.312838:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf347cd5880 with x1866777609361920 ext(2793472->2797567) 00002000:00010000:3.0:1780298642.312840:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf347cd5880 00000100:00100000:3.0:1780298642.313567:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.313601:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609362048 00000100:00100000:0.0:1780298642.313606:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609362048 00002000:00100000:0.0:1780298642.313626:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (2801664->2805759) req@ffff9cf244ad8380 x1866777609362048/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.313641:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244ad8380 with x1866777609362048 ext(2801664->2805759) 00002000:00010000:0.0:1780298642.313644:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf244ad8380 00000100:00100000:0.0:1780298642.313658:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5958 00000100:00100000:0.0:1780298642.313664:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609362048:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298642.313729:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609362048:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 66us (163us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298642.313734:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5958 00002000:00100000:0.0:1780298642.313738:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (2801664->2805759) req@ffff9cf244ad8380 x1866777609362048/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/4536 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.313744:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244ad8380 with x1866777609362048 ext(2801664->2805759) 00002000:00010000:0.0:1780298642.313746:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf244ad8380 00000100:00100000:0.0:1780298642.314459:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.314489:0:9976:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609362176 00000100:00100000:2.0:1780298642.314494:0:9976:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609362176 00000100:00100000:2.0:1780298642.314522:0:9976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61928 00000100:00100000:2.0:1780298642.314529:0:9976:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254919f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+90:29360:x1866777609362176:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.314540:0:9976:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.314551:0:9976:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfa00/0xaf35132e63032f13 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 54 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d21b1 expref: -99 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.314603:0:9976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be600/0xaf35132e63032f1a lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 55 type: IBT flags: 0x40000000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.314614:0:9976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be600/0xaf35132e63032f1a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 55 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.314622:0:9976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be600/0xaf35132e63032f1a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 55 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298642.314641:0:9976:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 9122829 00000004:00200000:2.0:1780298642.314643:0:9976:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298642.314662:0:9976:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be600/0xaf35132e63032f1a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 55 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.314678:0:9976:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfa00/0xaf35132e63032f13 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 55 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d21b1 expref: 92 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.314688:0:9976:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be600/0xaf35132e63032f1a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 54 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d21b1 expref: 91 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.314693:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.314696:0:9976:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298642.314697:0:9976:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf2445be600, rc 0) 00000100:00100000:2.0:1780298642.314719:0:9976:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254919f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+91:29360:x1866777609362176:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 192us (260us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.314724:0:9976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61928 00000100:00100000:2.0:1780298642.316445:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.316472:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609362304 00000100:00100000:0.0:1780298642.316476:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609362304 00000100:00100000:2.0:1780298642.316498:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.316512:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609362432 00000100:00100000:0.0:1780298642.316516:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609362432 00000100:00100000:3.0:1780298642.316524:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8814 00000100:00100000:3.0:1780298642.316534:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2397:x1866777609362304:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298642.316550:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8815 00000100:00100000:0.0:1780298642.316561:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609362432:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.316568:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.316571:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.316577:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d4afe0b/0xaf35132e63032f21 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d21bf expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.316587:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e63032f28 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d21b8 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.316624:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ada300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604868224:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.316627:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254919c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604868352:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.316677:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.316677:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484173c0 going to sleep for 11 seconds 00002000:00010000:1.0:1780298642.317300:0:19992:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks: 0 -> 1 00000100:00100000:0.0:1780298642.317304:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ada300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604868224:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.317313:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254919c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604868352:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.317327:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e63032f28 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d21b8 expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.317333:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001d4afe0b/0xaf35132e63032f21 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d21bf expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.317343:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e63032f28 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d21b8 expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.317346:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d4afe0b/0xaf35132e63032f21 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d21bf expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.317362:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2800, rc 0) 00010000:00010000:1.0:1780298642.317364:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3000, rc 0) 00000100:00100000:0.0:1780298642.317388:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2397:x1866777609362304:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 858us (944us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.317395:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609362432:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 835us (896us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.317398:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8814 00000100:00100000:1.0:1780298642.317406:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8815 00000100:00100000:1.0:1780298642.318181:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.318205:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609362560 00000100:00100000:2.0:1780298642.318210:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609362560 00000100:00100000:2.0:1780298642.318241:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8816 00000100:00100000:2.0:1780298642.318259:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255091880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:29360:x1866777609362560:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.318268:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.318276:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000a323d4b/0xaf35132e63032f2f lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d21c6 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.318309:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000488a7b88/0xaf35132e63032edb lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2180 expref: 6 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.318316:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(100 177) 00010000:00010000:2.0:1780298642.318319:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(101 177) 00010000:00010000:2.0:1780298642.318322:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000a323d4b/0xaf35132e63032f2f lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [765952->770047] (req 765952->770047) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d21c6 expref: 9 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.318337:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000488a7b88/0xaf35132e63032edb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2180 expref: 6 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.318349:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000488a7b88/0xaf35132e63032edb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2180 expref: 6 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.318356:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604868480:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298642.318386:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.318960:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604868480:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298642.318976:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000a323d4b/0xaf35132e63032f2f lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [765952->770047] (req 765952->770047) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d21c6 expref: 9 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.318995:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf2445bf000, rc 0) 00000100:00100000:1.0:1780298642.319026:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255091880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609362560:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 772us (844us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.319033:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8816 00000100:00100000:3.0:1780298642.319188:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.319217:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609362688 00000100:00100000:0.0:1780298642.319221:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609362688 00010000:00010000:0.0:1780298642.319236:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000488a7b88/0xaf35132e63032edb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2180 expref: 7 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.319256:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14976 00000100:00100000:0.0:1780298642.319260:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609362688:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298642.319268:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298642.319276:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000488a7b88/0xaf35132e63032edb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2180 expref: 7 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.319285:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000488a7b88/0xaf35132e63032edb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2180 expref: 7 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.319308:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000488a7b88/0xaf35132e63032edb lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2180 expref: 7 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.319316:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(101 177) 00010000:00010000:0.0:1780298642.319318:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(101 177) 00010000:00010000:0.0:1780298642.319324:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000000a323d4b/0xaf35132e63032f2f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [765952->34320383] (req 765952->770047) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d21c6 expref: 8 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.319330:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000a323d4b/0xaf35132e63032f2f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [765952->34320383] (req 765952->770047) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d21c6 expref: 8 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.319346:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000000a323d4b/0xaf35132e63032f2f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [765952->34320383] (req 765952->770047) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d21c6 expref: 8 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.319358:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25491a680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777604868608:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298642.319385:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417540 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.320173:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25491a680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777604868608:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298642.320183:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298642.320204:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609362688:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 943us (1018us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.320209:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14976 00000100:00100000:2.0:1780298642.320410:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.320439:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609362816 00000100:00100000:0.0:1780298642.320443:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609362816 00002000:00100000:0.0:1780298642.320463:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (765952->770047) req@ffff9cf347cd7b80 x1866777609362816/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.320469:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf347cd7b80 with x1866777609362816 ext(765952->770047) 00002000:00010000:0.0:1780298642.320472:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf347cd7b80 00000100:00100000:0.0:1780298642.320488:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5959 00000100:00100000:0.0:1780298642.320492:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609362816:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298642.320616:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609362816:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 122us (204us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298642.320624:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5959 00002000:00100000:0.0:1780298642.320631:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (765952->770047) req@ffff9cf347cd7b80 x1866777609362816/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/4536 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.320644:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf347cd7b80 with x1866777609362816 ext(765952->770047) 00002000:00010000:0.0:1780298642.320647:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf347cd7b80 00000100:00100000:3.0:1780298642.321552:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.321616:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609362944 00000100:00100000:0.0:1780298642.321619:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609362944 00002000:00100000:0.0:1780298642.321634:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (770048->774143) req@ffff9cf244ada680 x1866777609362944/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.321640:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244ada680 with x1866777609362944 ext(770048->774143) 00002000:00010000:0.0:1780298642.321642:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf244ada680 00000100:00100000:0.0:1780298642.321658:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5960 00000100:00100000:0.0:1780298642.321662:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ada680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609362944:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298642.321722:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ada680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609362944:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 60us (171us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298642.321727:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5960 00002000:00100000:0.0:1780298642.321730:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (770048->774143) req@ffff9cf244ada680 x1866777609362944/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/4536 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.321735:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244ada680 with x1866777609362944 ext(770048->774143) 00002000:00010000:0.0:1780298642.321737:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf244ada680 00000100:00100000:1.0:1780298642.322627:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.322658:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609363072 00000100:00100000:2.0:1780298642.322662:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609363072 00002000:00100000:2.0:1780298642.322677:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (774144->778239) req@ffff9cf255090000 x1866777609363072/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.322683:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255090000 with x1866777609363072 ext(774144->778239) 00002000:00010000:2.0:1780298642.322686:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255090000 00000100:00100000:2.0:1780298642.322705:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5961 00000100:00100000:2.0:1780298642.322711:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255090000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609363072:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298642.322765:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255090000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609363072:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 54us (140us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298642.322769:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5961 00002000:00100000:2.0:1780298642.322774:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (774144->778239) req@ffff9cf255090000 x1866777609363072/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/4536 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.322780:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255090000 with x1866777609363072 ext(774144->778239) 00002000:00010000:2.0:1780298642.322782:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255090000 00000100:00100000:2.0:1780298642.323711:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.323786:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609363200 00000100:00100000:3.0:1780298642.323791:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609363200 00002000:00100000:3.0:1780298642.323815:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (765952->778239) req@ffff9cf36e56df80 x1866777609363200/t0(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 12776/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298642.323824:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e56df80 with x1866777609363200 ext(765952->778239) 00002000:00010000:3.0:1780298642.323827:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf36e56df80 00000100:00100000:3.0:1780298642.323848:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5962 00000100:00100000:3.0:1780298642.323854:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609363200:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:3.0:1780298642.324275:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609363200:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 419us (564us total) trans 4294969619 rc 0/0 00000100:00100000:3.0:1780298642.324283:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5962 00002000:00100000:3.0:1780298642.324301:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (765952->778239) req@ffff9cf36e56df80 x1866777609363200/t4294969619(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 12776/448 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298642.324311:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e56df80 with x1866777609363200 ext(765952->778239) 00002000:00010000:3.0:1780298642.324315:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf36e56df80 00000100:00100000:3.0:1780298642.324924:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.324953:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609363328 00000100:00100000:0.0:1780298642.324957:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609363328 00000100:00100000:0.0:1780298642.324987:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8817 00000100:00100000:0.0:1780298642.324995:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609363328:12345-192.168.203.25@tcp:16:fsx.0 00000001:00080000:1.0:1780298642.325693:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969619 is committed 00000100:00100000:0.0:1780298642.325778:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609363328:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 783us (851us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.325786:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8817 00000100:00100000:1.0:1780298642.326854:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.326891:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.326910:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609363456 00000100:00100000:2.0:1780298642.326916:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609363456 00000100:00100000:0.0:1780298642.326928:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609363584 00000100:00100000:0.0:1780298642.326934:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609363584 00000100:00100000:2.0:1780298642.326945:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8818 00000100:00100000:2.0:1780298642.326954:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255093800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609363456:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.326964:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298642.326971:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8819 00010000:00010000:2.0:1780298642.326980:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002de18a35/0xaf35132e63032f36 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d21cd expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.326984:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255093100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609363584:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.327000:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.327014:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0cae770/0xaf35132e63032f3d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d21d4 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.327035:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bf800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604868736:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.327054:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25491b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604868864:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.327087:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.327090:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.328053:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25491b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604868864:192.168.203.25@tcp:106: 00002000:00010000:3.0:1780298642.328062:0:8359:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks: 432 -> 456 00000100:00100000:3.0:1780298642.328077:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bf800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604868736:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.328079:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e0cae770/0xaf35132e63032f3d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d21d4 expref: 14 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.328099:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0cae770/0xaf35132e63032f3d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d21d4 expref: 14 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.328108:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2600, rc 0) 00010000:00010000:3.0:1780298642.328110:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002de18a35/0xaf35132e63032f36 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d21cd expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.328146:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255093100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609363584:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1167us (1256us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.328151:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8819 00010000:00010000:3.0:1780298642.328155:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002de18a35/0xaf35132e63032f36 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d21cd expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.328169:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf2445bfa00, rc 0) 00000100:00100000:3.0:1780298642.328196:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255093800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609363456:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1243us (1344us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.328203:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8818 00000100:00100000:3.0:1780298642.329230:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.329269:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609363712 00000100:00100000:1.0:1780298642.329274:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609363712 00000100:00100000:3.0:1780298642.329278:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.329315:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609363840 00000100:00100000:1.0:1780298642.329318:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609363840 00000100:00100000:2.0:1780298642.329334:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8820 00000100:00100000:1.0:1780298642.329345:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8821 00000100:00100000:2.0:1780298642.329346:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e580050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609363712:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.329357:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298642.329359:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609363840:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.329368:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000408198ca/0xaf35132e63032f44 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d21db expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.329370:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.329380:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078c7f460/0xaf35132e63032f4b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d21e2 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.329415:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bf100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604868992:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.329422:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255093480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604869120:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.329466:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.329467:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4f00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.330244:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255093480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604869120:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.330249:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bf100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604868992:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.330269:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000078c7f460/0xaf35132e63032f4b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d21e2 expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.330274:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000408198ca/0xaf35132e63032f44 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d21db expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.330304:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078c7f460/0xaf35132e63032f4b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d21e2 expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.330313:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000408198ca/0xaf35132e63032f44 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d21db expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.330316:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36c00, rc 0) 00010000:00010000:3.0:1780298642.330329:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf2445bec00, rc 0) 00000100:00100000:3.0:1780298642.330358:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e580050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609363712:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1012us (1124us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.330358:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609363840:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1000us (1077us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.330373:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8820 00000100:00100000:2.0:1780298642.330380:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8821 00000100:00100000:3.0:1780298642.331314:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.331345:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609363968 00000100:00100000:0.0:1780298642.331350:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609363968 00000100:00100000:0.0:1780298642.331385:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5963 00000100:00100000:0.0:1780298642.331391:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609363968:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298642.331531:0:8364:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:0.0:1780298642.331534:0:8364:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298642.331536:0:8364:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298642.331539:0:8364:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298642.331541:0:8364:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 1 -> 0 00000100:00100000:0.0:1780298642.331577:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609363968:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 185us (263us total) trans 4294969179 rc 0/0 00000100:00100000:0.0:1780298642.331584:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5963 00000100:00100000:1.0:1780298642.332445:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.332493:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609364096 00000100:00100000:2.0:1780298642.332498:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609364096 00000100:00100000:2.0:1780298642.332534:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8822 00000100:00100000:2.0:1780298642.332542:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255090a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609364096:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.332552:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.332565:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d1b192d6/0xaf35132e63032f52 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d21e9 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.332604:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604869248:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.332654:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.333303:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604869248:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.333327:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d1b192d6/0xaf35132e63032f52 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d21e9 expref: 7 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.333343:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d1b192d6/0xaf35132e63032f52 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d21e9 expref: 7 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.333353:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341692400, rc 0) 00000100:00100000:2.0:1780298642.333385:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255090a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609364096:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 844us (941us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.333392:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8822 00000100:00100000:3.0:1780298642.334238:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.334282:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609364224 00000100:00100000:2.0:1780298642.334288:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609364224 00000100:00100000:2.0:1780298642.334330:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8823 00000100:00100000:2.0:1780298642.334339:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609364224:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.334350:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.334359:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e06fa3fe/0xaf35132e63032f59 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d21f0 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.334388:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604869376:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.334429:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.335061:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604869376:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.335077:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e06fa3fe/0xaf35132e63032f59 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d21f0 expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.335098:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e06fa3fe/0xaf35132e63032f59 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d21f0 expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.335109:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341693e00, rc 0) 00000100:00100000:2.0:1780298642.335160:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609364224:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 823us (922us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.335167:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8823 00000100:00100000:1.0:1780298642.336033:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.336072:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609364352 00000100:00100000:0.0:1780298642.336077:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609364352 00000100:00100000:0.0:1780298642.336111:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5964 00000100:00100000:0.0:1780298642.336116:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255093b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609364352:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298642.336277:0:19950:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4928525 -> 4928525 00002000:00010000:0.0:1780298642.336280:0:19950:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298642.336283:0:19950:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298642.336285:0:19950:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298642.336287:0:19950:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 456 -> 456 00000100:00100000:0.0:1780298642.336329:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255093b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609364352:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 212us (297us total) trans 4294969620 rc 0/0 00000100:00100000:0.0:1780298642.336337:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5964 00000100:00100000:3.0:1780298642.337193:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.337227:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609364480 00000100:00100000:3.0:1780298642.337228:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.337232:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609364480 00000100:00100000:2.0:1780298642.337259:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609364608 00000100:00100000:2.0:1780298642.337270:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609364608 00000100:00100000:1.0:1780298642.337270:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8824 00000100:00100000:1.0:1780298642.337279:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609364480:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.337286:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.337308:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003c8dab73/0xaf35132e63032f60 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d21f7 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.337322:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8825 00000100:00100000:2.0:1780298642.337332:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609364608:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.337341:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298642.337349:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255092d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604869504:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.337353:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000036bac5fe/0xaf35132e63032f67 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d21fe expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.337386:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604869632:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.337409:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4f00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.337417:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.338218:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255092d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604869504:192.168.203.25@tcp:106: 00002000:00010000:2.0:1780298642.338218:0:12101:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks: 0 -> 1 00000100:00100000:2.0:1780298642.338235:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604869632:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.338252:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003c8dab73/0xaf35132e63032f60 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d21f7 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.338257:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000036bac5fe/0xaf35132e63032f67 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d21fe expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.338283:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003c8dab73/0xaf35132e63032f60 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d21f7 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.338283:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000036bac5fe/0xaf35132e63032f67 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d21fe expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.338320:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36800, rc 0) 00010000:00010000:2.0:1780298642.338321:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341693400, rc 0) 00000100:00100000:3.0:1780298642.338359:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609364480:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1080us (1166us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.338360:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609364608:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1028us (1131us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.338376:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8824 00000100:00100000:2.0:1780298642.338385:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8825 00000100:00100000:0.0:1780298642.339426:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.339465:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609364736 00000100:00100000:2.0:1780298642.339471:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609364736 00000100:00100000:0.0:1780298642.339487:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.339525:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609364864 00000100:00100000:2.0:1780298642.339529:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609364864 00000100:00100000:2.0:1780298642.339560:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8826 00000100:00100000:1.0:1780298642.339570:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8827 00000100:00100000:2.0:1780298642.339572:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609364736:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.339584:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298642.339587:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254918a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609364864:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.339597:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.339599:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e63032f6e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2205 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.339608:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032f75 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d220c expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.339641:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604869760:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.339655:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255091500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604869888:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.339700:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.339708:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4f00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.340562:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255091500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604869888:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.340582:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032f75 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d220c expref: 14 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.340598:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032f75 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d220c expref: 14 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.340609:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37000, rc 0) 00000100:00100000:1.0:1780298642.340642:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254918a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609364864:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1062us (1155us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.340649:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8827 00000100:00100000:2.0:1780298642.340742:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604869760:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.340759:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e63032f6e lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2205 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.340770:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e63032f6e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2205 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.340778:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156400, rc 0) 00000100:00100000:2.0:1780298642.340806:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609364736:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1238us (1382us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.340813:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8826 00000100:00100000:0.0:1780298642.342078:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.342116:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609364992 00000100:00100000:3.0:1780298642.342150:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609364992 00000100:00100000:3.0:1780298642.342190:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8828 00000100:00100000:3.0:1780298642.342201:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:29360:x1866777609364992:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.342213:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.342226:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e63032f7c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2213 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.342250:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e63032ee2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2187 expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.342260:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(72 130) 00010000:00010000:3.0:1780298642.342265:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(73 130) 00010000:00010000:3.0:1780298642.342282:0:20252:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e63032f7c lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3010560->3014655] (req 3010560->3014655) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2213 expref: 14 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.342324:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e63032ee2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2187 expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.342339:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e63032ee2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2187 expref: 8 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.342353:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604870016:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298642.342403:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.343106:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604870016:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298642.343141:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e63032f7c lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3010560->3014655] (req 3010560->3014655) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2213 expref: 14 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.343152:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2000, rc 0) 00000100:00100000:3.0:1780298642.343177:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:29360:x1866777609364992:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 980us (1105us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298642.343182:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8828 00000100:00100000:1.0:1780298642.343527:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.343571:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609365120 00000100:00100000:0.0:1780298642.343577:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609365120 00002000:00100000:0.0:1780298642.343609:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (2793472->2805759) req@ffff9cf375f56450 x1866777609365120/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 12776/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.343619:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf375f56450 with x1866777609365120 ext(2793472->2805759) 00010000:00010000:0.0:1780298642.343625:0:8362:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e63032ee2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2187 expref: 9 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.343638:0:8362:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e63032ee2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2187 expref: 9 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1780298642.343645:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9cf375f56450 00000100:00100000:0.0:1780298642.343674:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 183 00000100:00100000:0.0:1780298642.343681:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf375f56450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609365120:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:0.0:1780298642.344280:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf375f56450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609365120:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 598us (753us total) trans 4294969180 rc 0/0 00000100:00100000:0.0:1780298642.344287:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 183 00002000:00100000:0.0:1780298642.344306:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (2793472->2805759) req@ffff9cf375f56450 x1866777609365120/t4294969180(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 12776/448 e 0 to 0 dl 1780298653 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.344315:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf375f56450 with x1866777609365120 ext(2793472->2805759) 00010000:00010000:0.0:1780298642.344320:0:8362:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e63032ee2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2187 expref: 11 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.344333:0:8362:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e63032ee2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2187 expref: 11 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1780298642.344340:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9cf375f56450 00000100:00100000:0.0:1780298642.345211:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.345243:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609365248 00000100:00100000:1.0:1780298642.345247:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609365248 00010000:00010000:1.0:1780298642.345267:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e63032ee2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2187 expref: 11 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.345308:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14977 00000100:00100000:1.0:1780298642.345318:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609365248:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298642.345329:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:1.0:1780298642.345339:0:20253:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 1 -> 24 00010000:00010000:1.0:1780298642.345345:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e63032ee2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2187 expref: 11 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.345354:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e63032ee2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2187 expref: 11 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.345371:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e63032ee2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2187 expref: 11 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.345380:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(73 130) 00010000:00010000:1.0:1780298642.345383:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(73 130) 00010000:00010000:1.0:1780298642.345390:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e63032f7c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3010560->36564991] (req 3010560->3014655) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2213 expref: 13 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.345396:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e63032f7c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3010560->36564991] (req 3010560->3014655) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2213 expref: 13 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.345422:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e63032f7c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3010560->36564991] (req 3010560->3014655) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2213 expref: 13 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.345431:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255091500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:20253:1866777604870144:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298642.345478:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4f00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.346348:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255091500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:20253:1866777604870144:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298642.346357:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:1.0:1780298642.346368:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298642.346403:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609365248:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1087us (1191us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.346405:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609365376 00000100:00100000:2.0:1780298642.346412:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609365376 00000100:00100000:1.0:1780298642.346416:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14977 00000100:00100000:2.0:1780298642.346458:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5965 00000100:00100000:2.0:1780298642.346468:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2398:x1866777609365376:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298642.346591:0:19928:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:2.0:1780298642.346595:0:19928:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.346597:0:19928:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.346599:0:19928:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.346601:0:19928:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 24 -> 24 00000100:00100000:2.0:1780298642.346631:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2398:x1866777609365376:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 166us (278us total) trans 4294969181 rc 0/0 00000100:00100000:2.0:1780298642.346637:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5965 00000100:00100000:3.0:1780298642.347743:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.347772:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.347795:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609365504 00000100:00100000:2.0:1780298642.347801:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609365504 00000100:00100000:0.0:1780298642.347823:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609365632 00000100:00100000:0.0:1780298642.347829:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609365632 00000100:00100000:2.0:1780298642.347842:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8829 00000100:00100000:2.0:1780298642.347851:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609365504:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.347862:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298642.347868:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8830 00010000:00010000:2.0:1780298642.347876:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63032f83 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d221a expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.347882:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2398:x1866777609365632:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.347896:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.347926:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000039c5f58c/0xaf35132e63032f8a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2221 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.347926:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604870272:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.347967:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244adad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604870400:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.347990:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.347999:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.348974:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604870272:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.348986:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244adad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604870400:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.348998:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63032f83 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d221a expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.349007:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000039c5f58c/0xaf35132e63032f8a lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2221 expref: 16 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.349015:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63032f83 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d221a expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.349028:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157c00, rc 0) 00010000:00010000:0.0:1780298642.349032:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000039c5f58c/0xaf35132e63032f8a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2221 expref: 16 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.349051:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3200, rc 0) 00000100:00100000:2.0:1780298642.349059:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609365504:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1210us (1317us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.349064:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8829 00000100:00100000:0.0:1780298642.349079:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2398:x1866777609365632:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1202us (1306us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.349087:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8830 00000100:00100000:0.0:1780298642.350201:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.350236:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609365760 00000100:00100000:2.0:1780298642.350241:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609365760 00000100:00100000:2.0:1780298642.350278:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8831 00000100:00100000:2.0:1780298642.350286:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609365760:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.350304:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.350314:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e63032f91 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2228 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.350324:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.350350:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604870528:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.350356:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609365888 00000100:00100000:0.0:1780298642.350360:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609365888 00000100:00100000:0.0:1780298642.350389:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8832 00000100:00100000:2.0:1780298642.350393:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.350399:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2397:x1866777609365888:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.350408:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.350418:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008361787d/0xaf35132e63032f98 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d222f expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.350445:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254919880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604870656:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.350468:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.351181:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604870528:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.351204:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e63032f91 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2228 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.351225:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e63032f91 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2228 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.351236:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156400, rc 0) 00000100:00100000:2.0:1780298642.351278:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609365760:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 990us (1076us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.351283:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254919880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604870656:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.351302:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8831 00010000:00010000:0.0:1780298642.351321:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008361787d/0xaf35132e63032f98 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d222f expref: 16 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.351338:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008361787d/0xaf35132e63032f98 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d222f expref: 16 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.351350:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2800, rc 0) 00000100:00100000:0.0:1780298642.351384:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2397:x1866777609365888:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 988us (1065us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.351391:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8832 00000100:00100000:3.0:1780298642.352614:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.352640:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.352655:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609366016 00000100:00100000:0.0:1780298642.352661:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609366016 00000100:00100000:2.0:1780298642.352670:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609366144 00000100:00100000:2.0:1780298642.352675:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609366144 00000100:00100000:0.0:1780298642.352696:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8833 00000100:00100000:3.0:1780298642.352704:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8834 00000100:00100000:0.0:1780298642.352706:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609366016:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298642.352713:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609366144:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.352716:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.352719:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.352728:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e63032f9f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d223d expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.352728:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a269a887/0xaf35132e63032fa6 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2236 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.352761:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604870912:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.352761:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254919880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604870784:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.352810:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.352810:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.353650:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604870912:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.353650:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254919880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604870784:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.353670:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e63032f9f lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d223d expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.353670:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a269a887/0xaf35132e63032fa6 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2236 expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.353687:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e63032f9f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d223d expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.353687:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a269a887/0xaf35132e63032fa6 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2236 expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.353698:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3e00, rc 0) 00010000:00010000:3.0:1780298642.353699:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3800, rc 0) 00000100:00100000:0.0:1780298642.353727:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609366016:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1024us (1114us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.353728:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609366144:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1017us (1086us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.353733:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8833 00000100:00100000:3.0:1780298642.353742:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8834 00000100:00100000:1.0:1780298642.354869:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.354891:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.354907:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609366400 00000100:00100000:0.0:1780298642.354913:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609366400 00000100:00100000:2.0:1780298642.354929:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609366272 00000100:00100000:2.0:1780298642.354933:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609366272 00000100:00100000:0.0:1780298642.354952:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8835 00000100:00100000:2.0:1780298642.354967:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8836 00000100:00100000:0.0:1780298642.354969:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255091500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609366400:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298642.354977:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255090700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609366272:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.354981:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.354984:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.354994:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004ba675d2/0xaf35132e63032fb4 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2244 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.354994:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000658b0d9c/0xaf35132e63032fad lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d224b expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.355045:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254919880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604871040:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.355054:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604871168:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.355088:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.355089:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.355853:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254919880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604871040:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.355861:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604871168:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.355873:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000658b0d9c/0xaf35132e63032fad lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d224b expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.355884:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004ba675d2/0xaf35132e63032fb4 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2244 expref: 7 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.355886:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000658b0d9c/0xaf35132e63032fad lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d224b expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.355908:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2c00, rc 0) 00010000:00010000:2.0:1780298642.355912:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004ba675d2/0xaf35132e63032fb4 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2244 expref: 7 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.355921:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156e00, rc 0) 00000100:00100000:0.0:1780298642.355936:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255091500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609366400:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 977us (1068us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.355945:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8835 00000100:00100000:2.0:1780298642.355946:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255090700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609366272:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 969us (1050us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.355953:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8836 00000100:00100000:1.0:1780298642.356921:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.356957:0:6480:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609366656 00000100:00100000:2.0:1780298642.356961:0:6480:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609366656 00000100:00100000:2.0:1780298642.356990:0:6480:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 134 00000100:00100000:2.0:1780298642.356996:0:6480:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255091c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+99:29360:x1866777609366656:12345-192.168.203.25@tcp:35:fsx.0 00000100:00100000:2.0:1780298642.357258:0:6480:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255091c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+100:29360:x1866777609366656:12345-192.168.203.25@tcp:35:fsx.0 Request processed in 257us (333us total) trans 4294991008 rc 0/0 00000100:00100000:2.0:1780298642.357264:0:6480:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 134 00000100:00100000:1.0:1780298642.358260:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.358309:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609366912 00000100:00100000:0.0:1780298642.358313:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609366912 00000100:00100000:0.0:1780298642.358347:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61929 00000100:00100000:0.0:1780298642.358353:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255091880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+100:29360:x1866777609366912:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.358363:0:6477:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.358377:0:6477:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2a00/0xaf35132e63032fbb lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2252 expref: -99 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.358420:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2e00/0xaf35132e63032fc2 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.358431:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2e00/0xaf35132e63032fc2 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.358439:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2e00/0xaf35132e63032fc2 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.358449:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2000/0xaf35132e63032fc9 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5950b203 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.358453:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2000/0xaf35132e63032fc9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5950b203 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.358457:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2000/0xaf35132e63032fc9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5950b203 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.358531:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3800/0xaf35132e63032fd0 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 55 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.358538:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3800/0xaf35132e63032fd0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 55 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.358542:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3800/0xaf35132e63032fd0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 55 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1780298642.358548:0:6477:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 9122829 00010000:00010000:0.0:1780298642.358597:0:6477:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2e00/0xaf35132e63032fc2 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.358611:0:6477:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2e00/0xaf35132e63032fc2 lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.358616:0:6477:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2e00/0xaf35132e63032fc2 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.358623:0:6477:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2e00/0xaf35132e63032fc2 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.358629:0:6477:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf344ff2e00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:0.0:1780298642.358634:0:6477:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2e00/0xaf35132e63032fc2 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.358641:0:6477:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2e00/0xaf35132e63032fc2 lrc: 5/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.358651:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9cf3600ff300) 00010000:00010000:0.0:1780298642.358654:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1780298642.358659:0:6477:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2e00/0xaf35132e63032fc2 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.358665:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2e00/0xaf35132e63032fc2 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.358677:0:6477:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2000/0xaf35132e63032fc9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5950b203 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.358684:0:6477:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2000/0xaf35132e63032fc9 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5950b203 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.358690:0:6477:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2000/0xaf35132e63032fc9 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5950b203 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.358711:0:6477:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf344ff2000 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:0.0:1780298642.358713:0:6477:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2000/0xaf35132e63032fc9 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5950b203 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.358719:0:6477:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2000/0xaf35132e63032fc9 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5950b203 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.358727:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5950b203 (ffff9cf37338ba00) 00010000:00010000:0.0:1780298642.358729:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1780298642.358732:0:6477:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2000/0xaf35132e63032fc9 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5950b203 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.358737:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2000/0xaf35132e63032fc9 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5950b203 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.358771:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2a00/0xaf35132e63032fbb lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2252 expref: 103 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.358785:0:6477:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3800/0xaf35132e63032fd0 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 55 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2252 expref: 102 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.358793:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298642.358796:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1780298642.358798:0:6477:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3800, rc 0) 00000100:00100000:0.0:1780298642.358835:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255091880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+102:29360:x1866777609366912:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 482us (576us total) trans 4294991009 rc 0/0 00000100:00100000:0.0:1780298642.358842:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61929 00000100:00100000:1.0:1780298642.359963:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.360031:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609367040 00000100:00100000:3.0:1780298642.360036:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609367040 00000100:00100000:3.0:1780298642.360071:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 2223 00000100:00100000:3.0:1780298642.360079:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255093100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+102:2399:x1866777609367040:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298642.360087:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298642.360099:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3800/0xaf35132e63032fd0 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 56 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2252 expref: 102 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.360112:0:6462:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00000100:00100000:1.0:1780298642.360112:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:3.0:1780298642.360116:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298642.360166:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609367168 00000100:00100000:2.0:1780298642.360170:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609367168 00000100:00100000:2.0:1780298642.360196:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8837 00000100:00100000:2.0:1780298642.360201:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255093800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609367168:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.360208:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.360216:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e63032fde lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2259 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.360234:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255093100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+101:2399:x1866777609367040:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 155us (273us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298642.360240:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 2223 00000100:00100000:2.0:1780298642.360254:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604871296:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.360309:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.360364:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.360415:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609367296 00000100:00100000:3.0:1780298642.360418:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609367296 00000100:00100000:3.0:1780298642.360438:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8838 00000100:00100000:3.0:1780298642.360442:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254919880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2397:x1866777609367296:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.360448:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.360456:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e63032fe5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2260 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.360488:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604871424:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.360523:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.361009:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604871296:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.361028:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e63032fde lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2259 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.361041:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e63032fde lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2259 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.361058:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156400, rc 0) 00000100:00100000:2.0:1780298642.361093:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255093800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609367168:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 892us (990us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.361097:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604871424:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.361101:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8837 00010000:00010000:3.0:1780298642.361182:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e63032fe5 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2260 expref: 16 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.361198:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e63032fe5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2260 expref: 16 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.361207:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3800, rc 0) 00000100:00100000:2.0:1780298642.361255:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254919880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2397:x1866777609367296:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 811us (893us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.361261:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8838 00000100:00100000:0.0:1780298642.362195:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.362247:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609367424 00000100:00100000:3.0:1780298642.362252:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609367424 00000100:00100000:3.0:1780298642.362287:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8839 00000100:00100000:3.0:1780298642.362306:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254918000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609367424:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.362314:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.362324:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000077813dbd/0xaf35132e63032fec lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2267 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.362334:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.362358:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604871552:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.362378:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609367552 00000100:00100000:2.0:1780298642.362384:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609367552 00000100:00100000:2.0:1780298642.362415:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8840 00000100:00100000:2.0:1780298642.362426:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255093480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2399:x1866777609367552:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298642.362430:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00010000:00010000:2.0:1780298642.362435:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.362446:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004ba675d2/0xaf35132e63032ff3 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d226e expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.362477:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604871680:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.362515:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.363067:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604871552:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.363083:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000077813dbd/0xaf35132e63032fec lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2267 expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.363097:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000077813dbd/0xaf35132e63032fec lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2267 expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.363192:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2200, rc 0) 00000100:00100000:3.0:1780298642.363227:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254918000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609367424:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 924us (1034us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.363232:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8839 00000100:00100000:2.0:1780298642.363268:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604871680:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.363286:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004ba675d2/0xaf35132e63032ff3 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d226e expref: 16 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.363318:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004ba675d2/0xaf35132e63032ff3 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d226e expref: 16 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.363335:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156e00, rc 0) 00000100:00100000:2.0:1780298642.363362:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255093480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2399:x1866777609367552:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 940us (1034us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.363369:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8840 00000100:00100000:0.0:1780298642.364352:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.364388:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609367680 00000100:00100000:1.0:1780298642.364393:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609367680 00000100:00100000:1.0:1780298642.364424:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8841 00000100:00100000:1.0:1780298642.364434:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254918e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:29360:x1866777609367680:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.364444:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.364453:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032ffa lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2275 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.364470:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e63032f7c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3010560->36564991] (req 3010560->3014655) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2213 expref: 14 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.364477:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(73 130) 00010000:00010000:1.0:1780298642.364480:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(74 130) 00010000:00010000:1.0:1780298642.364483:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032ffa lrc: 4/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3452928->3465215] (req 3452928->3465215) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2275 expref: 11 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.364508:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e63032f7c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3010560->36564991] (req 3010560->3014655) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2213 expref: 14 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.364522:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e63032f7c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3010560->36564991] (req 3010560->3014655) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2213 expref: 14 pid: 20252 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.364535:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255092300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604871808:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298642.364580:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4f00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.365122:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255092300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604871808:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298642.365181:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032ffa lrc: 3/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3452928->3465215] (req 3452928->3465215) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2275 expref: 11 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.365193:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37000, rc 0) 00000100:00100000:2.0:1780298642.365220:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254918e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:29360:x1866777609367680:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 789us (868us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.365227:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8841 00000100:00100000:3.0:1780298642.365419:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.365455:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609367808 00000100:00100000:1.0:1780298642.365459:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609367808 00010000:00010000:1.0:1780298642.365474:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e63032f7c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3010560->36564991] (req 3010560->3014655) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2213 expref: 15 pid: 20252 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.365499:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14978 00000100:00100000:1.0:1780298642.365504:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b03850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2400:x1866777609367808:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298642.365512:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298642.365521:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e63032f7c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3010560->36564991] (req 3010560->3014655) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2213 expref: 15 pid: 20252 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.365529:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e63032f7c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3010560->36564991] (req 3010560->3014655) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2213 expref: 15 pid: 20252 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.365540:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e63032f7c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3010560->36564991] (req 3010560->3014655) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2213 expref: 15 pid: 20252 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.365548:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(74 130) 00010000:00010000:1.0:1780298642.365550:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(74 130) 00010000:00010000:1.0:1780298642.365556:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032ffa lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3452928->3465215) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2275 expref: 10 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.365562:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032ffa lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3452928->3465215) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2275 expref: 10 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.365576:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032ffa lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3452928->3465215) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2275 expref: 10 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.365583:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6461:1866777604871936:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298642.365615:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4f00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.366239:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6461:1866777604871936:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298642.366250:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298642.366274:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b03850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609367808:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 769us (855us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.366283:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14978 00000100:00100000:3.0:1780298642.366780:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.366822:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609367936 00000100:00100000:0.0:1780298642.366826:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609367936 00002000:00100000:0.0:1780298642.366850:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x12:0x0] (3452928->3465215) req@ffff9cf244ad8e00 x1866777609367936/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.366857:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244ad8e00 with x1866777609367936 ext(3452928->3465215) 00002000:00010000:0.0:1780298642.366860:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf244ad8e00 00000100:00100000:0.0:1780298642.366877:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5966 00000100:00100000:0.0:1780298642.366882:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609367936:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298642.367020:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609367936:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 137us (239us total) trans 0 rc 12288/12288 00000100:00100000:0.0:1780298642.367028:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5966 00002000:00100000:0.0:1780298642.367035:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x12:0x0] (3452928->3465215) req@ffff9cf244ad8e00 x1866777609367936/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/12728 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 12288/12288 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.367043:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244ad8e00 with x1866777609367936 ext(3452928->3465215) 00002000:00010000:0.0:1780298642.367047:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf244ad8e00 00000100:00100000:3.0:1780298642.369159:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.369203:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609368064 00000100:00100000:2.0:1780298642.369210:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609368064 00000100:00100000:2.0:1780298642.369247:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8842 00000100:00100000:2.0:1780298642.369257:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609368064:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.369270:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.369287:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e63033001 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d227c expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.369343:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604872064:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.369394:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.370090:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604872064:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.370111:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e63033001 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d227c expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.370145:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e63033001 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d227c expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.370156:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156400, rc 0) 00000100:00100000:2.0:1780298642.370186:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609368064:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 931us (1096us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.370193:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8842 00000100:00100000:3.0:1780298642.371167:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.371205:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609368192 00000100:00100000:0.0:1780298642.371211:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609368192 00000100:00100000:0.0:1780298642.371245:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8843 00000100:00100000:0.0:1780298642.371255:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609368192:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.371265:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.371275:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e63033008 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2283 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.371322:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254918700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604872192:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.371360:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417540 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.372014:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254918700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604872192:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.372032:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e63033008 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2283 expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.372046:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e63033008 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2283 expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.372056:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2a00, rc 0) 00000100:00100000:0.0:1780298642.372084:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609368192:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 832us (919us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.372091:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8843 00000100:00100000:1.0:1780298642.373164:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.373202:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609368320 00000100:00100000:2.0:1780298642.373207:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609368320 00000100:00100000:2.0:1780298642.373239:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8844 00000100:00100000:2.0:1780298642.373247:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255093100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609368320:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.373257:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.373275:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004ba675d2/0xaf35132e6303300f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d228a expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.373318:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604872320:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.373362:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.374048:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604872320:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.374062:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004ba675d2/0xaf35132e6303300f lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d228a expref: 7 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.374073:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004ba675d2/0xaf35132e6303300f lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d228a expref: 7 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.374080:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156e00, rc 0) 00000100:00100000:1.0:1780298642.374103:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255093100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609368320:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 859us (943us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.374109:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8844 00000100:00100000:3.0:1780298642.375010:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.375045:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609368448 00000100:00100000:2.0:1780298642.375050:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609368448 00000100:00100000:2.0:1780298642.375081:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8845 00000100:00100000:2.0:1780298642.375089:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:29360:x1866777609368448:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.375099:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.375109:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000700ba1da/0xaf35132e63033016 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2291 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.375153:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032ffa lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3452928->3465215) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2275 expref: 12 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.375164:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(74 130) 00010000:00010000:2.0:1780298642.375169:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(74 130) 00010000:00010000:2.0:1780298642.375174:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000700ba1da/0xaf35132e63033016 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2334720->2342911] (req 2334720->2342911) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2291 expref: 12 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.375203:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032ffa lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3452928->3465215) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2275 expref: 12 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.375217:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032ffa lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3452928->3465215) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2275 expref: 12 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.375228:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604872448:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298642.375267:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.375880:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604872448:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298642.375893:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000700ba1da/0xaf35132e63033016 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2334720->2342911] (req 2334720->2342911) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2291 expref: 12 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.375900:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156000, rc 0) 00000100:00100000:2.0:1780298642.375923:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:29360:x1866777609368448:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 836us (915us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.375930:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8845 00000100:00100000:1.0:1780298642.376184:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.376214:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609368576 00000100:00100000:2.0:1780298642.376218:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609368576 00010000:00010000:2.0:1780298642.376232:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032ffa lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3452928->3465215) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2275 expref: 12 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.376255:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14979 00000100:00100000:2.0:1780298642.376262:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255092680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609368576:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298642.376272:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298642.376285:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032ffa lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3452928->3465215) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2275 expref: 12 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.376305:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032ffa lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3452928->3465215) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2275 expref: 12 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.376317:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63032ffa lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3452928->3465215) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2275 expref: 12 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.376324:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(74 130) 00010000:00010000:2.0:1780298642.376327:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(74 130) 00010000:00010000:2.0:1780298642.376332:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000700ba1da/0xaf35132e63033016 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2334720->35889151] (req 2334720->2342911) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2291 expref: 11 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.376339:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000700ba1da/0xaf35132e63033016 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2334720->35889151] (req 2334720->2342911) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2291 expref: 11 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.376357:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000700ba1da/0xaf35132e63033016 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2334720->35889151] (req 2334720->2342911) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2291 expref: 11 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.376364:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bc700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:7400:1866777604872576:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298642.376390:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.377027:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bc700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:7400:1866777604872576:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298642.377034:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298642.377055:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255092680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609368576:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 794us (871us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.377063:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14979 00000100:00100000:0.0:1780298642.377306:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.377341:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609368704 00000100:00100000:2.0:1780298642.377345:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609368704 00000100:00100000:2.0:1780298642.377383:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5967 00000100:00100000:2.0:1780298642.377390:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254918700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609368704:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298642.377526:0:8364:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:2.0:1780298642.377529:0:8364:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.377530:0:8364:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.377532:0:8364:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.377533:0:8364:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 24 -> 40 00000100:00100000:2.0:1780298642.377555:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254918700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609368704:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 167us (253us total) trans 4294969182 rc 0/0 00000100:00100000:2.0:1780298642.377559:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5967 00000100:00100000:1.0:1780298642.378563:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.378595:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609368832 00000100:00100000:2.0:1780298642.378600:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609368832 00000100:00100000:2.0:1780298642.378627:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8846 00000100:00100000:2.0:1780298642.378634:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255091180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609368832:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.378645:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.378658:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009c278703/0xaf35132e6303301d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2298 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.378691:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.378703:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604872704:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.378723:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609368960 00000100:00100000:3.0:1780298642.378729:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609368960 00000100:00100000:2.0:1780298642.378749:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4a80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.378801:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8847 00000100:00100000:3.0:1780298642.378810:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609368960:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.378819:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.378827:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e63033024 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d229f expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.378883:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ada680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604872832:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.378907:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.379395:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604872704:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.379413:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009c278703/0xaf35132e6303301d lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2298 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.379429:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009c278703/0xaf35132e6303301d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2298 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.379439:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156600, rc 0) 00000100:00100000:2.0:1780298642.379467:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255091180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609368832:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 834us (906us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.379474:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8846 00000100:00100000:0.0:1780298642.379584:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ada680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604872832:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.379599:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e63033024 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d229f expref: 15 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.379612:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e63033024 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d229f expref: 15 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.379622:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2200, rc 0) 00000100:00100000:0.0:1780298642.379651:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609368960:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 843us (962us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.379657:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8847 00000100:00100000:3.0:1780298642.380558:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.380596:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609369088 00000100:00100000:1.0:1780298642.380601:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609369088 00000100:00100000:1.0:1780298642.380632:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8848 00000100:00100000:1.0:1780298642.380639:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609369088:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.380649:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.380660:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e6303302b lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d22a6 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.380686:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.380697:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255092300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604872960:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.380720:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609369216 00000100:00100000:2.0:1780298642.380725:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609369216 00000100:00100000:1.0:1780298642.380739:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3629a4600 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.380756:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8849 00000100:00100000:2.0:1780298642.380764:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ada300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2398:x1866777609369216:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.380772:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.380782:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bed20494/0xaf35132e63033032 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d22ad expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.380812:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604873088:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.380843:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.381379:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255092300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604872960:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.381393:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e6303302b lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22a6 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.381404:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e6303302b lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22a6 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.381410:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37200, rc 0) 00000100:00100000:1.0:1780298642.381434:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609369088:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 796us (877us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.381441:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8848 00000100:00100000:2.0:1780298642.381628:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604873088:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.381643:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bed20494/0xaf35132e63033032 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22ad expref: 15 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.381657:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bed20494/0xaf35132e63033032 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22ad expref: 15 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.381667:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157000, rc 0) 00000100:00100000:2.0:1780298642.381696:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ada300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2398:x1866777609369216:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 934us (1010us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.381704:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8849 00000100:00100000:0.0:1780298642.382573:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.382610:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609369344 00000100:00100000:2.0:1780298642.382615:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609369344 00000100:00100000:2.0:1780298642.382639:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8850 00000100:00100000:2.0:1780298642.382644:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:29360:x1866777609369344:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.382650:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.382657:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000004b2c7ae/0xaf35132e63033039 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d22b4 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.382671:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(74 130) 00010000:00010000:2.0:1780298642.382674:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(74 130) 00010000:00010000:2.0:1780298642.382681:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000004b2c7ae/0xaf35132e63033039 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [638976->2334719] (req 638976->643071) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22b4 expref: 13 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.382692:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000004b2c7ae/0xaf35132e63033039 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [638976->2334719] (req 638976->643071) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22b4 expref: 13 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.382697:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156400, rc 0) 00000100:00100000:2.0:1780298642.382718:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:29360:x1866777609369344:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 74us (149us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.382722:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8850 00000100:00100000:3.0:1780298642.383638:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.383674:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609369472 00000100:00100000:0.0:1780298642.383680:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609369472 00002000:00100000:0.0:1780298642.383717:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (638976->643071) req@ffff9cf244ada680 x1866777609369472/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.383727:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244ada680 with x1866777609369472 ext(638976->643071) 00002000:00010000:0.0:1780298642.383732:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf244ada680 00000100:00100000:0.0:1780298642.383753:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5968 00000100:00100000:0.0:1780298642.383764:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ada680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2399:x1866777609369472:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298642.383912:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ada680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2399:x1866777609369472:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 149us (274us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298642.383920:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5968 00002000:00100000:0.0:1780298642.383926:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (638976->643071) req@ffff9cf244ada680 x1866777609369472/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/4536 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.383935:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244ada680 with x1866777609369472 ext(638976->643071) 00002000:00010000:0.0:1780298642.383939:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf244ada680 00000100:00100000:2.0:1780298642.384921:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.384952:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609369600 00000100:00100000:0.0:1780298642.384957:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609369600 00000100:00100000:0.0:1780298642.384985:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8851 00000100:00100000:0.0:1780298642.384990:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609369600:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.384998:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.385009:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bde78cfa/0xaf35132e63033040 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d22bb expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.385025:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.385048:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25491ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604873216:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.385071:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609369728 00000100:00100000:1.0:1780298642.385080:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609369728 00000100:00100000:0.0:1780298642.385098:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484173c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.385145:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8852 00000100:00100000:1.0:1780298642.385156:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b68380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2397:x1866777609369728:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.385165:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.385176:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003c8dab73/0xaf35132e63033047 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d22c2 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.385208:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255092300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604873344:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.385253:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3629a4600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.385825:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25491ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604873216:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.385855:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bde78cfa/0xaf35132e63033040 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22bb expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.385877:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bde78cfa/0xaf35132e63033040 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22bb expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.385891:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3a00, rc 0) 00000100:00100000:0.0:1780298642.385923:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609369600:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 931us (1003us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.385931:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8851 00000100:00100000:1.0:1780298642.385943:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255092300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604873344:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.385963:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003c8dab73/0xaf35132e63033047 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22c2 expref: 15 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.385979:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003c8dab73/0xaf35132e63033047 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22c2 expref: 15 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.385990:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36800, rc 0) 00000100:00100000:1.0:1780298642.386019:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b68380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2397:x1866777609369728:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 866us (994us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.386025:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8852 00000100:00100000:2.0:1780298642.386984:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.387034:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609369856 00000100:00100000:1.0:1780298642.387040:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609369856 00000100:00100000:2.0:1780298642.387063:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.387185:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609369984 00000100:00100000:1.0:1780298642.387189:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609369984 00000100:00100000:0.0:1780298642.387200:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8853 00000100:00100000:0.0:1780298642.387212:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609369856:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.387222:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298642.387226:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8854 00010000:00010000:0.0:1780298642.387233:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e6303304e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d22c9 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.387238:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609369984:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.387251:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.387261:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078c7f460/0xaf35132e63033055 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d22d0 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.387280:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25491ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604873472:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.387304:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255092300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604873600:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.387351:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3629a4600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.387352:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484173c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.388115:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25491ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604873472:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.388159:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e6303304e lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22c9 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.388173:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255092300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604873600:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.388179:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e6303304e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22c9 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.388190:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3200, rc 0) 00010000:00010000:1.0:1780298642.388192:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000078c7f460/0xaf35132e63033055 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22d0 expref: 15 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.388204:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078c7f460/0xaf35132e63033055 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22d0 expref: 15 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.388212:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36c00, rc 0) 00000100:00100000:0.0:1780298642.388222:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609369856:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1015us (1239us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.388228:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8853 00000100:00100000:1.0:1780298642.388233:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609369984:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 998us (1169us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.388239:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8854 00000100:00100000:3.0:1780298642.389379:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.389416:0:7411:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609370240 00000100:00100000:1.0:1780298642.389423:0:7411:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609370240 00000100:00100000:1.0:1780298642.389458:0:7411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61930 00000100:00100000:1.0:1780298642.389464:0:7411:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b69c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+91:29360:x1866777609370240:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:1.0:1780298642.389502:0:7411:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298642.389519:0:7411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be600/0xaf35132e63032f1a lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 55 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d21b1 expref: 91 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.389530:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298642.389533:0:7411:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298642.389683:0:7411:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37800/0xaf35132e6303305c lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 54 type: IBT flags: 0x40000000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.389706:0:7411:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be000/0xaf35132e63032ef0 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 54 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d218e expref: 100 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.389721:0:7411:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37800/0xaf35132e6303305c lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 54 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.389747:0:7411:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be000/0xaf35132e63032ef0 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 54 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d218e expref: 100 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.389760:0:7411:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be000/0xaf35132e63032ef0 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 54 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d218e expref: 100 pid: 6476 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.389771:0:7411:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255092300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7411:1866777604873728:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298642.389823:0:7411:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3629a4180 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.390435:0:7411:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255092300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7411:1866777604873728:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298642.390449:0:7411:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37800/0xaf35132e6303305c lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 54 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.390630:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.390673:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609370368 00000100:00100000:2.0:1780298642.390678:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609370368 00010000:00010000:2.0:1780298642.390699:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be000/0xaf35132e63032ef0 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 54 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d218e expref: 101 pid: 6476 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.390729:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14980 00000100:00100000:2.0:1780298642.390738:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b6b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+101:2400:x1866777609370368:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298642.390747:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298642.390752:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be000/0xaf35132e63032ef0 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 55 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d218e expref: 101 pid: 6476 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.390764:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be000/0xaf35132e63032ef0 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 55 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d218e expref: 101 pid: 6476 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.390783:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be000/0xaf35132e63032ef0 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 55 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d218e expref: 101 pid: 6476 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.390794:0:6463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.390798:0:6463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37800/0xaf35132e6303305c lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 54 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.390809:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37800/0xaf35132e6303305c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 54 type: IBT flags: 0x40290000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.390818:0:6463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37800/0xaf35132e6303305c lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 54 type: IBT flags: 0x40290000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.390842:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298642.390871:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b6b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+100:2400:x1866777609370368:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 134us (240us total) trans 0 rc 0/0 00010000:00010000:1.0:1780298642.390879:0:7411:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37800/0xaf35132e6303305c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 53 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.390880:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14980 00010000:00010000:1.0:1780298642.390888:0:7411:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37800/0xaf35132e6303305c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 53 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1780298642.390932:0:7411:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991010 00000004:00010000:1.0:1780298642.390950:0:7411:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000447a2e3a reply state 00000000e393ea61 transno 4294991011 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37800/0xaf35132e6303305c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 53 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.390959:0:7411:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37800/0xaf35132e6303305c lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 53 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.390964:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298642.390966:0:7411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1780298642.390968:0:7411:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1780298642.391092:0:7411:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b69c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+93:29360:x1866777609370240:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1618us (1705us total) trans 4294991011 rc 0/0 00000100:00100000:1.0:1780298642.391102:0:7411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61930 00000020:00080000:1.0:1780298642.391109:0:7411:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1780298642.391752:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.391805:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609370496 00000100:00100000:1.0:1780298642.391813:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609370496 00000100:00100000:1.0:1780298642.391859:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8855 00000100:00100000:1.0:1780298642.391869:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b69f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:29360:x1866777609370496:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.391885:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.391902:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63033063 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d22d7 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.391927:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000000a323d4b/0xaf35132e63032f2f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [765952->34320383] (req 765952->770047) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d21c6 expref: 9 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.391934:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(101 177) 00010000:00010000:1.0:1780298642.391937:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(102 177) 00010000:00010000:1.0:1780298642.391940:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63033063 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22d7 expref: 7 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.391969:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000000a323d4b/0xaf35132e63032f2f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [765952->34320383] (req 765952->770047) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d21c6 expref: 9 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.391982:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000000a323d4b/0xaf35132e63032f2f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [765952->34320383] (req 765952->770047) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d21c6 expref: 9 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.391995:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255092300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604873856:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298642.392074:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3629a4600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.392724:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255092300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604873856:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298642.392745:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63033063 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22d7 expref: 7 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.392756:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36a00, rc 0) 00000100:00100000:1.0:1780298642.392785:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b69f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609370496:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 921us (1036us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.392790:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8855 00000100:00100000:3.0:1780298642.393192:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.393231:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609370624 00000100:00100000:1.0:1780298642.393236:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609370624 00010000:00010000:1.0:1780298642.393257:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000000a323d4b/0xaf35132e63032f2f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [765952->34320383] (req 765952->770047) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d21c6 expref: 10 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.393287:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14981 00000100:00100000:1.0:1780298642.393309:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b68700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609370624:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298642.393319:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298642.393340:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000000a323d4b/0xaf35132e63032f2f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [765952->34320383] (req 765952->770047) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d21c6 expref: 10 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.393352:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000000a323d4b/0xaf35132e63032f2f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [765952->34320383] (req 765952->770047) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d21c6 expref: 10 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.393366:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000a323d4b/0xaf35132e63032f2f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [765952->34320383] (req 765952->770047) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d21c6 expref: 10 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.393379:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(102 177) 00010000:00010000:1.0:1780298642.393381:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(102 177) 00010000:00010000:1.0:1780298642.393385:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63033063 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22d7 expref: 6 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.393391:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63033063 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22d7 expref: 6 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.393408:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63033063 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22d7 expref: 6 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.393415:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255092300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777604873984:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298642.393449:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3629a4600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.394115:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255092300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777604873984:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298642.394160:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298642.394202:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b68700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609370624:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 895us (1012us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.394211:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14981 00000100:00100000:3.0:1780298642.394299:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.394355:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609370752 00000100:00100000:1.0:1780298642.394360:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609370752 00000100:00100000:1.0:1780298642.394391:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8856 00000100:00100000:1.0:1780298642.394399:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b6bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:29360:x1866777609370752:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.394408:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.394420:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a10929f0/0xaf35132e6303306a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d22de expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.394436:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000700ba1da/0xaf35132e63033016 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2334720->35889151] (req 2334720->2342911) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2291 expref: 14 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.394444:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000004b2c7ae/0xaf35132e63033039 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [638976->2334719] (req 638976->643071) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22b4 expref: 14 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.394451:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 2 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(74 130) 00010000:00010000:1.0:1780298642.394455:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(76 130) 00010000:00010000:1.0:1780298642.394459:0:20252:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a10929f0/0xaf35132e6303306a lrc: 5/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22de expref: 14 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.394477:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000004b2c7ae/0xaf35132e63033039 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [638976->2334719] (req 638976->643071) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22b4 expref: 14 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.394488:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000004b2c7ae/0xaf35132e63033039 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [638976->2334719] (req 638976->643071) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22b4 expref: 14 pid: 19992 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.394498:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255092300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604874112:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298642.394552:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000700ba1da/0xaf35132e63033016 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2334720->35889151] (req 2334720->2342911) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2291 expref: 14 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.394561:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000700ba1da/0xaf35132e63033016 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2334720->35889151] (req 2334720->2342911) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2291 expref: 14 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.394577:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255091f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604874240:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298642.394621:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3629a4600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.395310:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255092300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604874112:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298642.395323:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255091f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604874240:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298642.395339:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a10929f0/0xaf35132e6303306a lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22de expref: 14 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.395351:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37200, rc 0) 00000100:00100000:1.0:1780298642.395380:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b6bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:29360:x1866777609370752:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 983us (1092us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.395385:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8856 00000100:00100000:3.0:1780298642.395702:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.395727:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.395747:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609370880 00000100:00100000:1.0:1780298642.395752:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609370880 00010000:00010000:1.0:1780298642.395775:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000700ba1da/0xaf35132e63033016 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2334720->35889151] (req 2334720->2342911) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2291 expref: 14 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.395779:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609371008 00000100:00100000:2.0:1780298642.395791:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609371008 00000100:00100000:1.0:1780298642.395814:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14982 00000100:00100000:1.0:1780298642.395829:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b6aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2400:x1866777609370880:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298642.395830:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000004b2c7ae/0xaf35132e63033039 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [638976->2334719] (req 638976->643071) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22b4 expref: 15 pid: 19992 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.395842:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298642.395863:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000700ba1da/0xaf35132e63033016 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [2334720->35889151] (req 2334720->2342911) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2291 expref: 15 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.395876:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000700ba1da/0xaf35132e63033016 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [2334720->35889151] (req 2334720->2342911) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2291 expref: 15 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.395887:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14983 00010000:00010000:1.0:1780298642.395895:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000700ba1da/0xaf35132e63033016 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [2334720->35889151] (req 2334720->2342911) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2291 expref: 15 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.395905:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b69880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2399:x1866777609371008:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298642.395912:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298642.395920:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298642.395940:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000004b2c7ae/0xaf35132e63033039 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [638976->2334719] (req 638976->643071) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22b4 expref: 14 pid: 19992 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.395949:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b6aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609370880:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 123us (244us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.395971:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14982 00010000:00010000:2.0:1780298642.395994:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000004b2c7ae/0xaf35132e63033039 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [638976->2334719] (req 638976->643071) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22b4 expref: 14 pid: 19992 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.396018:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000004b2c7ae/0xaf35132e63033039 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [638976->2334719] (req 638976->643071) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22b4 expref: 13 pid: 19992 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.396034:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(76 130) 00010000:00010000:2.0:1780298642.396038:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(76 130) 00010000:00010000:2.0:1780298642.396045:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000a10929f0/0xaf35132e6303306a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22de expref: 12 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.396055:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a10929f0/0xaf35132e6303306a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22de expref: 12 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.396098:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000a10929f0/0xaf35132e6303306a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22de expref: 12 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.396112:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bd880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:7400:1866777604874368:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298642.396204:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.396925:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bd880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:7400:1866777604874368:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298642.396937:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298642.396963:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b69880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609371008:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1068us (1236us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.396972:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14983 00000100:00100000:3.0:1780298642.397189:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.397228:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.397246:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609371136 00000100:00100000:2.0:1780298642.397251:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609371136 00002000:00010000:2.0:1780298642.397275:0:8362:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00000100:00100000:1.0:1780298642.397276:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609371264 00000100:00100000:1.0:1780298642.397283:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609371264 00002000:00100000:2.0:1780298642.397284:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b6b100 with x1866777609371136 ext(4194304->18446744073709551615) 00002000:00010000:2.0:1780298642.397306:0:8362:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf242b6b100. 00002000:00010000:1.0:1780298642.397317:0:19949:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (2314690->18446744073709551615) 00002000:00100000:1.0:1780298642.397322:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b6a300 with x1866777609371264 ext(2314690->18446744073709551615) 00002000:00010000:1.0:1780298642.397331:0:19949:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf242b6a300. 00000100:00100000:2.0:1780298642.397337:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5969 00000100:00100000:2.0:1780298642.397354:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b6b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609371136:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:3.0:1780298642.397360:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5970 00000100:00100000:3.0:1780298642.397376:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b6a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609371264:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:2.0:1780298642.397627:0:8362:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4928525 -> 4194304 00002000:00010000:2.0:1780298642.397632:0:8362:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.397635:0:8362:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.397637:0:8362:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.397640:0:8362:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 456 -> 440 00000100:00100000:2.0:1780298642.397690:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b6b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609371136:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 343us (505us total) trans 4294969621 rc 0/0 00000100:00100000:2.0:1780298642.397697:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5969 00002000:00010000:2.0:1780298642.397700:0:8362:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:2.0:1780298642.397703:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b6b100 with x1866777609371136 ext(4194304->18446744073709551615) 00002000:00010000:2.0:1780298642.397709:0:8362:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf242b6b100. 00002000:00010000:3.0:1780298642.397712:0:19928:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4194304 -> 2314690 00002000:00010000:3.0:1780298642.397718:0:19928:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:3.0:1780298642.397721:0:19928:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:3.0:1780298642.397723:0:19928:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:3.0:1780298642.397726:0:19928:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 40 -> 0 00000100:00100000:3.0:1780298642.397762:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b6a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609371264:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 390us (534us total) trans 4294969183 rc 0/0 00000100:00100000:3.0:1780298642.397769:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5970 00002000:00010000:3.0:1780298642.397773:0:19928:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (2314690->18446744073709551615) 00002000:00100000:3.0:1780298642.397776:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b6a300 with x1866777609371264 ext(2314690->18446744073709551615) 00002000:00010000:3.0:1780298642.397782:0:19928:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf242b6a300. 00000100:00100000:3.0:1780298642.398831:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.398872:0:6476:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609371392 00000100:00100000:2.0:1780298642.398877:0:6476:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609371392 00000100:00100000:2.0:1780298642.398908:0:6476:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61931 00000100:00100000:2.0:1780298642.398915:0:6476:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+100:29360:x1866777609371392:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.398927:0:6476:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.398941:0:6476:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156600/0xaf35132e63033071 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 54 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d22e5 expref: -99 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.399008:0:6476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157200/0xaf35132e63033078 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 55 type: IBT flags: 0x40000000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.399020:0:6476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157200/0xaf35132e63033078 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 55 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.399027:0:6476:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157200/0xaf35132e63033078 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 55 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298642.399044:0:6476:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 6508994 00000004:00200000:2.0:1780298642.399047:0:6476:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298642.399064:0:6476:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157200/0xaf35132e63033078 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 55 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.399088:0:6476:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156600/0xaf35132e63033071 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 55 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22e5 expref: 102 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.399104:0:6476:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157200/0xaf35132e63033078 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 54 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22e5 expref: 101 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.399113:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.399115:0:6476:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298642.399118:0:6476:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157200, rc 0) 00000100:00100000:2.0:1780298642.399174:0:6476:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+101:29360:x1866777609371392:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 260us (345us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.399180:0:6476:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61931 00000100:00100000:0.0:1780298642.400150:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.400202:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609371520 00000100:00100000:2.0:1780298642.400207:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609371520 00000100:00100000:0.0:1780298642.400232:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.400253:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609371648 00000100:00100000:2.0:1780298642.400259:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609371648 00000100:00100000:3.0:1780298642.400272:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8857 00000100:00100000:1.0:1780298642.400302:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8858 00000100:00100000:1.0:1780298642.400320:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254919180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609371648:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.400334:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298642.400336:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609371520:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.400347:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.400348:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078c7f460/0xaf35132e6303307f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d22f3 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.400359:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e63033086 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d22ec expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.400403:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604874624:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.400403:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255091f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604874496:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.400463:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.400463:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3629a4600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.401335:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24310ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604874624:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.401342:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255091f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604874496:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.401369:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e63033086 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22ec expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.401375:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000078c7f460/0xaf35132e6303307f lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22f3 expref: 15 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.401405:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e63033086 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22ec expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.401450:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3a00, rc 0) 00010000:00010000:1.0:1780298642.401456:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078c7f460/0xaf35132e6303307f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22f3 expref: 15 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.401467:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36c00, rc 0) 00000100:00100000:0.0:1780298642.401494:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609371520:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1163us (1369us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.401499:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254919180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609371648:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1184us (1265us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.401526:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8857 00000100:00100000:1.0:1780298642.401535:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8858 00000100:00100000:2.0:1780298642.402643:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.402675:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.402707:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609371776 00000100:00100000:3.0:1780298642.402712:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609371904 00000100:00100000:1.0:1780298642.402718:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609371776 00000100:00100000:3.0:1780298642.402719:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609371904 00000100:00100000:1.0:1780298642.402768:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8859 00000100:00100000:1.0:1780298642.402777:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609371776:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.402788:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298642.402800:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8860 00010000:00010000:1.0:1780298642.402804:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003c8dab73/0xaf35132e6303308d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d22fa expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.402815:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7be300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609371904:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.402825:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.402836:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e63033094 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2301 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.402842:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255091f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604874752:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.402877:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604874880:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.402910:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3629a4600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.402911:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.403892:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255091f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604874752:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.403916:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003c8dab73/0xaf35132e6303308d lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22fa expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.403926:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24310fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604874880:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.403931:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003c8dab73/0xaf35132e6303308d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22fa expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.403952:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36800, rc 0) 00010000:00010000:2.0:1780298642.403964:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e63033094 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2301 expref: 15 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.403984:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e63033094 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2301 expref: 15 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.403996:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609371776:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1217us (1358us total) trans 0 rc 301/301 00010000:00010000:2.0:1780298642.404000:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3800, rc 0) 00000100:00100000:1.0:1780298642.404007:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8859 00000100:00100000:2.0:1780298642.404043:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7be300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609371904:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1232us (1366us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.404053:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8860 00000100:00100000:2.0:1780298642.405656:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.405696:0:9976:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609372160 00000100:00100000:0.0:1780298642.405702:0:9976:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609372160 00000100:00100000:0.0:1780298642.405743:0:9976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61932 00000100:00100000:0.0:1780298642.405750:0:9976:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+93:29360:x1866777609372160:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:0.0:1780298642.405947:0:9976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2200/0xaf35132e6303309b lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 55 type: IBT flags: 0x40000000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.405965:0:9976:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157200/0xaf35132e63033078 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 55 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22e5 expref: 100 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.405981:0:9976:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2200/0xaf35132e6303309b lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 55 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.406012:0:9976:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157200/0xaf35132e63033078 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 55 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22e5 expref: 100 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.406027:0:9976:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157200/0xaf35132e63033078 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 55 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22e5 expref: 100 pid: 6476 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.406041:0:9976:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25491bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:9976:1866777604875008:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298642.406094:0:9976:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417e40 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.406989:0:9976:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25491bb80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:9976:1866777604875008:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298642.407004:0:9976:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2200/0xaf35132e6303309b lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 55 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.407252:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.407286:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609372288 00000100:00100000:0.0:1780298642.407297:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609372288 00010000:00010000:0.0:1780298642.407313:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157200/0xaf35132e63033078 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 55 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22e5 expref: 101 pid: 6476 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.407340:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14984 00000100:00100000:0.0:1780298642.407345:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bdf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+101:2400:x1866777609372288:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298642.407354:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298642.407360:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157200/0xaf35132e63033078 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 56 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22e5 expref: 101 pid: 6476 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.407372:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157200/0xaf35132e63033078 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 56 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22e5 expref: 101 pid: 6476 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.407391:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157200/0xaf35132e63033078 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 56 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22e5 expref: 101 pid: 6476 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.407405:0:6461:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298642.407409:0:6461:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2200/0xaf35132e6303309b lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 55 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.407420:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2200/0xaf35132e6303309b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 55 type: IBT flags: 0x40290000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.407428:0:6461:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2200/0xaf35132e6303309b lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 55 type: IBT flags: 0x40290000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.407448:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298642.407472:0:9976:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2200/0xaf35132e6303309b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 54 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.407482:0:9976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2200/0xaf35132e6303309b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 54 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.407489:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bdf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+100:2400:x1866777609372288:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 140us (234us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.407499:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14984 00000001:00080000:2.0:1780298642.407546:0:9976:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991012 00000004:00010000:2.0:1780298642.407572:0:9976:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000b79a9244 reply state 000000007de068a9 transno 4294991013 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2200/0xaf35132e6303309b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 54 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.407584:0:9976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2200/0xaf35132e6303309b lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 54 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.407591:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.407594:0:9976:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1780298642.407597:0:9976:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1780298642.407658:0:9976:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+96:29360:x1866777609372160:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1907us (2003us total) trans 4294991013 rc 0/0 00000100:00100000:2.0:1780298642.407667:0:9976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61932 00000020:00080000:2.0:1780298642.407673:0:9976:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1780298642.408627:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.408664:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609372416 00000100:00100000:2.0:1780298642.408665:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.408669:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609372416 00002000:00010000:0.0:1780298642.408688:0:16972:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:0.0:1780298642.408692:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f7bce00 with x1866777609372416 ext(4194304->18446744073709551615) 00002000:00010000:0.0:1780298642.408700:0:16972:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f7bce00. 00000100:00100000:1.0:1780298642.408701:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609372544 00000100:00100000:1.0:1780298642.408706:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609372544 00000100:00100000:0.0:1780298642.408718:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5971 00002000:00010000:1.0:1780298642.408722:0:8363:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4090333->18446744073709551615) 00002000:00100000:1.0:1780298642.408727:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f7bd500 with x1866777609372544 ext(4090333->18446744073709551615) 00000100:00100000:0.0:1780298642.408727:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609372416:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:1.0:1780298642.408737:0:8363:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f7bd500. 00000100:00100000:1.0:1780298642.408758:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5972 00000100:00100000:1.0:1780298642.408767:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609372544:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:0.0:1780298642.408809:0:16972:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:0.0:1780298642.408811:0:16972:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298642.408813:0:16972:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298642.408815:0:16972:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298642.408816:0:16972:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 440 -> 440 00000100:00100000:0.0:1780298642.408849:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609372416:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 124us (224us total) trans 4294969622 rc 0/0 00000100:00100000:0.0:1780298642.408853:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5971 00002000:00010000:0.0:1780298642.408855:0:16972:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:0.0:1780298642.408858:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f7bce00 with x1866777609372416 ext(4194304->18446744073709551615) 00002000:00010000:0.0:1780298642.408861:0:16972:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f7bce00. 00002000:00010000:1.0:1780298642.408890:0:8363:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 2314690 -> 4090333 00002000:00010000:1.0:1780298642.408893:0:8363:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:1.0:1780298642.408895:0:8363:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:1.0:1780298642.408899:0:8363:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:1.0:1780298642.408901:0:8363:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:1.0:1780298642.408931:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609372544:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 163us (262us total) trans 4294969184 rc 0/0 00000100:00100000:1.0:1780298642.408939:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5972 00002000:00010000:1.0:1780298642.408943:0:8363:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4090333->18446744073709551615) 00002000:00100000:1.0:1780298642.408946:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f7bd500 with x1866777609372544 ext(4090333->18446744073709551615) 00002000:00010000:1.0:1780298642.408953:0:8363:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f7bd500. 00000100:00100000:3.0:1780298642.410414:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.410464:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609372672 00000100:00100000:0.0:1780298642.410470:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609372672 00002000:00100000:0.0:1780298642.410499:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (4079616->4083711) req@ffff9cf24310ed80 x1866777609372672/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.410510:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24310ed80 with x1866777609372672 ext(4079616->4083711) 00002000:00010000:0.0:1780298642.410515:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24310ed80 00000100:00100000:0.0:1780298642.410544:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5973 00000100:00100000:0.0:1780298642.410553:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2398:x1866777609372672:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298642.410695:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2398:x1866777609372672:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 143us (287us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298642.410703:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5973 00002000:00100000:0.0:1780298642.410710:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (4079616->4083711) req@ffff9cf24310ed80 x1866777609372672/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/4536 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.410718:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24310ed80 with x1866777609372672 ext(4079616->4083711) 00002000:00010000:0.0:1780298642.410722:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24310ed80 00000100:00100000:3.0:1780298642.412179:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.412230:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609372800 00000100:00100000:0.0:1780298642.412238:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609372800 00002000:00100000:0.0:1780298642.412265:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (4083712->4087807) req@ffff9cf24310df80 x1866777609372800/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.412275:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24310df80 with x1866777609372800 ext(4083712->4087807) 00002000:00010000:0.0:1780298642.412280:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24310df80 00000100:00100000:0.0:1780298642.412317:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5974 00000100:00100000:0.0:1780298642.412330:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609372800:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298642.412428:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609372800:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 104us (317us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298642.412436:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5974 00002000:00100000:0.0:1780298642.412443:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (4083712->4087807) req@ffff9cf24310df80 x1866777609372800/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/4536 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.412452:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24310df80 with x1866777609372800 ext(4083712->4087807) 00002000:00010000:0.0:1780298642.412456:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24310df80 00000100:00100000:2.0:1780298642.414119:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.414223:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609372928 00000100:00100000:3.0:1780298642.414234:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609372928 00002000:00100000:3.0:1780298642.414274:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x12:0x0] (4059136->4091903) req@ffff9cf37f7be680 x1866777609372928/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 504/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298642.414282:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f7be680 with x1866777609372928 ext(4059136->4091903) 00002000:00010000:3.0:1780298642.414287:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f7be680 00000100:00100000:3.0:1780298642.414322:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5975 00000100:00100000:3.0:1780298642.414330:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7be680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2398:x1866777609372928:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298642.415346:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7be680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2398:x1866777609372928:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 1015us (1228us total) trans 0 rc 24576/24576 00000100:00100000:0.0:1780298642.415355:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5975 00002000:00100000:0.0:1780298642.415363:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x12:0x0] (4059136->4091903) req@ffff9cf37f7be680 x1866777609372928/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 504/440 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 24576/24576 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.415373:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f7be680 with x1866777609372928 ext(4059136->4091903) 00002000:00010000:0.0:1780298642.415378:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f7be680 00000100:00100000:2.0:1780298642.416941:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.416993:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609373056 00000100:00100000:1.0:1780298642.417000:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609373056 00002000:00100000:1.0:1780298642.417047:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x12:0x0] (4079616->4090332) req@ffff9cf37f7bf100 x1866777609373056/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 11208/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298642.417056:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f7bf100 with x1866777609373056 ext(4079616->4090332) 00002000:00010000:1.0:1780298642.417059:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f7bf100 00000100:00100000:1.0:1780298642.417084:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5976 00000100:00100000:1.0:1780298642.417093:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609373056:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:1.0:1780298642.417570:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609373056:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 478us (633us total) trans 4294969185 rc 0/0 00000100:00100000:1.0:1780298642.417576:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5976 00002000:00100000:1.0:1780298642.417583:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x12:0x0] (4079616->4090332) req@ffff9cf37f7bf100 x1866777609373056/t4294969185(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 11208/448 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298642.417592:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f7bf100 with x1866777609373056 ext(4079616->4090332) 00002000:00010000:1.0:1780298642.417596:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f7bf100 00000100:00100000:2.0:1780298642.418545:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.418610:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609373184 00000100:00100000:0.0:1780298642.418617:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609373184 00000100:00100000:0.0:1780298642.418665:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8861 00000100:00100000:0.0:1780298642.418676:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609373184:12345-192.168.203.25@tcp:16:fsx.0 00000001:00080000:2.0:1780298642.419608:0:8351:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0000: transno 4294969185 is committed 00000001:00080000:2.0:1780298642.419629:0:8351:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0000: transno 4294969181 is committed 00000100:00100000:0.0:1780298642.419764:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609373184:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 1089us (1219us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.419773:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8861 00000100:00100000:3.0:1780298642.420906:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.420961:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609373312 00000100:00100000:0.0:1780298642.420970:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609373312 00000100:00100000:0.0:1780298642.421015:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61933 00000100:00100000:0.0:1780298642.421025:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+96:29360:x1866777609373312:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.421048:0:6477:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.421068:0:6477:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2600/0xaf35132e630330a2 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 55 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2308 expref: -99 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.421186:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3000/0xaf35132e630330a9 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 56 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.421203:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3000/0xaf35132e630330a9 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 56 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.421220:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3000/0xaf35132e630330a9 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 56 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1780298642.421251:0:6477:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 8284637 00000004:00200000:0.0:1780298642.421256:0:6477:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1780298642.421277:0:6477:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3000/0xaf35132e630330a9 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 56 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.421320:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2600/0xaf35132e630330a2 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 56 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2308 expref: 98 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.421341:0:6477:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3000/0xaf35132e630330a9 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 55 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2308 expref: 97 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.421351:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298642.421354:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1780298642.421357:0:6477:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3000, rc 0) 00000100:00100000:0.0:1780298642.421388:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+97:29360:x1866777609373312:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 366us (484us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.421395:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61933 00000100:00100000:2.0:1780298642.423578:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.423653:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609373440 00000100:00100000:1.0:1780298642.423664:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609373440 00002000:00100000:1.0:1780298642.423711:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x12:0x0] (2392064->2396159) req@ffff9cf255493800 x1866777609373440/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298642.423720:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255493800 with x1866777609373440 ext(2392064->2396159) 00002000:00010000:1.0:1780298642.423724:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf255493800 00000100:00100000:1.0:1780298642.423745:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5977 00000100:00100000:1.0:1780298642.423752:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255493800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609373440:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298642.423901:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255493800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609373440:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 149us (326us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298642.423909:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5977 00002000:00100000:1.0:1780298642.423914:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x12:0x0] (2392064->2396159) req@ffff9cf255493800 x1866777609373440/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/4536 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298642.423920:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255493800 with x1866777609373440 ext(2392064->2396159) 00002000:00010000:1.0:1780298642.423923:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf255493800 00000100:00100000:2.0:1780298642.425496:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.425545:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609373568 00000100:00100000:0.0:1780298642.425553:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609373568 00002000:00100000:0.0:1780298642.425592:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (2396160->2400255) req@ffff9cf255491500 x1866777609373568/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.425604:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255491500 with x1866777609373568 ext(2396160->2400255) 00002000:00010000:0.0:1780298642.425608:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf255491500 00000100:00100000:0.0:1780298642.425630:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5978 00000100:00100000:0.0:1780298642.425639:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255491500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609373568:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298642.425791:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255491500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609373568:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 153us (297us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298642.425798:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5978 00002000:00100000:0.0:1780298642.425806:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (2396160->2400255) req@ffff9cf255491500 x1866777609373568/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/4536 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.425815:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255491500 with x1866777609373568 ext(2396160->2400255) 00002000:00010000:0.0:1780298642.425819:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf255491500 00000100:00100000:3.0:1780298642.427265:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.427305:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609373696 00000100:00100000:0.0:1780298642.427311:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609373696 00002000:00100000:0.0:1780298642.427333:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (2392064->2400255) req@ffff9cf24310c000 x1866777609373696/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 8680/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.427343:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24310c000 with x1866777609373696 ext(2392064->2400255) 00002000:00010000:0.0:1780298642.427347:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24310c000 00000100:00100000:0.0:1780298642.427367:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5979 00000100:00100000:0.0:1780298642.427374:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609373696:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:0.0:1780298642.427819:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609373696:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 444us (553us total) trans 4294969186 rc 0/0 00000100:00100000:0.0:1780298642.427828:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5979 00002000:00100000:0.0:1780298642.427835:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (2392064->2400255) req@ffff9cf24310c000 x1866777609373696/t4294969186(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 8680/448 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.427844:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24310c000 with x1866777609373696 ext(2392064->2400255) 00002000:00010000:0.0:1780298642.427848:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24310c000 00000100:00100000:2.0:1780298642.428823:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.428880:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609373824 00000100:00100000:1.0:1780298642.428889:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609373824 00000100:00100000:1.0:1780298642.428929:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8862 00000100:00100000:1.0:1780298642.428936:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255492a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609373824:12345-192.168.203.25@tcp:16:fsx.0 00000001:00080000:2.0:1780298642.429763:0:8351:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0000: transno 4294969186 is committed 00000100:00100000:1.0:1780298642.429860:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255492a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609373824:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 923us (1037us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.429866:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8862 00000100:00100000:2.0:1780298642.431235:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.431279:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609373952 00000100:00100000:0.0:1780298642.431285:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609373952 00000100:00100000:0.0:1780298642.431333:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5980 00000100:00100000:0.0:1780298642.431339:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255490380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609373952:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298642.431494:0:19928:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4090333 -> 4090333 00002000:00010000:0.0:1780298642.431498:0:19928:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298642.431501:0:19928:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298642.431503:0:19928:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298642.431505:0:19928:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 0 -> 40 00000100:00100000:0.0:1780298642.431555:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255490380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609373952:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 215us (322us total) trans 4294969187 rc 0/0 00000100:00100000:0.0:1780298642.431562:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5980 00000100:00100000:3.0:1780298642.432985:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.433048:0:7411:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609374208 00000100:00100000:1.0:1780298642.433053:0:7411:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609374208 00000100:00100000:1.0:1780298642.433082:0:7411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61934 00000100:00100000:1.0:1780298642.433087:0:7411:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+100:29360:x1866777609374208:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:1.0:1780298642.433272:0:7411:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36800/0xaf35132e630330b0 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 56 type: IBT flags: 0x40000000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.433286:0:7411:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3000/0xaf35132e630330a9 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 56 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2308 expref: 96 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.433314:0:7411:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36800/0xaf35132e630330b0 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 56 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.433343:0:7411:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3000/0xaf35132e630330a9 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 56 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2308 expref: 96 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.433360:0:7411:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3000/0xaf35132e630330a9 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 56 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2308 expref: 96 pid: 6477 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.433370:0:7411:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255091f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7411:1866777604875136:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298642.433437:0:7411:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3629a4180 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.434160:0:7411:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255091f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7411:1866777604875136:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298642.434185:0:7411:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36800/0xaf35132e630330b0 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 56 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.434260:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.434326:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609374336 00000100:00100000:0.0:1780298642.434332:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609374336 00010000:00010000:0.0:1780298642.434357:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3000/0xaf35132e630330a9 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 56 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2308 expref: 97 pid: 6477 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.434388:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14985 00000100:00100000:0.0:1780298642.434398:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b02c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+97:2400:x1866777609374336:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298642.434411:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298642.434418:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3000/0xaf35132e630330a9 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 57 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2308 expref: 97 pid: 6477 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.434431:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3000/0xaf35132e630330a9 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 57 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2308 expref: 97 pid: 6477 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.434453:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3000/0xaf35132e630330a9 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 57 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2308 expref: 97 pid: 6477 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.434470:0:6462:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298642.434476:0:6462:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36800/0xaf35132e630330b0 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 56 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.434492:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36800/0xaf35132e630330b0 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 56 type: IBT flags: 0x40290000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.434500:0:6462:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36800/0xaf35132e630330b0 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 56 type: IBT flags: 0x40290000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.434524:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298642.434553:0:7411:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36800/0xaf35132e630330b0 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 55 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.434559:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b02c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+96:2400:x1866777609374336:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 161us (297us total) trans 0 rc 0/0 00010000:00010000:1.0:1780298642.434567:0:7411:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36800/0xaf35132e630330b0 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 55 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.434574:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14985 00000001:00080000:1.0:1780298642.434653:0:7411:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991014 00000004:00010000:1.0:1780298642.434687:0:7411:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 000000007f884e3f reply state 00000000a760081d transno 4294991015 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36800/0xaf35132e630330b0 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 55 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.434701:0:7411:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36800/0xaf35132e630330b0 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 55 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.434708:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298642.434711:0:7411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1780298642.434713:0:7411:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1780298642.434785:0:7411:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+103:29360:x1866777609374208:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1696us (1800us total) trans 4294991015 rc 0/0 00000100:00100000:1.0:1780298642.434792:0:7411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61934 00000020:00080000:1.0:1780298642.434796:0:7411:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1780298642.435557:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.435612:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609374464 00000100:00100000:2.0:1780298642.435617:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609374464 00000100:00100000:2.0:1780298642.435653:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8863 00000100:00100000:2.0:1780298642.435662:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609374464:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.435674:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.435687:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004ba675d2/0xaf35132e630330b7 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d230f expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.435729:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63033063 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22d7 expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.435738:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(102 177) 00010000:00010000:2.0:1780298642.435743:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(103 177) 00010000:00010000:2.0:1780298642.435748:0:20252:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004ba675d2/0xaf35132e630330b7 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d230f expref: 10 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.435774:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63033063 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22d7 expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.435786:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63033063 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22d7 expref: 8 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.435799:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255493100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604875264:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298642.435856:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.436648:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255493100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604875264:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298642.436662:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004ba675d2/0xaf35132e630330b7 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d230f expref: 10 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.436673:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156e00, rc 0) 00000100:00100000:1.0:1780298642.436695:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609374464:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1037us (1141us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.436703:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8863 00000100:00100000:3.0:1780298642.436902:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.436981:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609374592 00000100:00100000:1.0:1780298642.436986:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609374592 00010000:00010000:1.0:1780298642.437006:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63033063 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22d7 expref: 9 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.437035:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14986 00000100:00100000:1.0:1780298642.437044:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609374592:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298642.437053:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298642.437070:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63033063 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22d7 expref: 9 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.437081:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63033063 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22d7 expref: 9 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.437096:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63033063 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22d7 expref: 9 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.437111:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(103 177) 00010000:00010000:1.0:1780298642.437114:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(103 177) 00010000:00010000:1.0:1780298642.437119:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000004ba675d2/0xaf35132e630330b7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d230f expref: 9 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.437148:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004ba675d2/0xaf35132e630330b7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d230f expref: 9 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.437172:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000004ba675d2/0xaf35132e630330b7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d230f expref: 9 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.437182:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255091f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777604875392:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298642.437220:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3629a4600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.438230:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.438234:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255091f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777604875392:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298642.438247:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298642.438272:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609374592:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1229us (1371us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.438276:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609374720 00000100:00100000:1.0:1780298642.438281:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14986 00000100:00100000:2.0:1780298642.438283:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609374720 00000100:00100000:2.0:1780298642.438336:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8864 00000100:00100000:2.0:1780298642.438346:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609374720:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.438357:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.438368:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e630330be lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2316 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.438393:0:19992:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000a10929f0/0xaf35132e6303306a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22de expref: 7 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.438402:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(76 130) 00010000:00010000:2.0:1780298642.438405:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(77 130) 00010000:00010000:2.0:1780298642.438410:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e630330be lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2316 expref: 10 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.438429:0:19992:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000a10929f0/0xaf35132e6303306a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22de expref: 7 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.438440:0:19992:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000a10929f0/0xaf35132e6303306a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22de expref: 7 pid: 20252 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.438450:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255492300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604875520:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298642.438495:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.439095:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255492300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604875520:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298642.439111:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e630330be lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2316 expref: 10 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.439141:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157c00, rc 0) 00000100:00100000:2.0:1780298642.439172:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609374720:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 828us (944us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.439184:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8864 00000100:00100000:0.0:1780298642.439328:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.439367:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609374848 00000100:00100000:1.0:1780298642.439372:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609374848 00010000:00010000:1.0:1780298642.439386:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000a10929f0/0xaf35132e6303306a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22de expref: 8 pid: 20252 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.439413:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14987 00000100:00100000:1.0:1780298642.439421:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609374848:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298642.439431:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298642.439446:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000a10929f0/0xaf35132e6303306a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22de expref: 8 pid: 20252 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.439457:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000a10929f0/0xaf35132e6303306a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22de expref: 8 pid: 20252 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.439472:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a10929f0/0xaf35132e6303306a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d22de expref: 8 pid: 20252 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.439482:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(77 130) 00010000:00010000:1.0:1780298642.439485:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(77 130) 00010000:00010000:1.0:1780298642.439490:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e630330be lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2316 expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.439498:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e630330be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2316 expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.439519:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e630330be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2316 expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.439529:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255091f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:8374:1866777604875648:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298642.439570:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3629a4600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.440480:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255091f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:8374:1866777604875648:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298642.440494:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298642.440496:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.440520:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609374848:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1100us (1193us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.440526:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14987 00000100:00100000:2.0:1780298642.440551:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609374976 00000100:00100000:2.0:1780298642.440557:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609374976 00002000:00010000:2.0:1780298642.440588:0:19943:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (1280618->18446744073709551615) 00002000:00100000:2.0:1780298642.440594:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f745f80 with x1866777609374976 ext(1280618->18446744073709551615) 00000100:00100000:0.0:1780298642.440595:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00002000:00010000:2.0:1780298642.440604:0:19943:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f745f80. 00000100:00100000:2.0:1780298642.440635:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609375104 00000100:00100000:2.0:1780298642.440639:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609375104 00000100:00100000:1.0:1780298642.440645:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5981 00002000:00010000:2.0:1780298642.440652:0:19943:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00000100:00100000:1.0:1780298642.440655:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609374976:12345-192.168.203.25@tcp:10:fsx.0 00002000:00100000:2.0:1780298642.440659:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf254919c00 with x1866777609375104 ext(0->18446744073709551615) 00002000:00010000:2.0:1780298642.440665:0:19943:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf254919c00. 00000100:00100000:2.0:1780298642.440689:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5982 00000100:00100000:2.0:1780298642.440698:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254919c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609375104:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:2.0:1780298642.441015:0:19943:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4090333 -> 0 00002000:00010000:2.0:1780298642.441019:0:19943:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.441022:0:19943:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.441025:0:19943:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.441028:0:19943:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 40 -> 0 00000100:00100000:2.0:1780298642.441061:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254919c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609375104:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 364us (468us total) trans 4294969188 rc 0/0 00000100:00100000:2.0:1780298642.441069:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5982 00002000:00010000:2.0:1780298642.441074:0:19943:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00002000:00100000:2.0:1780298642.441077:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf254919c00 with x1866777609375104 ext(0->18446744073709551615) 00002000:00010000:2.0:1780298642.441082:0:19943:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf254919c00. 00002000:00010000:1.0:1780298642.441376:0:19949:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 1280618 00002000:00010000:1.0:1780298642.441380:0:19949:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:1.0:1780298642.441382:0:19949:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:1.0:1780298642.441384:0:19949:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:1.0:1780298642.441385:0:19949:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 440 -> 392 00000100:00100000:1.0:1780298642.441417:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609374976:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 763us (923us total) trans 4294969623 rc 0/0 00000100:00100000:1.0:1780298642.441424:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5981 00002000:00010000:1.0:1780298642.441428:0:19949:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (1280618->18446744073709551615) 00002000:00100000:1.0:1780298642.441431:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f745f80 with x1866777609374976 ext(1280618->18446744073709551615) 00002000:00010000:1.0:1780298642.441436:0:19949:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f745f80. 00000100:00100000:0.0:1780298642.442093:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.442156:0:6476:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609375232 00000100:00100000:2.0:1780298642.442162:0:6476:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609375232 00000100:00100000:2.0:1780298642.442197:0:6476:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61935 00000100:00100000:2.0:1780298642.442207:0:6476:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b6b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+103:29360:x1866777609375232:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.442223:0:6476:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.442238:0:6476:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156c00/0xaf35132e630330c5 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 56 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d231d expref: -99 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.442324:0:6476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156800/0xaf35132e630330cc lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 57 type: IBT flags: 0x40000000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.442343:0:6476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156800/0xaf35132e630330cc lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 57 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.442354:0:6476:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156800/0xaf35132e630330cc lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 57 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298642.442379:0:6476:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 1280618 00000004:00200000:2.0:1780298642.442384:0:6476:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298642.442401:0:6476:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156800/0xaf35132e630330cc lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 57 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.442424:0:6476:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156c00/0xaf35132e630330c5 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 57 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d231d expref: 105 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.442437:0:6476:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156800/0xaf35132e630330cc lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 56 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d231d expref: 104 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.442445:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.442448:0:6476:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298642.442451:0:6476:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156800, rc 0) 00000100:00100000:2.0:1780298642.442478:0:6476:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b6b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+104:29360:x1866777609375232:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 274us (385us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.442485:0:6476:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61935 00000100:00100000:3.0:1780298642.445079:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.445122:0:9976:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609375488 00000100:00100000:2.0:1780298642.445149:0:9976:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609375488 00000100:00100000:2.0:1780298642.445187:0:9976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61936 00000100:00100000:2.0:1780298642.445196:0:9976:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f744e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+96:29360:x1866777609375488:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:2.0:1780298642.445364:0:9976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157e00/0xaf35132e630330d3 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 57 type: IBT flags: 0x40000000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.445378:0:9976:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156800/0xaf35132e630330cc lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 57 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d231d expref: 103 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.445396:0:9976:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157e00/0xaf35132e630330d3 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 57 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.445423:0:9976:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156800/0xaf35132e630330cc lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 57 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d231d expref: 103 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.445436:0:9976:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156800/0xaf35132e630330cc lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 57 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d231d expref: 103 pid: 6476 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.445448:0:9976:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255492300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:9976:1866777604875776:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298642.445503:0:9976:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.446204:0:9976:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255492300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:9976:1866777604875776:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298642.446220:0:9976:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157e00/0xaf35132e630330d3 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 57 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.446338:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.446389:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609375616 00000100:00100000:2.0:1780298642.446393:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609375616 00010000:00010000:2.0:1780298642.446412:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156800/0xaf35132e630330cc lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 57 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d231d expref: 104 pid: 6476 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.446436:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14988 00000100:00100000:2.0:1780298642.446443:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f744380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+104:2400:x1866777609375616:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298642.446452:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298642.446458:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156800/0xaf35132e630330cc lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 58 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d231d expref: 104 pid: 6476 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.446468:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156800/0xaf35132e630330cc lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 58 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d231d expref: 104 pid: 6476 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.446485:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156800/0xaf35132e630330cc lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 58 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d231d expref: 104 pid: 6476 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.446497:0:6463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.446502:0:6463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157e00/0xaf35132e630330d3 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 57 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.446512:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157e00/0xaf35132e630330d3 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 57 type: IBT flags: 0x40290000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.446519:0:6463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157e00/0xaf35132e630330d3 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 57 type: IBT flags: 0x40290000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.446536:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1780298642.446556:0:9976:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157e00/0xaf35132e630330d3 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 56 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.446566:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f744380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+103:2400:x1866777609375616:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 122us (226us total) trans 0 rc 0/0 00010000:00010000:0.0:1780298642.446566:0:9976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157e00/0xaf35132e630330d3 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 56 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.446582:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14988 00000001:00080000:0.0:1780298642.446632:0:9976:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991016 00000004:00010000:0.0:1780298642.446657:0:9976:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 000000009945bf04 reply state 000000006250a5fc transno 4294991017 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157e00/0xaf35132e630330d3 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 56 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.446667:0:9976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157e00/0xaf35132e630330d3 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 56 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.446672:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298642.446673:0:9976:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1780298642.446675:0:9976:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1780298642.446733:0:9976:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f744e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+99:29360:x1866777609375488:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1538us (1656us total) trans 4294991017 rc 0/0 00000100:00100000:0.0:1780298642.446741:0:9976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61936 00000020:00080000:0.0:1780298642.446745:0:9976:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1780298642.447365:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.447402:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609375744 00000100:00100000:0.0:1780298642.447407:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609375744 00000100:00100000:0.0:1780298642.447442:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8865 00000100:00100000:0.0:1780298642.447450:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f744700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:29360:x1866777609375744:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.447461:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.447471:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bde78cfa/0xaf35132e630330da lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2324 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.447494:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000004ba675d2/0xaf35132e630330b7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d230f expref: 10 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.447503:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(103 177) 00010000:00010000:0.0:1780298642.447507:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(104 177) 00010000:00010000:0.0:1780298642.447512:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000bde78cfa/0xaf35132e630330da lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2324 expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.447534:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000004ba675d2/0xaf35132e630330b7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d230f expref: 10 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.447552:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000004ba675d2/0xaf35132e630330b7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d230f expref: 10 pid: 20252 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.447563:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b68700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604875904:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298642.447606:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.448171:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b68700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604875904:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298642.448192:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bde78cfa/0xaf35132e630330da lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2324 expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.448205:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3a00, rc 0) 00000100:00100000:0.0:1780298642.448231:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f744700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:29360:x1866777609375744:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 784us (866us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.448236:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8865 00000100:00100000:3.0:1780298642.448449:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.448552:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609375872 00000100:00100000:2.0:1780298642.448557:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609375872 00010000:00010000:2.0:1780298642.448589:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000004ba675d2/0xaf35132e630330b7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d230f expref: 11 pid: 20252 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.448658:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14989 00000100:00100000:2.0:1780298642.448676:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f747100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609375872:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298642.448684:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298642.448707:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000004ba675d2/0xaf35132e630330b7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d230f expref: 11 pid: 20252 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.448720:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000004ba675d2/0xaf35132e630330b7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d230f expref: 11 pid: 20252 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.448747:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004ba675d2/0xaf35132e630330b7 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d230f expref: 11 pid: 20252 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.448756:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(104 177) 00010000:00010000:2.0:1780298642.448759:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(104 177) 00010000:00010000:2.0:1780298642.448772:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000bde78cfa/0xaf35132e630330da lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2324 expref: 8 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.448779:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000bde78cfa/0xaf35132e630330da lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2324 expref: 8 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.448795:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000bde78cfa/0xaf35132e630330da lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2324 expref: 8 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.448811:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255492300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777604876032:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298642.448849:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.449536:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255492300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777604876032:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298642.449549:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298642.449573:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f747100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609375872:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 908us (1124us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.449580:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14989 00000100:00100000:3.0:1780298642.449738:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.449774:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609376000 00000100:00100000:1.0:1780298642.449780:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609376000 00000100:00100000:1.0:1780298642.449812:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8866 00000100:00100000:1.0:1780298642.449819:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609376000:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.449828:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.449835:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e630330e1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d232b expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.449853:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e630330be lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2316 expref: 10 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.449858:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(77 130) 00010000:00010000:1.0:1780298642.449861:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(78 130) 00010000:00010000:1.0:1780298642.449864:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e630330e1 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d232b expref: 8 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.449881:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e630330be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2316 expref: 10 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.449893:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e630330be lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2316 expref: 10 pid: 19992 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.449904:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255091f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604876160:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298642.449939:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3629a4600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.450523:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255091f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604876160:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298642.450553:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e630330e1 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d232b expref: 8 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.450569:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36a00, rc 0) 00000100:00100000:1.0:1780298642.450602:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:29360:x1866777609376000:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 783us (864us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.450614:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8866 00000100:00100000:0.0:1780298642.450690:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.450733:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609376128 00000100:00100000:2.0:1780298642.450738:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609376128 00010000:00010000:2.0:1780298642.450754:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e630330be lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2316 expref: 11 pid: 19992 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.450779:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14990 00000100:00100000:2.0:1780298642.450786:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b68700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609376128:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298642.450794:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298642.450807:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e630330be lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2316 expref: 11 pid: 19992 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.450817:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e630330be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2316 expref: 11 pid: 19992 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.450838:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e630330be lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2316 expref: 11 pid: 19992 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.450848:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(78 130) 00010000:00010000:2.0:1780298642.450851:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(78 130) 00010000:00010000:2.0:1780298642.450856:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e630330e1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d232b expref: 7 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.450865:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e630330e1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d232b expref: 7 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.450884:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e630330e1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d232b expref: 7 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.450894:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255492300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6461:1866777604876288:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298642.450927:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.451549:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255492300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6461:1866777604876288:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298642.451560:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298642.451586:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b68700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609376128:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 801us (896us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.451594:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14990 00000100:00100000:1.0:1780298642.451615:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.451677:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609376384 00000100:00100000:0.0:1780298642.451687:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609376384 00002000:00010000:0.0:1780298642.451710:0:19950:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4017167->18446744073709551615) 00002000:00100000:0.0:1780298642.451715:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255091f80 with x1866777609376384 ext(4017167->18446744073709551615) 00002000:00010000:0.0:1780298642.451721:0:19950:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf255091f80. 00000100:00100000:0.0:1780298642.451748:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5983 00000100:00100000:3.0:1780298642.451751:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.451760:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255091f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609376384:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:2.0:1780298642.451765:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609376256 00000100:00100000:2.0:1780298642.451770:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609376256 00002000:00010000:2.0:1780298642.451784:0:8364:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:2.0:1780298642.451788:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f746680 with x1866777609376256 ext(4194304->18446744073709551615) 00002000:00010000:2.0:1780298642.451794:0:8364:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f746680. 00000100:00100000:2.0:1780298642.451815:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5984 00000100:00100000:2.0:1780298642.451823:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f746680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609376256:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:2.0:1780298642.451941:0:8364:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 1280618 -> 4194304 00002000:00010000:0.0:1780298642.451942:0:19950:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 0 -> 4017167 00002000:00010000:2.0:1780298642.451947:0:8364:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298642.451948:0:19950:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.451952:0:8364:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298642.451954:0:19950:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.451957:0:8364:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298642.451958:0:19950:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.451961:0:8364:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 392 -> 392 00002000:00010000:0.0:1780298642.451963:0:19950:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:2.0:1780298642.451993:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f746680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609376256:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 171us (247us total) trans 4294969624 rc 0/0 00000100:00100000:0.0:1780298642.451996:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255091f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609376384:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 237us (380us total) trans 4294969189 rc 0/0 00000100:00100000:2.0:1780298642.452008:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5984 00002000:00010000:2.0:1780298642.452011:0:8364:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:2.0:1780298642.452016:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f746680 with x1866777609376256 ext(4194304->18446744073709551615) 00000100:00100000:0.0:1780298642.452017:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5983 00002000:00010000:0.0:1780298642.452023:0:19950:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4017167->18446744073709551615) 00002000:00010000:2.0:1780298642.452024:0:8364:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f746680. 00002000:00100000:0.0:1780298642.452028:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255091f80 with x1866777609376384 ext(4017167->18446744073709551615) 00002000:00010000:0.0:1780298642.452035:0:19950:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf255091f80. 00000100:00100000:1.0:1780298642.453021:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.453077:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609376512 00000100:00100000:0.0:1780298642.453082:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609376512 00002000:00100000:0.0:1780298642.453109:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x12:0x0] (4009984->4014079) req@ffff9cf255092300 x1866777609376512/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.453120:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255092300 with x1866777609376512 ext(4009984->4014079) 00002000:00010000:0.0:1780298642.453147:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf255092300 00000100:00100000:0.0:1780298642.453170:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5985 00000100:00100000:0.0:1780298642.453178:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255092300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609376512:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298642.453346:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255092300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609376512:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 169us (325us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298642.453354:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5985 00002000:00100000:0.0:1780298642.453361:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x12:0x0] (4009984->4014079) req@ffff9cf255092300 x1866777609376512/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/4536 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.453371:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255092300 with x1866777609376512 ext(4009984->4014079) 00002000:00010000:0.0:1780298642.453375:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf255092300 00000100:00100000:1.0:1780298642.454694:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.454749:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609376640 00000100:00100000:0.0:1780298642.454755:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609376640 00002000:00100000:0.0:1780298642.454775:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (4014080->4018175) req@ffff9cf255090000 x1866777609376640/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.454785:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255090000 with x1866777609376640 ext(4014080->4018175) 00002000:00010000:0.0:1780298642.454788:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf255090000 00000100:00100000:0.0:1780298642.454804:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5986 00000100:00100000:0.0:1780298642.454810:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255090000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609376640:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298642.454878:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255090000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609376640:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 68us (186us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298642.454882:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5986 00002000:00100000:0.0:1780298642.454892:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (4014080->4018175) req@ffff9cf255090000 x1866777609376640/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/4536 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.454900:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255090000 with x1866777609376640 ext(4014080->4018175) 00002000:00010000:0.0:1780298642.454903:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf255090000 00000100:00100000:3.0:1780298642.455925:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.455963:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609376768 00000100:00100000:1.0:1780298642.455968:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609376768 00002000:00100000:1.0:1780298642.455991:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x12:0x0] (4009984->4017166) req@ffff9cf37f744000 x1866777609376768/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 7672/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298642.456001:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f744000 with x1866777609376768 ext(4009984->4017166) 00002000:00010000:1.0:1780298642.456005:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f744000 00000100:00100000:1.0:1780298642.456026:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5987 00000100:00100000:1.0:1780298642.456032:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f744000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609376768:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:1.0:1780298642.456486:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f744000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609376768:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 454us (561us total) trans 4294969190 rc 0/0 00000100:00100000:1.0:1780298642.456494:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5987 00002000:00100000:1.0:1780298642.456501:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x12:0x0] (4009984->4017166) req@ffff9cf37f744000 x1866777609376768/t4294969190(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 7672/448 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298642.456511:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f744000 with x1866777609376768 ext(4009984->4017166) 00002000:00010000:1.0:1780298642.456515:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f744000 00000100:00100000:1.0:1780298642.457179:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.457222:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609376896 00000100:00100000:2.0:1780298642.457227:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609376896 00000100:00100000:2.0:1780298642.457256:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8867 00000100:00100000:2.0:1780298642.457264:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e580850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609376896:12345-192.168.203.25@tcp:16:fsx.0 00000001:00080000:0.0:1780298642.457954:0:8351:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0000: transno 4294969190 is committed 00000001:00080000:0.0:1780298642.457989:0:8351:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0000: transno 4294969188 is committed 00000100:00100000:2.0:1780298642.458069:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e580850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609376896:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 805us (889us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.458085:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8867 00000100:00100000:0.0:1780298642.458741:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.458774:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609377024 00000100:00100000:2.0:1780298642.458778:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609377024 00000100:00100000:2.0:1780298642.458809:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61937 00000100:00100000:2.0:1780298642.458818:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b6aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+103:29360:x1866777609377024:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.458831:0:6477:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.458848:0:6477:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157c00/0xaf35132e630330e8 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 57 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2332 expref: -99 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.458922:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156e00/0xaf35132e630330ef lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 58 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.458936:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156e00/0xaf35132e630330ef lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 58 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.458948:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156e00/0xaf35132e630330ef lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 58 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298642.458970:0:6477:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 8211471 00000004:00200000:2.0:1780298642.458974:0:6477:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298642.458992:0:6477:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156e00/0xaf35132e630330ef lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 58 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.459017:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157c00/0xaf35132e630330e8 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 58 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2332 expref: 105 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.459033:0:6477:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156e00/0xaf35132e630330ef lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 57 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2332 expref: 104 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.459041:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.459044:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298642.459047:0:6477:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156e00, rc 0) 00000100:00100000:2.0:1780298642.459076:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b6aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+104:29360:x1866777609377024:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 260us (335us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.459086:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61937 00000100:00100000:3.0:1780298642.459832:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.459863:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.459880:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609377152 00000100:00100000:2.0:1780298642.459885:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609377152 00000100:00100000:2.0:1780298642.459912:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609377280 00000100:00100000:2.0:1780298642.459915:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609377280 00000100:00100000:0.0:1780298642.459931:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8868 00000100:00100000:0.0:1780298642.459942:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609377152:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.459953:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298642.459954:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8869 00000100:00100000:2.0:1780298642.459965:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609377280:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.459966:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e630330f6 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2339 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.459975:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.459988:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042c16ab0/0xaf35132e630330fd lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2340 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.460021:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b69880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604876416:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.460024:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255492a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604876544:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.460098:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4a80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.460099:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417f00 going to sleep for 11 seconds 00002000:00010000:2.0:1780298642.460837:0:19992:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks: 0 -> 16 00000100:00100000:2.0:1780298642.460849:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255492a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604876544:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.460871:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000042c16ab0/0xaf35132e630330fd lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2340 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.460888:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042c16ab0/0xaf35132e630330fd lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2340 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.460898:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156800, rc 0) 00000100:00100000:0.0:1780298642.460909:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b69880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604876416:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.460928:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609377280:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 964us (1065us total) trans 0 rc 301/301 00010000:00010000:0.0:1780298642.460933:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e630330f6 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2339 expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.460939:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8869 00010000:00010000:0.0:1780298642.460956:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e630330f6 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2339 expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.460967:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3800, rc 0) 00000100:00100000:0.0:1780298642.460999:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609377152:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1060us (1168us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.461007:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8868 00000100:00100000:1.0:1780298642.461878:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.461913:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609377408 00000100:00100000:2.0:1780298642.461917:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609377408 00000100:00100000:1.0:1780298642.461933:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.461950:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609377536 00000100:00100000:2.0:1780298642.461953:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609377536 00000100:00100000:2.0:1780298642.461978:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8870 00000100:00100000:0.0:1780298642.461988:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8871 00000100:00100000:2.0:1780298642.461989:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255093b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609377408:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.461998:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298642.462002:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255092d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609377536:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.462009:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009e0b8e7b/0xaf35132e63033104 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2347 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.462012:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.462024:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a269a887/0xaf35132e6303310b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d234e expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.462039:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255492a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604876672:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.462063:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b69880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604876800:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.462083:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4a80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.462098:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417f00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.462674:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255492a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604876672:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.462704:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009e0b8e7b/0xaf35132e63033104 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2347 expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.462718:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009e0b8e7b/0xaf35132e63033104 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2347 expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.462728:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156c00, rc 0) 00000100:00100000:2.0:1780298642.462766:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255093b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609377408:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 780us (890us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.462774:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8870 00000100:00100000:0.0:1780298642.462788:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b69880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604876800:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.462809:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a269a887/0xaf35132e6303310b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d234e expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.462830:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a269a887/0xaf35132e6303310b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d234e expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.462841:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3e00, rc 0) 00000100:00100000:0.0:1780298642.462874:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255092d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609377536:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 878us (942us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.462881:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8871 00000100:00100000:1.0:1780298642.463809:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.463828:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.463844:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609377664 00000100:00100000:3.0:1780298642.463848:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609377664 00000100:00100000:2.0:1780298642.463864:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609377792 00000100:00100000:3.0:1780298642.463876:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8872 00000100:00100000:2.0:1780298642.463880:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609377792 00000100:00100000:3.0:1780298642.463883:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255090a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609377664:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.463890:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.463900:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e63033112 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2355 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.463916:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8873 00000100:00100000:2.0:1780298642.463924:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255091500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609377792:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298642.463933:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f746300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604876928:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.463933:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.463942:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a58d48e9/0xaf35132e63033119 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d235c expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.463972:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255492a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604877056:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.463981:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.464004:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4a80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.466649:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f746300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604876928:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.466700:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255492a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604877056:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.466717:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a58d48e9/0xaf35132e63033119 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d235c expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.466730:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e63033112 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2355 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.466730:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a58d48e9/0xaf35132e63033119 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d235c expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.466754:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156a00, rc 0) 00000100:00100000:2.0:1780298642.466785:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255091500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609377792:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 2862us (2956us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.466793:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8873 00010000:00010000:3.0:1780298642.466801:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e63033112 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2355 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.466820:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2a00, rc 0) 00000100:00100000:3.0:1780298642.466884:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255090a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609377664:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 3001us (3074us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.466959:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8872 00000100:00100000:1.0:1780298642.467759:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.467786:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609377920 00000100:00100000:3.0:1780298642.467793:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609377920 00000100:00100000:3.0:1780298642.467823:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8874 00000100:00100000:3.0:1780298642.467829:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255090700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609377920:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.467840:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.467851:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c47b04a/0xaf35132e63033120 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2363 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.467870:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e630330e1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d232b expref: 6 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.467876:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(78 130) 00010000:00010000:3.0:1780298642.467879:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(79 130) 00010000:00010000:3.0:1780298642.467882:0:20252:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c47b04a/0xaf35132e63033120 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [790528->794623] (req 790528->794623) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2363 expref: 10 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.467901:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e630330e1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d232b expref: 6 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.467918:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e630330e1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d232b expref: 6 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.467927:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f746300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604877184:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298642.467980:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.468628:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f746300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604877184:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298642.468645:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005c47b04a/0xaf35132e63033120 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [790528->794623] (req 790528->794623) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2363 expref: 10 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.468656:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3400, rc 0) 00000100:00100000:3.0:1780298642.468681:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255090700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609377920:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 853us (925us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298642.468725:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8874 00000100:00100000:0.0:1780298642.468809:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.468877:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609378048 00000100:00100000:3.0:1780298642.468883:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609378048 00010000:00010000:3.0:1780298642.468902:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e630330e1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d232b expref: 7 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.468932:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14991 00000100:00100000:3.0:1780298642.468941:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b69880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609378048:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298642.468951:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298642.468964:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e630330e1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d232b expref: 7 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.468988:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e630330e1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d232b expref: 7 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.469004:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e630330e1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d232b expref: 7 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.469013:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(79 130) 00010000:00010000:3.0:1780298642.469017:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(79 130) 00010000:00010000:3.0:1780298642.469025:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000005c47b04a/0xaf35132e63033120 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 790528->794623) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2363 expref: 9 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.469034:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005c47b04a/0xaf35132e63033120 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 790528->794623) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2363 expref: 9 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.469058:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000005c47b04a/0xaf35132e63033120 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 790528->794623) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2363 expref: 9 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.469069:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f746300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777604877312:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298642.469113:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.469737:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f746300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777604877312:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298642.469746:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298642.469777:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b69880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609378048:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 838us (971us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298642.469795:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14991 00000100:00100000:0.0:1780298642.470344:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.470381:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609378176 00000100:00100000:3.0:1780298642.470386:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609378176 00002000:00100000:3.0:1780298642.470414:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x12:0x0] (790528->794623) req@ffff9cf242b68380 x1866777609378176/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298642.470425:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b68380 with x1866777609378176 ext(790528->794623) 00002000:00010000:3.0:1780298642.470429:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf242b68380 00000100:00100000:3.0:1780298642.470449:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5988 00000100:00100000:3.0:1780298642.470455:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b68380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609378176:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298642.470598:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b68380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609378176:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 143us (254us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1780298642.470603:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5988 00002000:00100000:3.0:1780298642.470607:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x12:0x0] (790528->794623) req@ffff9cf242b68380 x1866777609378176/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/4536 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298642.470612:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b68380 with x1866777609378176 ext(790528->794623) 00002000:00010000:3.0:1780298642.470615:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf242b68380 00000100:00100000:0.0:1780298642.471708:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.471759:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609378304 00000100:00100000:2.0:1780298642.471765:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609378304 00002000:00100000:2.0:1780298642.471790:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (794624->798719) req@ffff9cf242b69f80 x1866777609378304/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.471798:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b69f80 with x1866777609378304 ext(794624->798719) 00002000:00010000:2.0:1780298642.471801:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf242b69f80 00000100:00100000:2.0:1780298642.471819:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5989 00000100:00100000:2.0:1780298642.471825:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b69f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609378304:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298642.471940:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b69f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609378304:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 115us (233us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298642.471946:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5989 00002000:00100000:2.0:1780298642.471951:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (794624->798719) req@ffff9cf242b69f80 x1866777609378304/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/4536 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.471957:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b69f80 with x1866777609378304 ext(794624->798719) 00002000:00010000:2.0:1780298642.471960:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf242b69f80 00000100:00100000:1.0:1780298642.473077:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.473116:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609378432 00000100:00100000:0.0:1780298642.473121:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609378432 00002000:00100000:0.0:1780298642.473173:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x12:0x0] (770048->835583) req@ffff9cf255093800 x1866777609378432/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 504/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.473179:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255093800 with x1866777609378432 ext(770048->835583) 00002000:00010000:0.0:1780298642.473183:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf255093800 00000100:00100000:0.0:1780298642.473200:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5990 00000100:00100000:0.0:1780298642.473209:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255093800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609378432:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298642.474163:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255093800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609378432:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 954us (1084us total) trans 0 rc 57344/57344 00000100:00100000:3.0:1780298642.474173:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5990 00002000:00100000:3.0:1780298642.474182:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x12:0x0] (770048->835583) req@ffff9cf255093800 x1866777609378432/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 504/440 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 57344/57344 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298642.474192:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255093800 with x1866777609378432 ext(770048->835583) 00002000:00010000:3.0:1780298642.474196:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf255093800 00000100:00100000:1.0:1780298642.475006:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.475047:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609378560 00000100:00100000:0.0:1780298642.475051:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609378560 00000100:00100000:0.0:1780298642.475078:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8875 00000100:00100000:0.0:1780298642.475083:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255093480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609378560:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.475092:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.475104:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a269a887/0xaf35132e63033127 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d236a expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.475164:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b6bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604877440:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.475208:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417f00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.475858:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b6bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604877440:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.475875:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a269a887/0xaf35132e63033127 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d236a expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.475888:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a269a887/0xaf35132e63033127 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d236a expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.475895:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3e00, rc 0) 00000100:00100000:0.0:1780298642.475921:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255093480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609378560:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 839us (917us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.475926:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8875 00000100:00100000:3.0:1780298642.476738:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.476771:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609378688 00000100:00100000:1.0:1780298642.476775:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609378688 00000100:00100000:1.0:1780298642.476804:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8876 00000100:00100000:1.0:1780298642.476812:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609378688:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.476822:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.476832:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e6303312e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2371 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.476859:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255093100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604877568:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.476892:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3629a4600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.477548:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255093100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604877568:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.477573:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e6303312e lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2371 expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.477582:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e6303312e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2371 expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.477589:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37200, rc 0) 00000100:00100000:3.0:1780298642.477612:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609378688:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 802us (875us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.477617:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8876 00000100:00100000:0.0:1780298642.478402:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.478427:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609378816 00000100:00100000:3.0:1780298642.478432:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609378816 00000100:00100000:3.0:1780298642.478468:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8877 00000100:00100000:3.0:1780298642.478475:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b6bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:29360:x1866777609378816:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.478482:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.478491:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ced6368f/0xaf35132e63033135 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2378 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.478506:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000005c47b04a/0xaf35132e63033120 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 790528->794623) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2363 expref: 9 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.478512:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(79 130) 00010000:00010000:3.0:1780298642.478515:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(79 130) 00010000:00010000:3.0:1780298642.478518:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ced6368f/0xaf35132e63033135 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [372736->376831] (req 372736->376831) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2378 expref: 7 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.478541:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000005c47b04a/0xaf35132e63033120 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 790528->794623) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2363 expref: 9 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.478551:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000005c47b04a/0xaf35132e63033120 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 790528->794623) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2363 expref: 9 pid: 20252 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.478559:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f745500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604877696:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298642.478589:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.479163:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f745500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604877696:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298642.479175:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ced6368f/0xaf35132e63033135 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [372736->376831] (req 372736->376831) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2378 expref: 7 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.479183:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2a00, rc 0) 00000100:00100000:3.0:1780298642.479203:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b6bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609378816:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 730us (802us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298642.479207:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8877 00000100:00100000:1.0:1780298642.479443:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.479482:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609378944 00000100:00100000:0.0:1780298642.479487:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609378944 00010000:00010000:0.0:1780298642.479507:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000005c47b04a/0xaf35132e63033120 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 790528->794623) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2363 expref: 10 pid: 20252 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.479528:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14992 00000100:00100000:0.0:1780298642.479533:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b00c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609378944:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298642.479540:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298642.479557:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000005c47b04a/0xaf35132e63033120 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 790528->794623) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2363 expref: 10 pid: 20252 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.479569:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000005c47b04a/0xaf35132e63033120 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 790528->794623) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2363 expref: 10 pid: 20252 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.479586:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c47b04a/0xaf35132e63033120 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 790528->794623) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2363 expref: 10 pid: 20252 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.479597:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(79 130) 00010000:00010000:0.0:1780298642.479601:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(79 130) 00010000:00010000:0.0:1780298642.479608:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000ced6368f/0xaf35132e63033135 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [372736->33927167] (req 372736->376831) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2378 expref: 6 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.479617:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ced6368f/0xaf35132e63033135 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [372736->33927167] (req 372736->376831) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2378 expref: 6 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.479641:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000ced6368f/0xaf35132e63033135 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [372736->33927167] (req 372736->376831) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2378 expref: 6 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.479652:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bc700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:20253:1866777604877824:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298642.479692:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417f00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.480433:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bc700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:20253:1866777604877824:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298642.480446:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298642.480483:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b00c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609378944:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 948us (1039us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.480491:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14992 00000100:00100000:3.0:1780298642.480504:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.480572:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609379072 00000100:00100000:2.0:1780298642.480580:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609379072 00000100:00100000:2.0:1780298642.480631:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5991 00000100:00100000:2.0:1780298642.480642:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609379072:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298642.480890:0:8362:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4017167 -> 4017167 00002000:00010000:2.0:1780298642.480895:0:8362:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.480897:0:8362:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.480899:0:8362:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.480902:0:8362:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 16 -> 24 00000100:00100000:2.0:1780298642.480944:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609379072:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 304us (439us total) trans 4294969191 rc 0/0 00000100:00100000:2.0:1780298642.480952:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5991 00000100:00100000:3.0:1780298642.481660:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.481714:0:7411:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609379200 00000100:00100000:1.0:1780298642.481720:0:7411:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609379200 00000100:00100000:1.0:1780298642.481758:0:7411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61938 00000100:00100000:1.0:1780298642.481768:0:7411:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f744380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+99:29360:x1866777609379200:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.481784:0:7411:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.481797:0:7411:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36e00/0xaf35132e6303313c lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 58 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d237f expref: -99 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.481879:0:7411:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37400/0xaf35132e63033143 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 59 type: IBT flags: 0x40000000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.481894:0:7411:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37400/0xaf35132e63033143 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 59 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.481905:0:7411:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37400/0xaf35132e63033143 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 59 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1780298642.481930:0:7411:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 8211471 00000004:00200000:1.0:1780298642.481934:0:7411:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1780298642.481951:0:7411:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37400/0xaf35132e63033143 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 59 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.481973:0:7411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36e00/0xaf35132e6303313c lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 59 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d237f expref: 101 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.481988:0:7411:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37400/0xaf35132e63033143 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 58 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d237f expref: 100 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.481996:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298642.481999:0:7411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1780298642.482002:0:7411:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37400, rc 0) 00000100:00100000:1.0:1780298642.482039:0:7411:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f744380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+100:29360:x1866777609379200:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 269us (376us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.482046:0:7411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61938 00000100:00100000:0.0:1780298642.482838:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.482870:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609379328 00000100:00100000:3.0:1780298642.482874:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609379328 00000100:00100000:3.0:1780298642.482902:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8878 00000100:00100000:3.0:1780298642.482909:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609379328:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.482918:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.482933:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e6303314a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2386 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.482961:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f747100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604877952:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.483004:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00002000:00010000:3.0:1780298642.483610:0:20252:0:(ofd_lvb.c:277:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime: 1780298642 -> 1780298643 00000100:00100000:3.0:1780298642.483621:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f747100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604877952:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.483635:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e6303314a lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2386 expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.483648:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e6303314a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2386 expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.483654:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3c00, rc 0) 00000100:00100000:3.0:1780298642.483676:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609379328:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 768us (839us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.483680:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8878 00000100:00100000:0.0:1780298642.484484:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.484509:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609379456 00000100:00100000:3.0:1780298642.484513:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609379456 00000100:00100000:3.0:1780298642.484534:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8879 00000100:00100000:3.0:1780298642.484539:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609379456:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.484544:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.484550:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e63033151 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d238d expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.484568:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f747100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604878080:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.484603:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.485213:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f747100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604878080:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.485225:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e63033151 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d238d expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.485234:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e63033151 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d238d expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.485241:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3600, rc 0) 00000100:00100000:3.0:1780298642.485263:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609379456:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 724us (779us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.485268:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8879 00000100:00100000:0.0:1780298642.486191:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.486223:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609379584 00000100:00100000:3.0:1780298642.486228:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609379584 00000100:00100000:3.0:1780298642.486256:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8880 00000100:00100000:3.0:1780298642.486263:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609379584:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.486270:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.486279:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e63033158 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2394 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.486303:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000bde78cfa/0xaf35132e630330da lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2324 expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.486310:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(104 177) 00010000:00010000:3.0:1780298642.486313:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(105 177) 00010000:00010000:3.0:1780298642.486321:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e63033158 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2781184->2785279] (req 2781184->2785279) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2394 expref: 11 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.486336:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000bde78cfa/0xaf35132e630330da lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2324 expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.486345:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000bde78cfa/0xaf35132e630330da lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2324 expref: 9 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.486352:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f747100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604878208:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298642.486378:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.486972:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f747100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604878208:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298642.486987:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e63033158 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2781184->2785279] (req 2781184->2785279) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2394 expref: 11 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.487000:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3e00, rc 0) 00000100:00100000:3.0:1780298642.487027:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:29360:x1866777609379584:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 765us (837us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298642.487031:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8880 00000100:00100000:0.0:1780298642.487112:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.487169:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609379712 00000100:00100000:1.0:1780298642.487175:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609379712 00010000:00010000:1.0:1780298642.487197:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000bde78cfa/0xaf35132e630330da lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2324 expref: 10 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.487225:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14993 00000100:00100000:1.0:1780298642.487231:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7be300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609379712:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298642.487239:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298642.487248:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000bde78cfa/0xaf35132e630330da lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2324 expref: 10 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.487259:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000bde78cfa/0xaf35132e630330da lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2324 expref: 10 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.487273:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bde78cfa/0xaf35132e630330da lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2324 expref: 10 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.487280:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(105 177) 00010000:00010000:1.0:1780298642.487282:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(105 177) 00010000:00010000:1.0:1780298642.487288:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e63033158 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2781184->2785279) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2394 expref: 10 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.487306:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e63033158 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2781184->2785279) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2394 expref: 10 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.487325:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e63033158 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2781184->2785279) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2394 expref: 10 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.487336:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255091180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777604878336:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298642.487370:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34a90d3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.487984:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255091180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777604878336:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298642.488006:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298642.488032:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7be300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609379712:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 801us (922us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.488040:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14993 00000100:00100000:0.0:1780298642.488209:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.488243:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609379840 00000100:00100000:1.0:1780298642.488248:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609379840 00002000:00100000:1.0:1780298642.488275:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (2781184->2785279) req@ffff9cf37f7bf800 x1866777609379840/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298642.488287:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f7bf800 with x1866777609379840 ext(2781184->2785279) 00002000:00010000:1.0:1780298642.488301:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f7bf800 00000100:00100000:1.0:1780298642.488319:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5992 00000100:00100000:1.0:1780298642.488325:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609379840:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298642.488467:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609379840:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 134us (250us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298642.488475:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5992 00002000:00100000:1.0:1780298642.488483:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (2781184->2785279) req@ffff9cf37f7bf800 x1866777609379840/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/4536 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298642.488489:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f7bf800 with x1866777609379840 ext(2781184->2785279) 00002000:00010000:1.0:1780298642.488491:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f7bf800 00000100:00100000:3.0:1780298642.489469:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.489499:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609379968 00000100:00100000:1.0:1780298642.489503:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609379968 00000100:00100000:1.0:1780298642.489527:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8881 00000100:00100000:1.0:1780298642.489532:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f747100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609379968:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.489539:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.489550:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e6303315f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d239b expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.489575:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604878464:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.489615:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34a90d3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.490229:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604878464:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.490255:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e6303315f lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d239b expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.490273:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e6303315f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d239b expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.490282:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36a00, rc 0) 00000100:00100000:1.0:1780298642.490326:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f747100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609379968:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 792us (856us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.490333:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8881 00000100:00100000:0.0:1780298642.491120:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.491162:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609380096 00000100:00100000:3.0:1780298642.491166:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609380096 00000100:00100000:3.0:1780298642.491195:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8882 00000100:00100000:3.0:1780298642.491200:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609380096:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.491207:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.491220:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e63033166 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d23a2 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.491241:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f744700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604878592:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.491284:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.491961:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f744700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604878592:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.491993:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e63033166 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23a2 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.492023:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e63033166 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23a2 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.492036:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2400, rc 0) 00000100:00100000:2.0:1780298642.492112:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609380096:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 909us (989us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.492182:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8882 00000100:00100000:0.0:1780298642.492968:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.493009:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609380224 00000100:00100000:3.0:1780298642.493014:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609380224 00000100:00100000:3.0:1780298642.493050:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5993 00000100:00100000:3.0:1780298642.493060:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609380224:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298642.493289:0:16972:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4017167 -> 4017167 00002000:00010000:2.0:1780298642.493306:0:16972:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.493309:0:16972:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.493311:0:16972:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298643 -> 1780298642 00002000:00010000:2.0:1780298642.493313:0:16972:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 24 -> 24 00000100:00100000:2.0:1780298642.493354:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609380224:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 295us (386us total) trans 4294969192 rc 0/0 00000100:00100000:2.0:1780298642.493363:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5993 00000100:00100000:0.0:1780298642.494169:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.494221:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.494230:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609380352 00000100:00100000:3.0:1780298642.494235:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609380352 00000100:00100000:2.0:1780298642.494269:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609380480 00000100:00100000:2.0:1780298642.494278:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609380480 00000100:00100000:1.0:1780298642.494303:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8883 00000100:00100000:1.0:1780298642.494316:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f755500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609380352:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.494324:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.494340:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009cb6f854/0xaf35132e6303316d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d23a9 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.494349:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8884 00010000:00010000:1.0:1780298642.494361:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(105 177) 00000100:00100000:2.0:1780298642.494369:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f756300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609380480:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.494370:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(105 177) 00010000:00010000:1.0:1780298642.494382:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009cb6f854/0xaf35132e6303316d lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23a9 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.494390:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.494397:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009cb6f854/0xaf35132e6303316d lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23a9 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.494405:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a58d48e9/0xaf35132e63033174 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d23b0 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.494406:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36e00, rc 0) 00000100:00100000:1.0:1780298642.494442:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f755500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609380352:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 131us (276us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.494447:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8883 00000100:00100000:2.0:1780298642.494464:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255492a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604878720:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.494522:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00002000:00010000:1.0:1780298642.495238:0:12101:0:(ofd_lvb.c:277:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime: 1780298642 -> 1780298643 00000100:00100000:1.0:1780298642.495252:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255492a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604878720:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.495281:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a58d48e9/0xaf35132e63033174 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23b0 expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.495309:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a58d48e9/0xaf35132e63033174 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23b0 expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.495320:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156a00, rc 0) 00000100:00100000:1.0:1780298642.495358:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f756300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609380480:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 997us (1137us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.495364:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8884 00000100:00100000:0.0:1780298642.496250:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.496284:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609380608 00000100:00100000:1.0:1780298642.496300:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609380608 00000100:00100000:1.0:1780298642.496326:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8885 00000100:00100000:1.0:1780298642.496331:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f757b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609380608:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.496340:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.496348:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078c7f460/0xaf35132e6303317b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d23b7 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.496375:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604878848:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.496412:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34a90d3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.497032:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604878848:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.497065:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000078c7f460/0xaf35132e6303317b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23b7 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.497078:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078c7f460/0xaf35132e6303317b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23b7 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.497087:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36c00, rc 0) 00000100:00100000:1.0:1780298642.497115:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f757b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609380608:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 783us (865us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.497122:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8885 00000100:00100000:2.0:1780298642.498022:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.498072:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609380736 00000100:00100000:3.0:1780298642.498078:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609380736 00000100:00100000:3.0:1780298642.498106:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8886 00000100:00100000:3.0:1780298642.498115:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255492300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609380736:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.498147:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.498160:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e63033182 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d23be expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.498193:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f745c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604878976:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.498228:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.498814:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f745c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604878976:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.498831:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e63033182 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23be expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.498843:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e63033182 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23be expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.498858:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3600, rc 0) 00000100:00100000:3.0:1780298642.498881:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255492300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609380736:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 769us (864us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.498886:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8886 00000100:00100000:0.0:1780298642.499731:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.499795:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609380864 00000100:00100000:2.0:1780298642.499803:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609380864 00000100:00100000:2.0:1780298642.499844:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8887 00000100:00100000:2.0:1780298642.499855:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f756680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609380864:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.499866:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.499877:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63033189 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d23c5 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.499918:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255491f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604879104:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.499971:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4000 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.500666:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255491f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604879104:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.500684:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63033189 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23c5 expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.500694:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63033189 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23c5 expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.500702:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157c00, rc 0) 00000100:00100000:0.0:1780298642.500735:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f756680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609380864:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 884us (1009us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.500743:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8887 00000100:00100000:0.0:1780298642.501549:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.501586:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609380992 00000100:00100000:1.0:1780298642.501591:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609380992 00000100:00100000:1.0:1780298642.501623:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8888 00000100:00100000:1.0:1780298642.501630:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f755c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609380992:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.501640:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.501651:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e63033190 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d23cc expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.501690:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604879232:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.501733:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34a90d3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.502270:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604879232:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.502306:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e63033190 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23cc expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.502316:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e63033190 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23cc expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.502322:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36a00, rc 0) 00000100:00100000:1.0:1780298642.502345:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f755c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609380992:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 716us (797us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.502352:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8888 00000100:00100000:2.0:1780298642.503163:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.503198:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609381120 00000100:00100000:3.0:1780298642.503202:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609381120 00000100:00100000:3.0:1780298642.503225:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8889 00000100:00100000:3.0:1780298642.503231:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255493480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609381120:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.503237:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.503244:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63033197 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d23d3 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.503263:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000ced6368f/0xaf35132e63033135 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [372736->33927167] (req 372736->376831) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2378 expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.503272:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(79 130) 00010000:00010000:3.0:1780298642.503277:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(80 130) 00010000:00010000:3.0:1780298642.503283:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63033197 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [618496->622591] (req 618496->622591) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23d3 expref: 10 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.503312:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000ced6368f/0xaf35132e63033135 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [372736->33927167] (req 372736->376831) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2378 expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.503323:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000ced6368f/0xaf35132e63033135 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [372736->33927167] (req 372736->376831) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2378 expref: 8 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.503339:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f745c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604879360:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298642.503384:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.503949:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f745c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604879360:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298642.503963:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63033197 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [618496->622591] (req 618496->622591) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23d3 expref: 10 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.503974:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3c00, rc 0) 00000100:00100000:3.0:1780298642.504061:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255493480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609381120:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 829us (934us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298642.504070:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8889 00000100:00100000:2.0:1780298642.504193:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.504231:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609381248 00000100:00100000:3.0:1780298642.504235:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609381248 00010000:00010000:3.0:1780298642.504250:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000ced6368f/0xaf35132e63033135 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [372736->33927167] (req 372736->376831) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2378 expref: 9 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.504275:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14994 00000100:00100000:3.0:1780298642.504282:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255490e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609381248:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298642.504302:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298642.504313:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000ced6368f/0xaf35132e63033135 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [372736->33927167] (req 372736->376831) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2378 expref: 9 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.504325:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000ced6368f/0xaf35132e63033135 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [372736->33927167] (req 372736->376831) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2378 expref: 9 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.504354:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ced6368f/0xaf35132e63033135 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [372736->33927167] (req 372736->376831) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2378 expref: 9 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.504363:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(80 130) 00010000:00010000:3.0:1780298642.504365:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(80 130) 00010000:00010000:3.0:1780298642.504371:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63033197 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 618496->622591) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23d3 expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.504376:0:6463:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63033197 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 618496->622591) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23d3 expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.504392:0:6463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63033197 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 618496->622591) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23d3 expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.504399:0:6463:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f745c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777604879488:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298642.504431:0:6463:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.505043:0:6463:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f745c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777604879488:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298642.505051:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298642.505073:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255490e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609381248:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 793us (883us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298642.505078:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14994 00000100:00100000:0.0:1780298642.505254:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.505308:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609381376 00000100:00100000:2.0:1780298642.505316:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609381376 00002000:00100000:2.0:1780298642.505352:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (618496->622591) req@ffff9cf37f754000 x1866777609381376/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.505364:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f754000 with x1866777609381376 ext(618496->622591) 00002000:00010000:2.0:1780298642.505369:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f754000 00000100:00100000:2.0:1780298642.505394:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5994 00000100:00100000:2.0:1780298642.505407:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f754000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609381376:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298642.505574:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f754000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609381376:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 172us (319us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298642.505582:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5994 00002000:00100000:2.0:1780298642.505590:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (618496->622591) req@ffff9cf37f754000 x1866777609381376/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/4536 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.505599:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f754000 with x1866777609381376 ext(618496->622591) 00002000:00010000:2.0:1780298642.505603:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f754000 00000100:00100000:2.0:1780298642.506523:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.506562:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609381504 00000100:00100000:3.0:1780298642.506567:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609381504 00000100:00100000:3.0:1780298642.506590:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8890 00000100:00100000:3.0:1780298642.506602:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255491f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609381504:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.506610:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.506619:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e6303319e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d23da expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.506635:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(80 130) 00010000:00010000:3.0:1780298642.506639:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(80 130) 00010000:00010000:3.0:1780298642.506645:0:20252:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e6303319e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23da expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.506657:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e6303319e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23da expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.506667:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2400, rc 0) 00000100:00100000:3.0:1780298642.506699:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255491f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609381504:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 97us (176us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298642.506706:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8890 00000100:00100000:0.0:1780298642.507535:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.507587:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609381632 00000100:00100000:2.0:1780298642.507593:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609381632 00000100:00100000:2.0:1780298642.507630:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8891 00000100:00100000:2.0:1780298642.507638:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f755880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:29360:x1866777609381632:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.507647:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.507657:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e630331a5 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d23e1 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.507674:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e63033158 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2781184->2785279) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2394 expref: 10 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.507681:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000009cb6f854/0xaf35132e6303316d lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23a9 expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.507685:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(105 177) 00010000:00010000:2.0:1780298642.507687:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(105 177) 00010000:00010000:2.0:1780298642.507690:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e630331a5 lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [573440->581631] (req 573440->581631) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23e1 expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.507714:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000009cb6f854/0xaf35132e6303316d lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23a9 expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.507727:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000009cb6f854/0xaf35132e6303316d lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23a9 expref: 11 pid: 8358 timeout: 931 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.507735:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255492a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604879616:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298642.507775:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e63033158 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2781184->2785279) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2394 expref: 10 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.507783:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e63033158 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2781184->2785279) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2394 expref: 10 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.507790:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255493100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604879744:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298642.507807:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4900 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.508342:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255492a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604879616:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298642.508498:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255493100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604879744:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298642.508514:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e630331a5 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [573440->581631] (req 573440->581631) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23e1 expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.508523:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157400, rc 0) 00000100:00100000:2.0:1780298642.508545:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f755880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:29360:x1866777609381632:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 909us (1011us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.508550:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8891 00000100:00100000:3.0:1780298642.508626:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.508662:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609381760 00000100:00100000:2.0:1780298642.508667:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609381760 00010000:00010000:2.0:1780298642.508681:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000009cb6f854/0xaf35132e6303316d lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23a9 expref: 11 pid: 8358 timeout: 931 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.508704:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14995 00000100:00100000:2.0:1780298642.508712:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609381760:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298642.508722:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298642.508736:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000009cb6f854/0xaf35132e6303316d lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23a9 expref: 11 pid: 8358 timeout: 931 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.508747:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000009cb6f854/0xaf35132e6303316d lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23a9 expref: 11 pid: 8358 timeout: 931 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.508762:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009cb6f854/0xaf35132e6303316d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23a9 expref: 11 pid: 8358 timeout: 931 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.508773:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298642.508798:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609381760:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 88us (173us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.508805:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14995 00000100:00100000:0.0:1780298642.508863:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.508895:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609381888 00000100:00100000:2.0:1780298642.508900:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609381888 00010000:00010000:2.0:1780298642.508916:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e63033158 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2781184->2785279) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2394 expref: 11 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.508941:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14996 00000100:00100000:2.0:1780298642.508950:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f757100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609381888:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298642.508959:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298642.508968:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e63033158 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2781184->2785279) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2394 expref: 11 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.508979:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e63033158 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2781184->2785279) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2394 expref: 11 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.508993:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e63033158 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2781184->2785279) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2394 expref: 11 pid: 8358 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.509001:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(105 177) 00010000:00010000:2.0:1780298642.509004:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(105 177) 00010000:00010000:2.0:1780298642.509008:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e630331a5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [573440->34127871] (req 573440->581631) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23e1 expref: 9 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.509013:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e630331a5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [573440->34127871] (req 573440->581631) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23e1 expref: 9 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.509028:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e630331a5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [573440->34127871] (req 573440->581631) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23e1 expref: 9 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.509035:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255493100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777604879872:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298642.509062:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4900 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.509788:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255493100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777604879872:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298642.509799:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298642.509825:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f757100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609381888:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 878us (963us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.509834:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14996 00000100:00100000:0.0:1780298642.509891:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.509941:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609382016 00000100:00100000:3.0:1780298642.509946:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609382016 00000100:00100000:3.0:1780298642.509977:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5995 00000100:00100000:3.0:1780298642.509984:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f755f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609382016:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:3.0:1780298642.510182:0:19928:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:3.0:1780298642.510187:0:19928:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:3.0:1780298642.510189:0:19928:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:3.0:1780298642.510192:0:19928:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298642 -> 1780298642 00002000:00010000:3.0:1780298642.510200:0:19928:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 392 -> 408 00000100:00100000:3.0:1780298642.510227:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f755f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609382016:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 243us (337us total) trans 4294969625 rc 0/0 00000100:00100000:3.0:1780298642.510234:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5995 00000100:00100000:3.0:1780298642.511055:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.511093:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609382144 00000100:00100000:1.0:1780298642.511099:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609382144 00000100:00100000:1.0:1780298642.511155:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8892 00000100:00100000:1.0:1780298642.511164:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f744700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609382144:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.511173:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.511194:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e630331ac lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d23e8 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.511228:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604880000:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.511264:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34a90d3c0 going to sleep for 11 seconds 00002000:00010000:1.0:1780298642.511864:0:12101:0:(ofd_lvb.c:277:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime: 1780298642 -> 1780298643 00000100:00100000:1.0:1780298642.511872:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604880000:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.511905:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e630331ac lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23e8 expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.511921:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e630331ac lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23e8 expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.511931:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36c00, rc 0) 00000100:00100000:1.0:1780298642.511960:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f744700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609382144:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 798us (906us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.511967:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8892 00000100:00100000:0.0:1780298642.512762:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.512797:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609382272 00000100:00100000:3.0:1780298642.512802:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609382272 00000100:00100000:3.0:1780298642.512834:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8893 00000100:00100000:3.0:1780298642.512841:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f754a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609382272:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.512849:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.512857:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005c47b04a/0xaf35132e630331b3 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d23ef expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.512885:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604880128:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.512923:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.513562:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604880128:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.513579:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005c47b04a/0xaf35132e630331b3 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23ef expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.513592:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005c47b04a/0xaf35132e630331b3 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23ef expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.513602:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3400, rc 0) 00000100:00100000:1.0:1780298642.513630:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f754a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609382272:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 790us (870us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.513636:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8893 00000100:00100000:2.0:1780298642.514519:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.514548:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609382400 00000100:00100000:3.0:1780298642.514552:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609382400 00000100:00100000:3.0:1780298642.514574:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8894 00000100:00100000:3.0:1780298642.514580:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255493100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609382400:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.514586:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.514593:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e630331ba lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d23f6 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.514614:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604880256:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.514644:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.515242:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604880256:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.515264:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e630331ba lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23f6 expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.515273:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e630331ba lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23f6 expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.515280:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3600, rc 0) 00000100:00100000:3.0:1780298642.515312:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255493100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609382400:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 733us (795us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.515317:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8894 00000100:00100000:1.0:1780298642.515984:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.516020:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609382528 00000100:00100000:2.0:1780298642.516029:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609382528 00000100:00100000:2.0:1780298642.516051:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8895 00000100:00100000:2.0:1780298642.516056:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255092680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609382528:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.516061:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.516068:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009c278703/0xaf35132e630331c1 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d23fd expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.516088:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255492a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604880384:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.516119:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4900 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.516777:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255492a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604880384:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.516796:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009c278703/0xaf35132e630331c1 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23fd expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.516810:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009c278703/0xaf35132e630331c1 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23fd expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.516819:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156600, rc 0) 00000100:00100000:0.0:1780298642.516847:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255092680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609382528:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 790us (863us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.516854:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8895 00000100:00100000:0.0:1780298642.517682:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.517716:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609382656 00000100:00100000:1.0:1780298642.517720:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609382656 00000100:00100000:1.0:1780298642.517750:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8896 00000100:00100000:1.0:1780298642.517759:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f755180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:29360:x1866777609382656:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.517767:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.517777:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e630331c8 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2404 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.517794:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(105 177) 00010000:00010000:1.0:1780298642.517799:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(105 177) 00010000:00010000:1.0:1780298642.517810:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e630331c8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [106496->573439] (req 106496->118783) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2404 expref: 12 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.517826:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e630331c8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [106496->573439] (req 106496->118783) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2404 expref: 12 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.517833:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36c00, rc 0) 00000100:00100000:1.0:1780298642.517857:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f755180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:29360:x1866777609382656:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 101us (176us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.517862:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8896 00000100:00100000:1.0:1780298642.518663:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.518704:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609382784 00000100:00100000:2.0:1780298642.518709:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609382784 00002000:00100000:2.0:1780298642.518738:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (106496->110591) req@ffff9cf255091180 x1866777609382784/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.518748:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255091180 with x1866777609382784 ext(106496->110591) 00002000:00010000:2.0:1780298642.518752:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255091180 00000100:00100000:2.0:1780298642.518768:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5996 00000100:00100000:2.0:1780298642.518774:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255091180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609382784:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298642.518867:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255091180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609382784:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 90us (201us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298642.518872:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5996 00002000:00100000:2.0:1780298642.518876:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (106496->110591) req@ffff9cf255091180 x1866777609382784/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/4536 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.518883:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255091180 with x1866777609382784 ext(106496->110591) 00002000:00010000:2.0:1780298642.518885:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255091180 00000100:00100000:2.0:1780298642.519705:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.519741:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609382912 00000100:00100000:1.0:1780298642.519745:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609382912 00002000:00100000:1.0:1780298642.519760:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (114688->118783) req@ffff9cf255492a00 x1866777609382912/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298642.519767:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255492a00 with x1866777609382912 ext(114688->118783) 00002000:00010000:1.0:1780298642.519769:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255492a00 00000100:00100000:1.0:1780298642.519786:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5997 00000100:00100000:1.0:1780298642.519791:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255492a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609382912:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298642.519862:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255492a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609382912:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 69us (157us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298642.519869:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5997 00002000:00100000:1.0:1780298642.519876:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (114688->118783) req@ffff9cf255492a00 x1866777609382912/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/4536 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298642.519885:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255492a00 with x1866777609382912 ext(114688->118783) 00002000:00010000:1.0:1780298642.519889:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255492a00 00000100:00100000:0.0:1780298642.520829:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.520868:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609383040 00000100:00100000:1.0:1780298642.520873:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609383040 00000100:00100000:1.0:1780298642.520908:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8897 00000100:00100000:1.0:1780298642.520919:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f756a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609383040:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.520928:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.520938:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e630331cf lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d240b expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.520987:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255093100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604880512:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.521038:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34a90d3c0 going to sleep for 11 seconds 00002000:00010000:1.0:1780298642.521672:0:8356:0:(ofd_lvb.c:265:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime: 1780298642 -> 1780298643 00000100:00100000:1.0:1780298642.521682:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255093100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604880512:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.521708:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e630331cf lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d240b expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.521724:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e630331cf lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d240b expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.521731:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37000, rc 0) 00000100:00100000:1.0:1780298642.521754:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f756a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609383040:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 840us (926us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.521759:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8897 00000100:00100000:2.0:1780298642.522561:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.522602:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609383168 00000100:00100000:3.0:1780298642.522608:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609383168 00000100:00100000:3.0:1780298642.522639:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8898 00000100:00100000:3.0:1780298642.522647:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609383168:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.522656:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.522663:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e630331d6 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2412 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.522690:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604880640:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.522729:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.523347:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604880640:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.523359:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e630331d6 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2412 expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.523368:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e630331d6 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2412 expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.523382:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3600, rc 0) 00000100:00100000:3.0:1780298642.523402:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609383168:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 758us (842us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.523407:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8898 00000100:00100000:1.0:1780298642.524091:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.524121:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609383296 00000100:00100000:0.0:1780298642.524140:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609383296 00000100:00100000:0.0:1780298642.524168:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8899 00000100:00100000:0.0:1780298642.524175:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255093100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:29360:x1866777609383296:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.524182:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.524190:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e9b43ec/0xaf35132e630331dd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2419 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.524204:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63033197 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 618496->622591) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23d3 expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.524214:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e6303319e lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23da expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.524219:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(80 130) 00010000:00010000:0.0:1780298642.524221:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(80 130) 00010000:00010000:0.0:1780298642.524224:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e9b43ec/0xaf35132e630331dd lrc: 5/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3813376->3821567] (req 3813376->3821567) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2419 expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.524238:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e6303319e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23da expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.524256:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e6303319e lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23da expref: 10 pid: 20252 timeout: 931 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.524263:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f754700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604880768:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298642.524311:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63033197 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 618496->622591) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23d3 expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.524321:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63033197 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 618496->622591) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23d3 expref: 9 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.524331:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f754380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604880896:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298642.524363:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.524918:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f754700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604880768:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298642.524924:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f754380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604880896:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298642.524936:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004e9b43ec/0xaf35132e630331dd lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3813376->3821567] (req 3813376->3821567) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2419 expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.524944:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2a00, rc 0) 00000100:00100000:0.0:1780298642.524966:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255093100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:29360:x1866777609383296:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 792us (876us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.524974:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8899 00000100:00100000:2.0:1780298642.525156:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.525186:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.525203:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609383424 00000100:00100000:3.0:1780298642.525207:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609383552 00000100:00100000:0.0:1780298642.525208:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609383424 00000100:00100000:3.0:1780298642.525213:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609383552 00010000:00010000:0.0:1780298642.525231:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63033197 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 618496->622591) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23d3 expref: 10 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.525262:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14997 00010000:00010000:2.0:1780298642.525266:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e6303319e lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23da expref: 10 pid: 20252 timeout: 931 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.525276:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad8a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609383424:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298642.525286:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1780298642.525305:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14998 00010000:00010000:0.0:1780298642.525312:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63033197 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 618496->622591) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23d3 expref: 10 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.525315:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adb480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609383552:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298642.525326:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298642.525332:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63033197 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 618496->622591) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23d3 expref: 10 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.525343:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e6303319e lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23da expref: 10 pid: 20252 timeout: 931 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.525352:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63033197 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 618496->622591) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23d3 expref: 10 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.525355:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e6303319e lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23da expref: 10 pid: 20252 timeout: 931 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.525382:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e6303319e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23da expref: 10 pid: 20252 timeout: 931 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.525383:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(80 130) 00010000:00010000:0.0:1780298642.525392:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(80 130) 00010000:00010000:0.0:1780298642.525405:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000004e9b43ec/0xaf35132e630331dd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3813376->37367807] (req 3813376->3821567) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2419 expref: 9 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.525412:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e9b43ec/0xaf35132e630331dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3813376->37367807] (req 3813376->3821567) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2419 expref: 9 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.525421:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1780298642.525432:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000004e9b43ec/0xaf35132e630331dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3813376->37367807] (req 3813376->3821567) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2419 expref: 9 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.525440:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f754380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777604881024:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298642.525452:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adb480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609383552:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 139us (265us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.525465:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14998 00000100:00100000:0.0:1780298642.525469:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.526045:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f754380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777604881024:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298642.526052:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298642.526072:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad8a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609383424:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 803us (918us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.526077:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14997 00000100:00100000:2.0:1780298642.526164:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.526201:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609383680 00000100:00100000:0.0:1780298642.526205:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609383680 00000100:00100000:0.0:1780298642.526234:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5998 00000100:00100000:0.0:1780298642.526240:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609383680:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298642.526381:0:16972:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4017167 -> 4017167 00002000:00010000:0.0:1780298642.526384:0:16972:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298642.526387:0:16972:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298642.526389:0:16972:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298643 -> 1780298642 00002000:00010000:0.0:1780298642.526392:0:16972:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 24 -> 24 00000100:00100000:0.0:1780298642.526420:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609383680:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 179us (257us total) trans 4294969193 rc 0/0 00000100:00100000:0.0:1780298642.526427:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5998 00000100:00100000:1.0:1780298642.527201:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.527231:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609383808 00000100:00100000:2.0:1780298642.527236:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609383808 00000100:00100000:1.0:1780298642.527266:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.527270:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8900 00000100:00100000:2.0:1780298642.527279:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b68700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609383808:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298642.527283:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609383936 00010000:00010000:2.0:1780298642.527287:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298642.527288:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609383936 00010000:00010000:2.0:1780298642.527303:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009c841b52/0xaf35132e630331e4 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2420 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.527327:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604881152:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.527356:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8901 00000100:00100000:3.0:1780298642.527364:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b69880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609383936:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298642.527367:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00010000:00010000:3.0:1780298642.527371:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.527379:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e630331eb lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2427 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.527403:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604881280:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.527425:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.527908:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604881152:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.527919:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009c841b52/0xaf35132e630331e4 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2420 expref: 13 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.527928:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009c841b52/0xaf35132e630331e4 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2420 expref: 13 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.527935:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157600, rc 0) 00000100:00100000:2.0:1780298642.527968:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b68700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609383808:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 690us (769us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.527976:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8900 00002000:00010000:2.0:1780298642.528046:0:19992:0:(ofd_lvb.c:265:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime: 1780298642 -> 1780298643 00002000:00010000:2.0:1780298642.528050:0:19992:0:(ofd_lvb.c:277:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime: 1780298642 -> 1780298643 00000100:00100000:2.0:1780298642.528058:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604881280:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.528076:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e630331eb lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2427 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.528088:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e630331eb lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2427 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.528096:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3600, rc 0) 00000100:00100000:2.0:1780298642.528122:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b69880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609383936:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 760us (860us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.528148:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8901 00000100:00100000:3.0:1780298642.528957:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.528980:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.528995:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609384064 00000100:00100000:2.0:1780298642.528999:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609384064 00000100:00100000:2.0:1780298642.529021:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609384192 00000100:00100000:2.0:1780298642.529022:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609384192 00000100:00100000:1.0:1780298642.529032:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8902 00000100:00100000:1.0:1780298642.529042:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609384064:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.529048:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298642.529052:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8903 00010000:00010000:1.0:1780298642.529055:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e630331f2 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d242e expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.529063:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609384192:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.529069:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.529077:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bed20494/0xaf35132e630331f9 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2435 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.529084:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b40a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604881408:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.529103:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604881536:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.529152:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34a90d3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.529180:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.529860:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604881536:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.529873:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bed20494/0xaf35132e630331f9 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2435 expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.529884:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bed20494/0xaf35132e630331f9 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2435 expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.529891:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157000, rc 0) 00000100:00100000:1.0:1780298642.529901:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b40a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604881408:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.529915:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609384192:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 857us (934us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.529921:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8903 00010000:00010000:1.0:1780298642.529941:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e630331f2 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d242e expref: 13 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.529956:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e630331f2 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d242e expref: 13 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.529964:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37000, rc 0) 00000100:00100000:1.0:1780298642.529988:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609384064:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 951us (1033us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.529994:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8902 00000100:00100000:3.0:1780298642.530916:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.530946:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.530967:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609384320 00000100:00100000:0.0:1780298642.530973:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609384320 00000100:00100000:2.0:1780298642.531005:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609384448 00000100:00100000:2.0:1780298642.531010:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609384448 00000100:00100000:0.0:1780298642.531011:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8904 00000100:00100000:0.0:1780298642.531024:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609384320:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.531032:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.531043:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e63033200 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d243c expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.531050:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8905 00000100:00100000:2.0:1780298642.531064:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609384448:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.531073:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298642.531082:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f754380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604881664:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.531083:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a58d48e9/0xaf35132e63033207 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2443 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.531115:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604881792:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.531175:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417600 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.531177:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.531882:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604881792:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.531882:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f754380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604881664:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.531898:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a58d48e9/0xaf35132e63033207 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2443 expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.531898:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e63033200 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d243c expref: 13 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.531912:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a58d48e9/0xaf35132e63033207 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2443 expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.531912:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e63033200 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d243c expref: 13 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.531923:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2e00, rc 0) 00010000:00010000:2.0:1780298642.531926:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156a00, rc 0) 00000100:00100000:2.0:1780298642.531951:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609384448:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 890us (1004us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.531951:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609384320:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 931us (1035us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.531958:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8905 00000100:00100000:0.0:1780298642.531962:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8904 00000100:00100000:3.0:1780298642.532869:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.532896:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609384576 00000100:00100000:0.0:1780298642.532900:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609384576 00000100:00100000:0.0:1780298642.532925:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8906 00000100:00100000:0.0:1780298642.532932:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:29360:x1866777609384576:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.532941:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.532949:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c703984a/0xaf35132e6303320e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d244a expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.532965:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(80 130) 00010000:00010000:0.0:1780298642.532970:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(80 130) 00010000:00010000:0.0:1780298642.532980:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c703984a/0xaf35132e6303320e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3063808->3809279] (req 3063808->3071999) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d244a expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.532994:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c703984a/0xaf35132e6303320e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3063808->3809279] (req 3063808->3071999) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d244a expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.533003:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2000, rc 0) 00000100:00100000:0.0:1780298642.533033:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:29360:x1866777609384576:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 101us (163us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.533040:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8906 00000100:00100000:3.0:1780298642.533673:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.533703:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609384704 00000100:00100000:2.0:1780298642.533706:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609384704 00000100:00100000:2.0:1780298642.533734:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 5999 00000100:00100000:2.0:1780298642.533739:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609384704:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298642.533861:0:19950:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4017167 -> 4017167 00002000:00010000:2.0:1780298642.533864:0:19950:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298643 -> 1780298642 00002000:00010000:2.0:1780298642.533865:0:19950:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:2.0:1780298642.533867:0:19950:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298643 -> 1780298642 00002000:00010000:2.0:1780298642.533868:0:19950:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 24 -> 40 00000100:00100000:2.0:1780298642.533890:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609384704:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 151us (217us total) trans 4294969194 rc 0/0 00000100:00100000:2.0:1780298642.533894:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 5999 00000100:00100000:3.0:1780298642.534721:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.534781:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609384832 00000100:00100000:1.0:1780298642.534786:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609384832 00000100:00100000:3.0:1780298642.534813:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.534823:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609384960 00000100:00100000:1.0:1780298642.534826:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609384960 00000100:00100000:2.0:1780298642.534838:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8907 00000100:00100000:2.0:1780298642.534848:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609384832:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.534856:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298642.534857:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8908 00010000:00010000:2.0:1780298642.534868:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009c841b52/0xaf35132e63033215 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2451 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.534868:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609384960:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.534877:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.534885:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e6303321c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2458 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.534902:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604881920:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.534913:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b40a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604882048:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.534948:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.534948:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34a90d3c0 going to sleep for 11 seconds 00002000:00010000:1.0:1780298642.535557:0:8358:0:(ofd_lvb.c:265:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime: 1780298642 -> 1780298643 00002000:00010000:1.0:1780298642.535561:0:8358:0:(ofd_lvb.c:277:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime: 1780298642 -> 1780298643 00000100:00100000:1.0:1780298642.535570:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b40a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604882048:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.535593:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604881920:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.535594:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e6303321c lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2458 expref: 13 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.535610:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e6303321c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2458 expref: 13 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.535620:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009c841b52/0xaf35132e63033215 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2451 expref: 13 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.535622:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36a00, rc 0) 00010000:00010000:2.0:1780298642.535639:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009c841b52/0xaf35132e63033215 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2451 expref: 13 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.535649:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157600, rc 0) 00000100:00100000:1.0:1780298642.535650:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609384960:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 785us (835us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.535656:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8908 00000100:00100000:2.0:1780298642.535678:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609384832:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 831us (955us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.535687:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8907 00000100:00100000:3.0:1780298642.536590:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.536629:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609385088 00000100:00100000:3.0:1780298642.536630:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.536633:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609385088 00000100:00100000:0.0:1780298642.536659:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609385216 00000100:00100000:0.0:1780298642.536661:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609385216 00000100:00100000:2.0:1780298642.536671:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8909 00000100:00100000:2.0:1780298642.536683:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609385088:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.536691:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298642.536691:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8910 00000100:00100000:0.0:1780298642.536701:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609385216:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.536703:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000eae09c9b/0xaf35132e63033223 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d245f expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.536709:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.536718:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bde78cfa/0xaf35132e6303322a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2466 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.536727:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604882176:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.536748:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f754380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604882304:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.536772:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.536774:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.537524:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f754380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604882304:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.537542:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bde78cfa/0xaf35132e6303322a lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2466 expref: 13 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.537552:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604882176:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.537555:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bde78cfa/0xaf35132e6303322a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2466 expref: 13 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.537565:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3a00, rc 0) 00010000:00010000:2.0:1780298642.537571:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000eae09c9b/0xaf35132e63033223 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d245f expref: 13 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.537589:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000eae09c9b/0xaf35132e63033223 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d245f expref: 13 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.537592:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609385216:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 890us (961us total) trans 0 rc 301/301 00010000:00010000:2.0:1780298642.537606:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157a00, rc 0) 00000100:00100000:0.0:1780298642.537606:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8910 00000100:00100000:2.0:1780298642.537628:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609385088:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 950us (1039us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.537633:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8909 00000100:00100000:3.0:1780298642.538551:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.538579:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609385344 00000100:00100000:2.0:1780298642.538583:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609385344 00000100:00100000:2.0:1780298642.538607:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8911 00000100:00100000:2.0:1780298642.538613:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481ffb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609385344:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.538620:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.538626:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e63033231 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d246d expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.538648:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604882432:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.538676:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.538685:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.538723:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609385472 00000100:00100000:0.0:1780298642.538728:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609385472 00000100:00100000:0.0:1780298642.538763:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8912 00000100:00100000:0.0:1780298642.538772:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481ff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609385472:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.538781:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.538788:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e2cfab1/0xaf35132e63033238 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2474 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.538819:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f754380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604882560:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.538854:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417600 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.539313:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604882432:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.539327:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e63033231 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d246d expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.539338:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e63033231 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d246d expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.539349:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157000, rc 0) 00000100:00100000:2.0:1780298642.539380:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481ffb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609385344:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 766us (827us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.539387:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8911 00000100:00100000:0.0:1780298642.539398:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f754380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604882560:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.539419:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003e2cfab1/0xaf35132e63033238 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2474 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.539435:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e2cfab1/0xaf35132e63033238 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2474 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.539446:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2e00, rc 0) 00000100:00100000:0.0:1780298642.539476:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481ff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609385472:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 706us (802us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.539484:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8912 00000100:00100000:3.0:1780298642.540256:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.540300:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609385600 00000100:00100000:0.0:1780298642.540304:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609385600 00000100:00100000:0.0:1780298642.540324:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8913 00000100:00100000:0.0:1780298642.540329:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609385600:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.540335:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.540348:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e6303323f lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d247b expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.540363:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e630331a5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [573440->34127871] (req 573440->581631) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23e1 expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.540374:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(105 177) 00010000:00010000:0.0:1780298642.540377:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(106 177) 00010000:00010000:0.0:1780298642.540380:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e6303323f lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [5627904->5640191] (req 5627904->5640191) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d247b expref: 11 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.540394:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e630331a5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [573440->34127871] (req 573440->581631) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23e1 expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.540404:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e630331a5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [573440->34127871] (req 573440->581631) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23e1 expref: 11 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.540412:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f754380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604882688:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298642.540439:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.541063:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f754380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604882688:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298642.541074:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e6303323f lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [5627904->5640191] (req 5627904->5640191) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d247b expref: 11 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.541082:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3800, rc 0) 00000100:00100000:0.0:1780298642.541102:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:29360:x1866777609385600:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 773us (849us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.541107:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8913 00000100:00100000:1.0:1780298642.541219:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.541246:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609385728 00000100:00100000:0.0:1780298642.541250:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609385728 00010000:00010000:0.0:1780298642.541264:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e630331a5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [573440->34127871] (req 573440->581631) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23e1 expref: 12 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.541284:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 14999 00000100:00100000:0.0:1780298642.541298:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b03c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609385728:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298642.541307:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298642.541315:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e630331a5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [573440->34127871] (req 573440->581631) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23e1 expref: 12 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.541325:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e630331a5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [573440->34127871] (req 573440->581631) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23e1 expref: 12 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.541337:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e630331a5 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [573440->34127871] (req 573440->581631) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d23e1 expref: 12 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.541345:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(106 177) 00010000:00010000:0.0:1780298642.541348:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(106 177) 00010000:00010000:0.0:1780298642.541356:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e6303323f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [5627904->39182335] (req 5627904->5640191) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d247b expref: 10 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.541365:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e6303323f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [5627904->39182335] (req 5627904->5640191) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d247b expref: 10 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.541392:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e6303323f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [5627904->39182335] (req 5627904->5640191) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d247b expref: 10 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.541403:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f754380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777604882816:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298642.541433:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.542059:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f754380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777604882816:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298642.542069:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298642.542093:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b03c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609385728:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 803us (876us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.542098:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 14999 00000100:00100000:3.0:1780298642.542235:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.542266:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609385856 00000100:00100000:0.0:1780298642.542270:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609385856 00000100:00100000:0.0:1780298642.542309:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6000 00000100:00100000:0.0:1780298642.542318:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481ff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609385856:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298642.542479:0:19928:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:0.0:1780298642.542483:0:19928:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298643 -> 1780298642 00002000:00010000:0.0:1780298642.542486:0:19928:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298642 -> 1780298642 00002000:00010000:0.0:1780298642.542488:0:19928:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298643 -> 1780298642 00002000:00010000:0.0:1780298642.542490:0:19928:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 408 -> 432 00000100:00100000:0.0:1780298642.542527:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481ff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2399:x1866777609385856:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 211us (293us total) trans 4294969626 rc 0/0 00000100:00100000:0.0:1780298642.542536:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6000 00000100:00100000:1.0:1780298642.543239:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.543265:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609385984 00000100:00100000:2.0:1780298642.543269:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609385984 00000100:00100000:1.0:1780298642.543270:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.543308:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609386112 00000100:00100000:2.0:1780298642.543311:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609386112 00000100:00100000:3.0:1780298642.543315:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8914 00000100:00100000:3.0:1780298642.543325:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e582450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609385984:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298642.543331:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8915 00010000:00010000:3.0:1780298642.543333:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.543342:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e63033246 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2482 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.543343:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b40a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609386112:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.543358:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.543369:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000658b0d9c/0xaf35132e6303324d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2489 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.543374:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604882944:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.543409:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f757100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604883072:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.543413:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.543440:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00002000:00010000:3.0:1780298642.544037:0:8358:0:(ofd_lvb.c:265:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime: 1780298642 -> 1780298643 00002000:00010000:3.0:1780298642.544041:0:8358:0:(ofd_lvb.c:271:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime: 1780298642 -> 1780298643 00002000:00010000:3.0:1780298642.544042:0:8358:0:(ofd_lvb.c:277:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime: 1780298642 -> 1780298643 00000100:00100000:3.0:1780298642.544049:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604882944:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.544064:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e63033246 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2482 expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.544075:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e63033246 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2482 expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.544079:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f757100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604883072:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.544091:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3a00, rc 0) 00010000:00010000:0.0:1780298642.544100:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000658b0d9c/0xaf35132e6303324d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2489 expref: 13 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.544111:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000658b0d9c/0xaf35132e6303324d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2489 expref: 13 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.544116:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e582450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609385984:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 793us (877us total) trans 0 rc 301/301 00010000:00010000:0.0:1780298642.544121:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2c00, rc 0) 00000100:00100000:3.0:1780298642.544148:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8914 00000100:00100000:0.0:1780298642.544161:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b40a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609386112:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 822us (890us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.544168:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8915 00000100:00100000:2.0:1780298642.544966:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.544994:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609386240 00000100:00100000:0.0:1780298642.544998:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609386240 00000100:00100000:2.0:1780298642.545002:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.545025:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609386368 00000100:00100000:0.0:1780298642.545028:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609386368 00000100:00100000:0.0:1780298642.545040:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8916 00000100:00100000:0.0:1780298642.545045:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609386240:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.545052:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298642.545056:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8917 00010000:00010000:0.0:1780298642.545059:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008361787d/0xaf35132e63033254 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2490 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.545067:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609386368:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.545074:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298642.545081:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f757100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604883200:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.545082:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8b6afab/0xaf35132e6303325b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2497 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.545110:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604883328:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.545115:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.545175:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.545766:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f757100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604883200:192.168.203.25@tcp:106: 00002000:00010000:2.0:1780298642.545781:0:20252:0:(ofd_lvb.c:271:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime: 1780298642 -> 1780298643 00010000:00010000:0.0:1780298642.545783:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008361787d/0xaf35132e63033254 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2490 expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.545790:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604883328:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.545795:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008361787d/0xaf35132e63033254 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2490 expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.545803:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2800, rc 0) 00010000:00010000:2.0:1780298642.545806:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e8b6afab/0xaf35132e6303325b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2497 expref: 13 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.545817:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8b6afab/0xaf35132e6303325b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2497 expref: 13 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.545827:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157800, rc 0) 00000100:00100000:0.0:1780298642.545828:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609386240:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 781us (861us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.545834:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8916 00000100:00100000:2.0:1780298642.545856:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609386368:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 793us (854us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.545861:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8917 00000100:00100000:1.0:1780298642.546717:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.546737:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.546748:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609386496 00000100:00100000:2.0:1780298642.546752:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609386496 00000100:00100000:0.0:1780298642.546766:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609386624 00000100:00100000:0.0:1780298642.546770:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609386624 00000100:00100000:2.0:1780298642.546779:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8918 00000100:00100000:2.0:1780298642.546786:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b41f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609386496:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.546793:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.546800:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e63033262 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d249e expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.546800:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8919 00000100:00100000:1.0:1780298642.546812:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609386624:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.546819:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298642.546823:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604883456:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.546827:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009cb6f854/0xaf35132e63033269 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d24a5 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.546855:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b40e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604883584:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.546859:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.546882:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34a90d3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.547493:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604883456:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.547506:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e63033262 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d249e expref: 13 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.547510:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b40e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604883584:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.547523:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e63033262 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d249e expref: 13 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.547529:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156a00, rc 0) 00010000:00010000:1.0:1780298642.547549:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009cb6f854/0xaf35132e63033269 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24a5 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.547555:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b41f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609386496:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 768us (837us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.547564:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8918 00010000:00010000:1.0:1780298642.547567:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009cb6f854/0xaf35132e63033269 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24a5 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.547579:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36e00, rc 0) 00000100:00100000:1.0:1780298642.547609:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609386624:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 800us (870us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.547618:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8919 00000100:00100000:3.0:1780298642.548314:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.548342:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609386752 00000100:00100000:3.0:1780298642.548343:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.548347:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609386752 00000100:00100000:1.0:1780298642.548372:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609386880 00000100:00100000:1.0:1780298642.548375:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609386880 00000100:00100000:0.0:1780298642.548382:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8920 00000100:00100000:2.0:1780298642.548395:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8921 00000100:00100000:0.0:1780298642.548396:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609386752:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.548405:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298642.548406:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609386880:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.548413:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.548414:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bde78cfa/0xaf35132e63033270 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d24ac expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.548420:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eae09c9b/0xaf35132e63033277 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d24b3 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.548444:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604883712:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.548450:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f757100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604883840:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.548481:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.548481:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.549098:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604883712:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.549112:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000eae09c9b/0xaf35132e63033277 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24b3 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.549122:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eae09c9b/0xaf35132e63033277 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24b3 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.549147:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157a00, rc 0) 00000100:00100000:2.0:1780298642.549175:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609386880:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 772us (833us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.549181:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8921 00000100:00100000:0.0:1780298642.549195:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f757100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604883840:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.549213:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bde78cfa/0xaf35132e63033270 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24ac expref: 13 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.549227:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bde78cfa/0xaf35132e63033270 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24ac expref: 13 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.549237:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3a00, rc 0) 00000100:00100000:0.0:1780298642.549267:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609386752:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 876us (953us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.549273:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8920 00000100:00100000:1.0:1780298642.549925:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.549957:0:6476:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609387136 00000100:00100000:2.0:1780298642.549960:0:6476:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609387136 00000100:00100000:2.0:1780298642.549988:0:6476:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61939 00000100:00100000:2.0:1780298642.549992:0:6476:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b40e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+104:29360:x1866777609387136:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:2.0:1780298642.550025:0:6476:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298642.550037:0:6476:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156e00/0xaf35132e630330ef lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 59 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2332 expref: 104 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.550046:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.550049:0:6476:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298642.550188:0:6476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157800/0xaf35132e6303327e lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 58 type: IBT flags: 0x40000000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.550198:0:6476:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37400/0xaf35132e63033143 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 58 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d237f expref: 99 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.550209:0:6476:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157800/0xaf35132e6303327e lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 58 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.550224:0:6476:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37400/0xaf35132e63033143 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 58 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d237f expref: 99 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.550233:0:6476:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37400/0xaf35132e63033143 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 58 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d237f expref: 99 pid: 7411 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.550240:0:6476:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad9500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6476:1866777604883968:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298642.550275:0:6476:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.550792:0:6476:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad9500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6476:1866777604883968:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298642.550801:0:6476:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157800/0xaf35132e6303327e lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 58 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.550928:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.550959:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609387264 00000100:00100000:0.0:1780298642.550963:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609387264 00010000:00010000:0.0:1780298642.550981:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37400/0xaf35132e63033143 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 58 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d237f expref: 100 pid: 7411 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.551006:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15000 00000100:00100000:0.0:1780298642.551013:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b42300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+100:2400:x1866777609387264:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298642.551021:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298642.551026:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37400/0xaf35132e63033143 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 59 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d237f expref: 100 pid: 7411 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.551037:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37400/0xaf35132e63033143 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 59 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d237f expref: 100 pid: 7411 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.551051:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37400/0xaf35132e63033143 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 59 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d237f expref: 100 pid: 7411 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.551061:0:20253:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298642.551066:0:20253:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157800/0xaf35132e6303327e lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 58 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.551081:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157800/0xaf35132e6303327e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 58 type: IBT flags: 0x40290000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.551088:0:20253:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157800/0xaf35132e6303327e lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 58 type: IBT flags: 0x40290000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.551105:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298642.551122:0:6476:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157800/0xaf35132e6303327e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 57 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.551145:0:6476:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157800/0xaf35132e6303327e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 57 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.551146:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b42300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+99:2400:x1866777609387264:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 133us (217us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.551159:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15000 00000001:00080000:2.0:1780298642.551192:0:6476:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991018 00000004:00010000:2.0:1780298642.551208:0:6476:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 000000000f85b591 reply state 000000005552332b transno 4294991019 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157800/0xaf35132e6303327e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 57 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.551216:0:6476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157800/0xaf35132e6303327e lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 57 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.551220:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.551222:0:6476:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1780298642.551224:0:6476:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1780298642.551271:0:6476:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b40e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+106:29360:x1866777609387136:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1276us (1345us total) trans 4294991019 rc 0/0 00000100:00100000:2.0:1780298642.551279:0:6476:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61939 00000020:00080000:2.0:1780298642.551283:0:6476:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1780298642.551839:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.551866:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609387392 00000100:00100000:3.0:1780298642.551870:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609387392 00000100:00100000:3.0:1780298642.551893:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8922 00000100:00100000:3.0:1780298642.551900:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b43100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:29360:x1866777609387392:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.551908:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.551916:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e63033285 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d24ba expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.551933:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e630331c8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [106496->573439] (req 106496->118783) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2404 expref: 10 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.551941:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e6303323f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [5627904->39182335] (req 5627904->5640191) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d247b expref: 13 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.551948:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 2 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(106 177) 00010000:00010000:3.0:1780298642.551951:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(108 177) 00010000:00010000:3.0:1780298642.551956:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e63033285 lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24ba expref: 13 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.551975:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e6303323f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [5627904->39182335] (req 5627904->5640191) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d247b expref: 13 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.551986:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e6303323f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [5627904->39182335] (req 5627904->5640191) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d247b expref: 13 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.552005:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604884096:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298642.552048:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e630331c8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [106496->573439] (req 106496->118783) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2404 expref: 10 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.552058:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e630331c8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [106496->573439] (req 106496->118783) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2404 expref: 10 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.552068:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604884224:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298642.552096:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.552693:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604884096:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298642.552703:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604884224:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298642.552714:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e63033285 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24ba expref: 13 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.552722:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3a00, rc 0) 00000100:00100000:3.0:1780298642.552743:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:29360:x1866777609387392:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 844us (905us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298642.552750:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8922 00000100:00100000:1.0:1780298642.552949:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.552985:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609387648 00000100:00100000:2.0:1780298642.552990:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609387648 00000100:00100000:3.0:1780298642.552992:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:2.0:1780298642.553005:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e6303323f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [5627904->39182335] (req 5627904->5640191) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d247b expref: 13 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.553028:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15001 00000100:00100000:0.0:1780298642.553032:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609387520 00000100:00100000:2.0:1780298642.553036:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b40000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609387648:12345-192.168.203.25@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1780298642.553037:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609387520 00010000:00010000:2.0:1780298642.553047:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298642.553058:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e6303323f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [5627904->39182335] (req 5627904->5640191) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d247b expref: 13 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00100000:0.0:1780298642.553061:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (106496->118783) req@ffff9cf3481fdc00 x1866777609387520/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 12776/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.553086:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fdc00 with x1866777609387520 ext(106496->118783) 00010000:00010000:2.0:1780298642.553092:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e6303323f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [5627904->39182335] (req 5627904->5640191) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d247b expref: 13 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.553093:0:8362:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e630331c8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [106496->573439] (req 106496->118783) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2404 expref: 11 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.553118:0:8362:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e630331c8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [106496->573439] (req 106496->118783) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2404 expref: 11 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.553142:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e6303323f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [5627904->39182335] (req 5627904->5640191) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d247b expref: 13 pid: 8356 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1780298642.553143:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf3481fdc00 00010000:00010000:2.0:1780298642.553157:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298642.553162:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 184 00000100:00100000:0.0:1780298642.553169:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609387520:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:2.0:1780298642.553182:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b40000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609387648:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 147us (232us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.553190:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15001 00000100:00100000:0.0:1780298642.553556:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609387520:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 388us (564us total) trans 4294969627 rc 0/0 00000100:00100000:0.0:1780298642.553564:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 184 00002000:00100000:0.0:1780298642.553569:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (106496->118783) req@ffff9cf3481fdc00 x1866777609387520/t4294969627(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 12776/448 e 0 to 0 dl 1780298653 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.553575:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fdc00 with x1866777609387520 ext(106496->118783) 00010000:00010000:0.0:1780298642.553583:0:8362:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e630331c8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [106496->573439] (req 106496->118783) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2404 expref: 13 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.553592:0:8362:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e630331c8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [106496->573439] (req 106496->118783) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2404 expref: 13 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1780298642.553598:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf3481fdc00 00000100:00100000:3.0:1780298642.554281:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.554318:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609387776 00000100:00100000:2.0:1780298642.554322:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609387776 00010000:00010000:2.0:1780298642.554335:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e630331c8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [106496->573439] (req 106496->118783) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2404 expref: 13 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.554353:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15002 00000100:00100000:2.0:1780298642.554358:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2397:x1866777609387776:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298642.554364:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298642.554371:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e630331c8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [106496->573439] (req 106496->118783) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2404 expref: 13 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.554377:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e630331c8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [106496->573439] (req 106496->118783) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2404 expref: 13 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.554387:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e630331c8 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [106496->573439] (req 106496->118783) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2404 expref: 13 pid: 12101 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.554394:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(108 177) 00010000:00010000:2.0:1780298642.554396:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(108 177) 00010000:00010000:2.0:1780298642.554400:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e63033285 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24ba expref: 11 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.554406:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e63033285 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24ba expref: 11 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.554421:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e63033285 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24ba expref: 11 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.554428:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad9500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777604884352:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298642.554459:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.555068:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad9500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777604884352:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298642.555080:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298642.555106:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609387776:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 747us (825us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.555111:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15002 00000100:00100000:3.0:1780298642.555195:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.555237:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609387904 00000100:00100000:1.0:1780298642.555241:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609387904 00000100:00100000:1.0:1780298642.555268:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8923 00000100:00100000:1.0:1780298642.555274:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609387904:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.555281:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.555300:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009cb6f854/0xaf35132e6303328c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d24c1 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.555316:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000c703984a/0xaf35132e6303320e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3063808->3809279] (req 3063808->3071999) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d244a expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.555322:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000004e9b43ec/0xaf35132e630331dd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3813376->37367807] (req 3813376->3821567) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2419 expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.555330:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 2 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(80 130) 00010000:00010000:1.0:1780298642.555333:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(82 130) 00010000:00010000:1.0:1780298642.555337:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009cb6f854/0xaf35132e6303328c lrc: 5/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24c1 expref: 10 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.555361:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000004e9b43ec/0xaf35132e630331dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3813376->37367807] (req 3813376->3821567) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2419 expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.555374:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000004e9b43ec/0xaf35132e630331dd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3813376->37367807] (req 3813376->3821567) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2419 expref: 11 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.555384:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b41180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604884480:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298642.555425:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000c703984a/0xaf35132e6303320e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3063808->3809279] (req 3063808->3071999) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d244a expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.555436:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000c703984a/0xaf35132e6303320e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3063808->3809279] (req 3063808->3071999) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d244a expref: 11 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.555446:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604884608:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298642.555472:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34a90d3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.555947:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b41180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604884480:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298642.556265:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.556279:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604884608:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298642.556310:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609388032 00000100:00100000:2.0:1780298642.556316:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609388032 00010000:00010000:1.0:1780298642.556323:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009cb6f854/0xaf35132e6303328c lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24c1 expref: 10 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.556332:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36e00, rc 0) 00010000:00010000:2.0:1780298642.556337:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000004e9b43ec/0xaf35132e630331dd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3813376->37367807] (req 3813376->3821567) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2419 expref: 12 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.556349:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.556363:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609387904:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1088us (1168us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.556368:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8923 00000100:00100000:2.0:1780298642.556370:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609388160 00000100:00100000:2.0:1780298642.556376:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609388160 00010000:00010000:2.0:1780298642.556392:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000c703984a/0xaf35132e6303320e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3063808->3809279] (req 3063808->3071999) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d244a expref: 13 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.556422:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15003 00000100:00100000:3.0:1780298642.556435:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15004 00000100:00100000:2.0:1780298642.556435:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609388032:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298642.556448:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1780298642.556452:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fd180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609388160:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298642.556461:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000004e9b43ec/0xaf35132e630331dd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [3813376->37367807] (req 3813376->3821567) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2419 expref: 13 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.556463:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298642.556476:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000004e9b43ec/0xaf35132e630331dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [3813376->37367807] (req 3813376->3821567) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2419 expref: 13 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.556492:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000c703984a/0xaf35132e6303320e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [3063808->3809279] (req 3063808->3071999) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d244a expref: 13 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.556495:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e9b43ec/0xaf35132e630331dd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [3813376->37367807] (req 3813376->3821567) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2419 expref: 13 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.556523:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000c703984a/0xaf35132e6303320e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [3063808->3809279] (req 3063808->3071999) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d244a expref: 12 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.556534:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1780298642.556543:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c703984a/0xaf35132e6303320e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3063808->3809279] (req 3063808->3071999) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d244a expref: 12 pid: 9128 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.556557:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(82 130) 00000100:00100000:2.0:1780298642.556558:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609388032:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 127us (293us total) trans 0 rc 0/0 00010000:00010000:3.0:1780298642.556567:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(82 130) 00000100:00100000:2.0:1780298642.556571:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15003 00010000:00010000:3.0:1780298642.556577:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009cb6f854/0xaf35132e6303328c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24c1 expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.556587:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009cb6f854/0xaf35132e6303328c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24c1 expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.556613:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000009cb6f854/0xaf35132e6303328c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24c1 expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.556625:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fc380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777604884736:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298642.556658:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.557244:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fc380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777604884736:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298642.557333:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298642.557379:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.557418:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609388288 00000100:00100000:1.0:1780298642.557422:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609388288 00002000:00010000:1.0:1780298642.557436:0:19949:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:1.0:1780298642.557439:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f757100 with x1866777609388288 ext(4194304->18446744073709551615) 00002000:00010000:1.0:1780298642.557444:0:19949:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f757100. 00000100:00100000:1.0:1780298642.557461:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6001 00000100:00100000:3.0:1780298642.557466:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fd180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609388160:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 917us (1019us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.557469:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f757100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2399:x1866777609388288:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:3.0:1780298642.557482:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15004 00000100:00100000:0.0:1780298642.557487:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.557514:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609388416 00000100:00100000:2.0:1780298642.557518:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609388416 00002000:00010000:2.0:1780298642.557529:0:16972:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (1502074->18446744073709551615) 00002000:00100000:2.0:1780298642.557532:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f754380 with x1866777609388416 ext(1502074->18446744073709551615) 00002000:00010000:2.0:1780298642.557536:0:16972:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f754380. 00000100:00100000:2.0:1780298642.557555:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6002 00000100:00100000:2.0:1780298642.557562:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f754380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609388416:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:1.0:1780298642.557568:0:19949:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:1.0:1780298642.557572:0:19949:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298642.557574:0:19949:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298642.557575:0:19949:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298642.557577:0:19949:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 432 -> 432 00000100:00100000:1.0:1780298642.557602:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f757100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609388288:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 135us (226us total) trans 4294969628 rc 0/0 00000100:00100000:1.0:1780298642.557608:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6001 00002000:00010000:1.0:1780298642.557611:0:19949:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:1.0:1780298642.557613:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f757100 with x1866777609388288 ext(4194304->18446744073709551615) 00002000:00010000:1.0:1780298642.557616:0:19949:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f757100. 00002000:00010000:2.0:1780298642.557740:0:16972:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4017167 -> 1502074 00002000:00010000:2.0:1780298642.557744:0:16972:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:2.0:1780298642.557746:0:16972:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298643 -> 1780298642 00002000:00010000:2.0:1780298642.557748:0:16972:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:2.0:1780298642.557751:0:16972:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 40 -> 8 00000100:00100000:2.0:1780298642.557779:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f754380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609388416:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 218us (294us total) trans 4294969195 rc 0/0 00000100:00100000:2.0:1780298642.557787:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6002 00002000:00010000:2.0:1780298642.557791:0:16972:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (1502074->18446744073709551615) 00002000:00100000:2.0:1780298642.557794:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f754380 with x1866777609388416 ext(1502074->18446744073709551615) 00002000:00010000:2.0:1780298642.557799:0:16972:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f754380. 00000100:00100000:0.0:1780298642.558455:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.558498:0:9976:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609388544 00000100:00100000:2.0:1780298642.558503:0:9976:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609388544 00000100:00100000:2.0:1780298642.558535:0:9976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61940 00000100:00100000:2.0:1780298642.558543:0:9976:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f754700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+99:29360:x1866777609388544:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.558553:0:9976:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.558564:0:9976:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156c00/0xaf35132e63033293 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 58 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d24c8 expref: -99 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.558636:0:9976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156800/0xaf35132e6303329a lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 59 type: IBT flags: 0x40000000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.558648:0:9976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156800/0xaf35132e6303329a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 59 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.558658:0:9976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156800/0xaf35132e6303329a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 59 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298642.558681:0:9976:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 5696378 00000004:00200000:2.0:1780298642.558685:0:9976:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298642.558699:0:9976:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156800/0xaf35132e6303329a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 59 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.558719:0:9976:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156c00/0xaf35132e63033293 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 59 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24c8 expref: 101 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.558730:0:9976:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156800/0xaf35132e6303329a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 58 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24c8 expref: 100 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.558739:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.558742:0:9976:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298642.558745:0:9976:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156800, rc 0) 00000100:00100000:2.0:1780298642.558771:0:9976:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f754700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+100:29360:x1866777609388544:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 229us (316us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.558777:0:9976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61940 00000100:00100000:0.0:1780298642.559505:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.559577:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609388672 00000100:00100000:3.0:1780298642.559580:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609388672 00000100:00100000:0.0:1780298642.559605:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.559635:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609388800 00000100:00100000:3.0:1780298642.559639:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609388800 00000100:00100000:2.0:1780298642.559646:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8924 00000100:00100000:2.0:1780298642.559654:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f755500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609388672:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.559663:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298642.559668:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8925 00010000:00010000:2.0:1780298642.559672:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e630332a1 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d24cf expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.559674:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f756300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609388800:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.559681:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.559688:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e630332a8 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d24d6 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.559703:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604884864:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.559707:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604884992:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.559744:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.559744:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.560534:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604884864:192.168.203.25@tcp:106: 00002000:00010000:3.0:1780298642.560536:0:9128:0:(ofd_lvb.c:271:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime: 1780298642 -> 1780298643 00000100:00100000:3.0:1780298642.560546:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604884992:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.560552:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e630332a1 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24cf expref: 13 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.560562:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e630332a8 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24d6 expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.560566:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e630332a1 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24cf expref: 13 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.560585:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157c00, rc 0) 00010000:00010000:3.0:1780298642.560588:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e630332a8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24d6 expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.560597:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3800, rc 0) 00000100:00100000:2.0:1780298642.560611:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f755500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609388672:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 958us (1105us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.560616:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8924 00000100:00100000:3.0:1780298642.560625:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f756300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609388800:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 950us (1019us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.560631:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8925 00000100:00100000:0.0:1780298642.561520:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.561548:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.561560:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609388928 00000100:00100000:3.0:1780298642.561566:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609388928 00000100:00100000:3.0:1780298642.561591:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609389056 00000100:00100000:3.0:1780298642.561594:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609389056 00000100:00100000:1.0:1780298642.561605:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8926 00000100:00100000:1.0:1780298642.561616:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f757b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609388928:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298642.561621:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8927 00010000:00010000:1.0:1780298642.561625:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298642.561632:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f756680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609389056:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.561634:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e630332af lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d24dd expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.561640:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.561650:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004ba675d2/0xaf35132e630332b6 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d24e4 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.561660:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604885120:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.561685:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604885248:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.561698:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34a90d3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.561712:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.562376:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604885248:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.562388:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604885120:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.562396:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004ba675d2/0xaf35132e630332b6 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24e4 expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.562408:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e630332af lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24dd expref: 13 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.562412:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004ba675d2/0xaf35132e630332b6 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24e4 expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.562427:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e630332af lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24dd expref: 13 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.562430:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156e00, rc 0) 00010000:00010000:1.0:1780298642.562442:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37000, rc 0) 00000100:00100000:2.0:1780298642.562456:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f756680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609389056:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 828us (908us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.562468:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8927 00000100:00100000:1.0:1780298642.562472:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f757b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609388928:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 855us (948us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.562481:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8926 00000100:00100000:0.0:1780298642.563348:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.563374:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609389184 00000100:00100000:1.0:1780298642.563377:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609389184 00000100:00100000:1.0:1780298642.563399:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8928 00000100:00100000:1.0:1780298642.563403:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f755c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:29360:x1866777609389184:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.563408:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.563414:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e630332bd lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d24eb expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.563431:0:19992:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e63033285 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24ba expref: 12 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.563439:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(108 177) 00010000:00010000:1.0:1780298642.563443:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(109 177) 00010000:00010000:1.0:1780298642.563448:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e630332bd lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3858432->3870719] (req 3858432->3870719) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24eb expref: 13 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.563468:0:19992:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e63033285 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24ba expref: 12 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.563480:0:19992:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e63033285 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24ba expref: 12 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.563487:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604885376:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298642.563516:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34a90d3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.564023:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604885376:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298642.564043:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e630332bd lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3858432->3870719] (req 3858432->3870719) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24eb expref: 13 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.564050:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37200, rc 0) 00000100:00100000:1.0:1780298642.564071:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f755c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:29360:x1866777609389184:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 668us (724us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.564078:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8928 00000100:00100000:0.0:1780298642.564236:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.564262:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609389312 00000100:00100000:1.0:1780298642.564266:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609389312 00010000:00010000:1.0:1780298642.564279:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e63033285 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24ba expref: 13 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.564306:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15005 00000100:00100000:1.0:1780298642.564312:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f755880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609389312:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298642.564319:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298642.564326:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e63033285 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24ba expref: 13 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.564332:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e63033285 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24ba expref: 13 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.564343:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e63033285 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24ba expref: 13 pid: 8359 timeout: 931 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.564349:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(109 177) 00010000:00010000:1.0:1780298642.564352:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(109 177) 00010000:00010000:1.0:1780298642.564356:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e630332bd lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3858432->37412863] (req 3858432->3870719) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24eb expref: 12 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.564361:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e630332bd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3858432->37412863] (req 3858432->3870719) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24eb expref: 12 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.564379:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e630332bd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3858432->37412863] (req 3858432->3870719) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24eb expref: 12 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.564386:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777604885504:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298642.564410:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34a90d3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.565009:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777604885504:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298642.565022:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298642.565046:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f755880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609389312:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 733us (810us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.565054:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15005 00000100:00100000:0.0:1780298642.565281:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.565315:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609389440 00000100:00100000:2.0:1780298642.565319:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609389440 00002000:00100000:2.0:1780298642.565338:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (3858432->3862527) req@ffff9cf37f754a80 x1866777609389440/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.565345:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f754a80 with x1866777609389440 ext(3858432->3862527) 00002000:00010000:2.0:1780298642.565349:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f754a80 00000100:00100000:2.0:1780298642.565365:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6003 00000100:00100000:2.0:1780298642.565371:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f754a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2397:x1866777609389440:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298642.565491:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f754a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2397:x1866777609389440:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 119us (210us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298642.565500:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6003 00002000:00100000:2.0:1780298642.565506:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (3858432->3862527) req@ffff9cf37f754a80 x1866777609389440/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/4536 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.565515:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f754a80 with x1866777609389440 ext(3858432->3862527) 00002000:00010000:2.0:1780298642.565519:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f754a80 00000100:00100000:0.0:1780298642.566222:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.566256:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609389568 00000100:00100000:3.0:1780298642.566261:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609389568 00002000:00100000:3.0:1780298642.566280:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (3866624->3870719) req@ffff9cf37f755180 x1866777609389568/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298642.566301:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f755180 with x1866777609389568 ext(3866624->3870719) 00002000:00010000:3.0:1780298642.566305:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f755180 00000100:00100000:3.0:1780298642.566323:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6004 00000100:00100000:3.0:1780298642.566332:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f755180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2397:x1866777609389568:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298642.566405:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f755180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2397:x1866777609389568:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 75us (182us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1780298642.566410:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6004 00002000:00100000:3.0:1780298642.566414:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (3866624->3870719) req@ffff9cf37f755180 x1866777609389568/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/4536 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298642.566420:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f755180 with x1866777609389568 ext(3866624->3870719) 00002000:00010000:3.0:1780298642.566422:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f755180 00000100:00100000:0.0:1780298642.567229:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.567265:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609389696 00000100:00100000:2.0:1780298642.567270:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609389696 00000100:00100000:2.0:1780298642.567311:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61941 00000100:00100000:2.0:1780298642.567320:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f756a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+106:29360:x1866777609389696:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.567333:0:6477:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.567347:0:6477:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157a00/0xaf35132e630332c4 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 59 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d24f2 expref: -99 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.567412:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156600/0xaf35132e630332cb lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 60 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.567425:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156600/0xaf35132e630332cb lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 60 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.567436:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156600/0xaf35132e630332cb lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 60 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298642.567465:0:6477:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 5696378 00000004:00200000:2.0:1780298642.567469:0:6477:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298642.567483:0:6477:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156600/0xaf35132e630332cb lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 60 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.567499:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157a00/0xaf35132e630332c4 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 60 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24f2 expref: 108 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.567508:0:6477:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156600/0xaf35132e630332cb lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 59 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24f2 expref: 107 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.567515:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.567518:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298642.567520:0:6477:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156600, rc 0) 00000100:00100000:2.0:1780298642.567555:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f756a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+107:29360:x1866777609389696:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 237us (326us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.567562:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61941 00000100:00100000:0.0:1780298642.568306:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.568323:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609389824 00000100:00100000:3.0:1780298642.568327:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609389824 00000100:00100000:3.0:1780298642.568351:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8929 00000100:00100000:3.0:1780298642.568358:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609389824:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.568373:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.568382:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b93eb96b/0xaf35132e630332d2 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d24f9 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.568414:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604885632:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.568447:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.569059:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604885632:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.569075:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b93eb96b/0xaf35132e630332d2 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24f9 expref: 13 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.569089:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b93eb96b/0xaf35132e630332d2 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24f9 expref: 13 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.569099:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3800, rc 0) 00000100:00100000:3.0:1780298642.569154:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609389824:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 795us (847us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.569162:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8929 00000100:00100000:0.0:1780298642.569928:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.569963:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609389952 00000100:00100000:2.0:1780298642.569968:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609389952 00000100:00100000:2.0:1780298642.569997:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8930 00000100:00100000:2.0:1780298642.570008:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609389952:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.570016:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.570024:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e630332d9 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2500 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.570049:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604885760:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.570085:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.570729:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604885760:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.570741:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e630332d9 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2500 expref: 13 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.570752:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e630332d9 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2500 expref: 13 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.570758:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157c00, rc 0) 00000100:00100000:2.0:1780298642.570781:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609389952:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 778us (853us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.570786:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8930 00000100:00100000:0.0:1780298642.571593:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.571611:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.571641:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609390080 00000100:00100000:2.0:1780298642.571646:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609390208 00000100:00100000:1.0:1780298642.571646:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609390080 00000100:00100000:2.0:1780298642.571651:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609390208 00000100:00100000:1.0:1780298642.571686:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8931 00000100:00100000:2.0:1780298642.571694:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8932 00000100:00100000:1.0:1780298642.571699:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2397:x1866777609390080:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298642.571706:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609390208:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.571709:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.571717:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.571720:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e630332e0 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2507 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.571725:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004ba675d2/0xaf35132e630332e7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d250e expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.571754:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604885888:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.571755:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604886016:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.571797:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.571799:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34a90d3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.572503:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604886016:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.572518:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004ba675d2/0xaf35132e630332e7 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d250e expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.572523:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604885888:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.572534:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004ba675d2/0xaf35132e630332e7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d250e expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.572541:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156e00, rc 0) 00010000:00010000:1.0:1780298642.572558:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e630332e0 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2507 expref: 14 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.572567:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609390208:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 863us (952us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.572581:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8932 00010000:00010000:1.0:1780298642.572587:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e630332e0 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2507 expref: 14 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.572598:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37000, rc 0) 00000100:00100000:1.0:1780298642.572630:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2397:x1866777609390080:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 937us (1036us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.572637:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8931 00000100:00100000:0.0:1780298642.573484:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.573509:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609390336 00000100:00100000:3.0:1780298642.573515:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609390336 00000100:00100000:3.0:1780298642.573548:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8933 00000100:00100000:3.0:1780298642.573557:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7be300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:29360:x1866777609390336:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.573565:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.573575:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ced6368f/0xaf35132e630332ee lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2515 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.573608:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000009cb6f854/0xaf35132e6303328c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24c1 expref: 10 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.573617:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(82 130) 00010000:00010000:3.0:1780298642.573620:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(83 130) 00010000:00010000:3.0:1780298642.573625:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ced6368f/0xaf35132e630332ee lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [290816->294911] (req 290816->294911) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2515 expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.573647:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000009cb6f854/0xaf35132e6303328c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24c1 expref: 10 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.573661:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000009cb6f854/0xaf35132e6303328c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24c1 expref: 10 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.573672:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604886144:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298642.573713:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.574313:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604886144:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298642.574327:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ced6368f/0xaf35132e630332ee lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [290816->294911] (req 290816->294911) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2515 expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.574339:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2a00, rc 0) 00000100:00100000:3.0:1780298642.574366:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7be300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:29360:x1866777609390336:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 812us (883us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298642.574383:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8933 00000100:00100000:0.0:1780298642.574515:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.574564:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609390464 00000100:00100000:2.0:1780298642.574569:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609390464 00010000:00010000:2.0:1780298642.574590:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000009cb6f854/0xaf35132e6303328c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24c1 expref: 11 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.574617:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15006 00000100:00100000:2.0:1780298642.574623:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b02850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609390464:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298642.574633:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298642.574658:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000009cb6f854/0xaf35132e6303328c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24c1 expref: 11 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.574669:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000009cb6f854/0xaf35132e6303328c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24c1 expref: 11 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.574682:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009cb6f854/0xaf35132e6303328c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24c1 expref: 11 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.574689:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(83 130) 00010000:00010000:2.0:1780298642.574691:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(83 130) 00010000:00010000:2.0:1780298642.574698:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000ced6368f/0xaf35132e630332ee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [290816->33845247] (req 290816->294911) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2515 expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.574707:0:6463:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ced6368f/0xaf35132e630332ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [290816->33845247] (req 290816->294911) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2515 expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.574728:0:6463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000ced6368f/0xaf35132e630332ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [290816->33845247] (req 290816->294911) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2515 expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.574740:0:6463:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad9500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777604886272:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298642.574771:0:6463:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.575494:0:6463:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad9500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777604886272:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298642.575506:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298642.575532:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b02850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609390464:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 909us (1016us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.575544:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15006 00000100:00100000:0.0:1780298642.575640:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.575674:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609390592 00000100:00100000:2.0:1780298642.575679:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609390592 00002000:00100000:2.0:1780298642.575702:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (290816->294911) req@ffff9cf255492300 x1866777609390592/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.575711:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255492300 with x1866777609390592 ext(290816->294911) 00002000:00010000:2.0:1780298642.575715:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf255492300 00000100:00100000:2.0:1780298642.575737:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6005 00000100:00100000:2.0:1780298642.575745:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255492300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609390592:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298642.575877:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255492300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609390592:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 133us (238us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298642.575884:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6005 00002000:00100000:2.0:1780298642.575891:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (290816->294911) req@ffff9cf255492300 x1866777609390592/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/4536 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.575899:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255492300 with x1866777609390592 ext(290816->294911) 00002000:00010000:2.0:1780298642.575903:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf255492300 00000100:00100000:0.0:1780298642.576902:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.576943:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609390720 00000100:00100000:1.0:1780298642.576948:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609390720 00002000:00100000:1.0:1780298642.576972:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (290816->294911) req@ffff9cf255493480 x1866777609390720/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 4584/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298642.576983:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255493480 with x1866777609390720 ext(290816->294911) 00002000:00010000:1.0:1780298642.576986:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf255493480 00000100:00100000:1.0:1780298642.577009:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6006 00000100:00100000:1.0:1780298642.577017:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255493480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609390720:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:1.0:1780298642.577414:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255493480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2397:x1866777609390720:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 399us (513us total) trans 4294969196 rc 0/0 00000100:00100000:1.0:1780298642.577419:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6006 00002000:00100000:1.0:1780298642.577425:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (290816->294911) req@ffff9cf255493480 x1866777609390720/t4294969196(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 4584/448 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298642.577431:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255493480 with x1866777609390720 ext(290816->294911) 00002000:00010000:1.0:1780298642.577434:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf255493480 00000100:00100000:0.0:1780298642.578106:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.578158:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609390848 00000100:00100000:2.0:1780298642.578162:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609390848 00000100:00100000:2.0:1780298642.578184:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8934 00000100:00100000:2.0:1780298642.578189:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255491f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2398:x1866777609390848:12345-192.168.203.25@tcp:16:fsx.0 00000001:00080000:0.0:1780298642.578896:0:8351:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0000: transno 4294969196 is committed 00000001:00080000:0.0:1780298642.578913:0:8351:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0000: transno 4294969195 is committed 00000100:00100000:2.0:1780298642.578989:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255491f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609390848:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 798us (882us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.578994:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8934 00000100:00100000:0.0:1780298642.579875:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.579896:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.579919:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609390976 00000100:00100000:3.0:1780298642.579924:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609390976 00000100:00100000:2.0:1780298642.579929:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609391104 00000100:00100000:2.0:1780298642.579935:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609391104 00000100:00100000:3.0:1780298642.579960:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8935 00000100:00100000:3.0:1780298642.579970:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255493100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609390976:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298642.579977:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8936 00010000:00010000:3.0:1780298642.579980:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298642.579989:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255490e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609391104:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.579993:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b93eb96b/0xaf35132e630332f5 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d251c expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.579998:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.580008:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eae09c9b/0xaf35132e630332fc lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2523 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.580039:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604886400:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.580039:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604886528:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.580102:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.580107:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00002000:00010000:2.0:1780298642.580916:0:20252:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks: 8 -> 16 00000100:00100000:2.0:1780298642.580928:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604886528:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.580949:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604886400:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.580952:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000eae09c9b/0xaf35132e630332fc lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2523 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.580971:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b93eb96b/0xaf35132e630332f5 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d251c expref: 13 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.580973:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eae09c9b/0xaf35132e630332fc lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2523 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.580997:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157a00, rc 0) 00010000:00010000:3.0:1780298642.581000:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b93eb96b/0xaf35132e630332f5 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d251c expref: 13 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.581013:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3800, rc 0) 00000100:00100000:2.0:1780298642.581026:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255490e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609391104:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1040us (1126us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.581033:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8936 00000100:00100000:3.0:1780298642.581041:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255493100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609390976:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1072us (1165us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.581048:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8935 00000100:00100000:0.0:1780298642.582037:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.582064:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609391232 00000100:00100000:0.0:1780298642.582066:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.582069:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609391232 00000100:00100000:3.0:1780298642.582094:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609391360 00000100:00100000:3.0:1780298642.582102:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609391360 00000100:00100000:1.0:1780298642.582108:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8937 00000100:00100000:1.0:1780298642.582118:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609391232:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298642.582142:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8938 00000100:00100000:3.0:1780298642.582151:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609391360:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.582159:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.582168:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e63033303 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2531 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.582172:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.582182:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e6303330a lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d252a expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.582207:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604886656:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.582235:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604886784:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.582255:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34a90d3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.582280:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.582933:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604886656:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.582952:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e6303330a lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d252a expref: 13 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.582962:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e6303330a lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d252a expref: 13 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.582969:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37000, rc 0) 00000100:00100000:1.0:1780298642.582996:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609391232:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 881us (958us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.583001:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8937 00000100:00100000:2.0:1780298642.583090:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604886784:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.583106:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e63033303 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2531 expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.583115:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e63033303 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2531 expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.583162:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2200, rc 0) 00000100:00100000:2.0:1780298642.583186:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609391360:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1038us (1120us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.583192:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8938 00000100:00100000:3.0:1780298642.584012:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.584052:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609391488 00000100:00100000:1.0:1780298642.584057:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609391488 00000100:00100000:1.0:1780298642.584089:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8939 00000100:00100000:1.0:1780298642.584098:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:29360:x1866777609391488:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.584108:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.584117:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009cb6f854/0xaf35132e63033311 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2538 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.584153:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(109 177) 00010000:00010000:1.0:1780298642.584158:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(109 177) 00010000:00010000:1.0:1780298642.584169:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009cb6f854/0xaf35132e63033311 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1470464->3858431] (req 1470464->1474559) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2538 expref: 13 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.584185:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009cb6f854/0xaf35132e63033311 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1470464->3858431] (req 1470464->1474559) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2538 expref: 13 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.584194:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36e00, rc 0) 00000100:00100000:1.0:1780298642.584226:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:29360:x1866777609391488:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 129us (214us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.584233:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8939 00000100:00100000:0.0:1780298642.584989:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.585021:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609391616 00000100:00100000:2.0:1780298642.585025:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609391616 00002000:00100000:2.0:1780298642.585046:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (1470464->1474559) req@ffff9cf37f746300 x1866777609391616/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.585052:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f746300 with x1866777609391616 ext(1470464->1474559) 00002000:00010000:2.0:1780298642.585055:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f746300 00000100:00100000:2.0:1780298642.585070:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6007 00000100:00100000:2.0:1780298642.585074:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609391616:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298642.585210:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609391616:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 134us (223us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298642.585218:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6007 00002000:00100000:2.0:1780298642.585224:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (1470464->1474559) req@ffff9cf37f746300 x1866777609391616/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/4536 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.585233:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f746300 with x1866777609391616 ext(1470464->1474559) 00002000:00010000:2.0:1780298642.585238:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f746300 00000100:00100000:0.0:1780298642.586054:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.586110:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609391744 00000100:00100000:3.0:1780298642.586114:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609391744 00000100:00100000:0.0:1780298642.586160:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.586166:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8940 00000100:00100000:2.0:1780298642.586172:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609391872 00000100:00100000:2.0:1780298642.586176:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609391872 00000100:00100000:3.0:1780298642.586178:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f747100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2397:x1866777609391744:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.586189:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.586200:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63033318 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d253f expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.586204:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8941 00000100:00100000:2.0:1780298642.586217:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f744700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609391872:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.586226:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.586238:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004ba675d2/0xaf35132e6303331f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2546 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.586242:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604886912:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.586275:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604887040:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.586315:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.586318:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.587061:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604886912:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.587083:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63033318 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d253f expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.587100:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63033318 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d253f expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.587111:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2000, rc 0) 00000100:00100000:2.0:1780298642.587141:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604887040:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.587160:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004ba675d2/0xaf35132e6303331f lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2546 expref: 8 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.587173:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004ba675d2/0xaf35132e6303331f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2546 expref: 8 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.587182:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156e00, rc 0) 00000100:00100000:3.0:1780298642.587198:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f747100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2397:x1866777609391744:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1024us (1144us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.587199:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f744700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609391872:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 987us (1042us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.587206:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8940 00000100:00100000:2.0:1780298642.587212:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8941 00000100:00100000:0.0:1780298642.588207:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.588243:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609392128 00000100:00100000:2.0:1780298642.588248:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609392128 00000100:00100000:2.0:1780298642.588284:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8942 00000100:00100000:2.0:1780298642.588306:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609392128:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.588315:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.588323:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63033326 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2554 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.588331:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.588348:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604887168:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.588368:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609392000 00000100:00100000:1.0:1780298642.588373:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609392000 00000100:00100000:2.0:1780298642.588385:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.588413:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8943 00000100:00100000:1.0:1780298642.588422:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2398:x1866777609392000:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.588431:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.588440:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e6303332d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d254d expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.588471:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604887296:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.588512:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34a90d3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.588993:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604887168:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.589006:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63033326 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2554 expref: 8 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.589025:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63033326 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2554 expref: 8 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.589037:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157c00, rc 0) 00000100:00100000:2.0:1780298642.589070:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609392128:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 778us (863us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.589079:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8942 00000100:00100000:1.0:1780298642.589174:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604887296:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.589202:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e6303332d lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d254d expref: 14 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.589219:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e6303332d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d254d expref: 14 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.589229:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36c00, rc 0) 00000100:00100000:1.0:1780298642.589257:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2398:x1866777609392000:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 837us (926us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.589265:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8943 00000100:00100000:0.0:1780298642.590069:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.590110:0:7411:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609392384 00000100:00100000:1.0:1780298642.590116:0:7411:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609392384 00000100:00100000:1.0:1780298642.590179:0:7411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61942 00000100:00100000:1.0:1780298642.590189:0:7411:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+107:29360:x1866777609392384:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:1.0:1780298642.590230:0:7411:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298642.590247:0:7411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156600/0xaf35132e630332cb lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 60 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24f2 expref: 107 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.590263:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298642.590267:0:7411:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298642.590427:0:7411:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37400/0xaf35132e63033334 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 59 type: IBT flags: 0x40000000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.590441:0:7411:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156800/0xaf35132e6303329a lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 59 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24c8 expref: 99 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.590458:0:7411:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37400/0xaf35132e63033334 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 59 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.590484:0:7411:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156800/0xaf35132e6303329a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 59 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24c8 expref: 99 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.590497:0:7411:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156800/0xaf35132e6303329a lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 59 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24c8 expref: 99 pid: 9976 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.590508:0:7411:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7411:1866777604887424:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298642.590550:0:7411:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3602870c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.591116:0:7411:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7411:1866777604887424:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298642.591169:0:7411:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37400/0xaf35132e63033334 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 59 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.591247:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.591282:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609392512 00000100:00100000:2.0:1780298642.591287:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609392512 00010000:00010000:2.0:1780298642.591316:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156800/0xaf35132e6303329a lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 59 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24c8 expref: 100 pid: 9976 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.591345:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15007 00000100:00100000:2.0:1780298642.591354:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+100:2399:x1866777609392512:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298642.591362:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298642.591368:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156800/0xaf35132e6303329a lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 60 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24c8 expref: 100 pid: 9976 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.591377:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156800/0xaf35132e6303329a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 60 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24c8 expref: 100 pid: 9976 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.591389:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156800/0xaf35132e6303329a lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 60 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24c8 expref: 100 pid: 9976 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.591398:0:20253:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.591403:0:20253:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37400/0xaf35132e63033334 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 59 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.591413:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37400/0xaf35132e63033334 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 59 type: IBT flags: 0x40290000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.591419:0:20253:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37400/0xaf35132e63033334 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 59 type: IBT flags: 0x40290000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.591437:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298642.591460:0:7411:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37400/0xaf35132e63033334 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 58 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.591462:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+99:2399:x1866777609392512:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 110us (215us total) trans 0 rc 0/0 00010000:00010000:1.0:1780298642.591477:0:7411:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37400/0xaf35132e63033334 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 58 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.591478:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15007 00000001:00080000:1.0:1780298642.591528:0:7411:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991020 00000004:00010000:1.0:1780298642.591550:0:7411:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000cb8e7494 reply state 000000009dcdf8cd transno 4294991021 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37400/0xaf35132e63033334 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 58 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.591562:0:7411:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37400/0xaf35132e63033334 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 58 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.591569:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298642.591571:0:7411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1780298642.591574:0:7411:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1780298642.591632:0:7411:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+109:29360:x1866777609392384:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1445us (1562us total) trans 4294991021 rc 0/0 00000100:00100000:1.0:1780298642.591640:0:7411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61942 00000020:00080000:1.0:1780298642.591644:0:7411:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1780298642.592297:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.592333:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609392640 00000100:00100000:1.0:1780298642.592337:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609392640 00000100:00100000:1.0:1780298642.592366:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8944 00000100:00100000:1.0:1780298642.592373:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:29360:x1866777609392640:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.592382:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.592391:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e6303333b lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d255b expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.592413:0:19992:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000009cb6f854/0xaf35132e63033311 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1470464->3858431] (req 1470464->1474559) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2538 expref: 14 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.592421:0:19992:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e630332bd lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3858432->37412863] (req 3858432->3870719) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24eb expref: 12 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.592429:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 2 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(109 177) 00010000:00010000:1.0:1780298642.592433:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(111 177) 00010000:00010000:1.0:1780298642.592438:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e6303333b lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d255b expref: 14 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.592457:0:19992:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e630332bd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3858432->37412863] (req 3858432->3870719) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24eb expref: 12 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.592468:0:19992:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e630332bd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3858432->37412863] (req 3858432->3870719) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24eb expref: 12 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.592479:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604887552:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298642.592529:0:19992:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000009cb6f854/0xaf35132e63033311 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1470464->3858431] (req 1470464->1474559) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2538 expref: 14 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.592540:0:19992:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000009cb6f854/0xaf35132e63033311 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1470464->3858431] (req 1470464->1474559) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2538 expref: 14 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.592550:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b41180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604887680:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298642.592579:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34a90d3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.593774:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604887552:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298642.593785:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b41180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604887680:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298642.593808:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e6303333b lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d255b expref: 14 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.593821:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36a00, rc 0) 00000100:00100000:1.0:1780298642.593849:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:29360:x1866777609392640:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1477us (1562us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.593856:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8944 00000100:00100000:0.0:1780298642.594025:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.594065:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609392768 00000100:00100000:2.0:1780298642.594071:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609392768 00002000:00100000:2.0:1780298642.594098:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (3858432->3870719) req@ffff9cf36e56e680 x1866777609392768/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 12776/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.594108:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e56e680 with x1866777609392768 ext(3858432->3870719) 00010000:00010000:2.0:1780298642.594115:0:19950:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e630332bd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3858432->37412863] (req 3858432->3870719) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24eb expref: 13 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.594162:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:2.0:1780298642.594165:0:19950:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e630332bd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3858432->37412863] (req 3858432->3870719) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24eb expref: 13 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1780298642.594174:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf36e56e680 00000100:00100000:1.0:1780298642.594195:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609392896 00000100:00100000:2.0:1780298642.594200:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 185 00000100:00100000:1.0:1780298642.594201:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609392896 00000100:00100000:2.0:1780298642.594213:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609392768:12345-192.168.203.25@tcp:4:fsx.0 00002000:00100000:1.0:1780298642.594224:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (1470464->1474559) req@ffff9cf36e56dc00 x1866777609392896/t0(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 4584/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298642.594240:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e56dc00 with x1866777609392896 ext(1470464->1474559) 00010000:00010000:1.0:1780298642.594247:0:8363:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000009cb6f854/0xaf35132e63033311 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1470464->3858431] (req 1470464->1474559) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2538 expref: 14 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.594258:0:8363:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000009cb6f854/0xaf35132e63033311 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1470464->3858431] (req 1470464->1474559) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2538 expref: 14 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1780298642.594265:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf36e56dc00 00000100:00100000:1.0:1780298642.594302:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 186 00000100:00100000:1.0:1780298642.594312:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2398:x1866777609392896:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:1.0:1780298642.594727:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2398:x1866777609392896:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 418us (566us total) trans 4294969630 rc 0/0 00000100:00100000:1.0:1780298642.594733:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 186 00002000:00100000:1.0:1780298642.594738:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (1470464->1474559) req@ffff9cf36e56dc00 x1866777609392896/t4294969630(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 4584/448 e 0 to 0 dl 1780298653 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298642.594744:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e56dc00 with x1866777609392896 ext(1470464->1474559) 00010000:00010000:1.0:1780298642.594751:0:8363:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000009cb6f854/0xaf35132e63033311 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1470464->3858431] (req 1470464->1474559) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2538 expref: 16 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.594752:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2400:x1866777609392768:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 540us (722us total) trans 4294969629 rc 0/0 00010000:00010000:1.0:1780298642.594768:0:8363:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000009cb6f854/0xaf35132e63033311 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1470464->3858431] (req 1470464->1474559) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2538 expref: 16 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.594771:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 185 00002000:00010000:1.0:1780298642.594778:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf36e56dc00 00002000:00100000:2.0:1780298642.594783:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (3858432->3870719) req@ffff9cf36e56e680 x1866777609392768/t4294969629(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 12776/448 e 0 to 0 dl 1780298653 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.594792:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e56e680 with x1866777609392768 ext(3858432->3870719) 00010000:00010000:2.0:1780298642.594798:0:19950:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e630332bd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3858432->37412863] (req 3858432->3870719) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24eb expref: 15 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.594809:0:19950:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e630332bd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3858432->37412863] (req 3858432->3870719) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24eb expref: 15 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1780298642.594816:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf36e56e680 00000100:00100000:0.0:1780298642.595708:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.595728:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.595750:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609393024 00000100:00100000:1.0:1780298642.595753:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609393152 00000100:00100000:2.0:1780298642.595755:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609393024 00000100:00100000:1.0:1780298642.595755:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609393152 00010000:00010000:2.0:1780298642.595773:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000009cb6f854/0xaf35132e63033311 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1470464->3858431] (req 1470464->1474559) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2538 expref: 16 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.595773:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e630332bd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3858432->37412863] (req 3858432->3870719) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24eb expref: 15 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.595800:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15008 00000100:00100000:1.0:1780298642.595808:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2400:x1866777609393152:12345-192.168.203.25@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1780298642.595812:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15009 00000100:00100000:2.0:1780298642.595819:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2398:x1866777609393024:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298642.595820:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298642.595827:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:1.0:1780298642.595830:0:8374:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 432 -> 464 00010000:00010000:2.0:1780298642.595843:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000009cb6f854/0xaf35132e63033311 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [1470464->3858431] (req 1470464->1474559) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2538 expref: 16 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.595843:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e630332bd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [3858432->37412863] (req 3858432->3870719) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24eb expref: 15 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.595854:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e630332bd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [3858432->37412863] (req 3858432->3870719) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24eb expref: 15 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.595865:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000009cb6f854/0xaf35132e63033311 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [1470464->3858431] (req 1470464->1474559) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2538 expref: 16 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.595881:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e630332bd lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [3858432->37412863] (req 3858432->3870719) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d24eb expref: 15 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.595885:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009cb6f854/0xaf35132e63033311 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [1470464->3858431] (req 1470464->1474559) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2538 expref: 16 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.595893:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(111 177) 00010000:00010000:1.0:1780298642.595897:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(111 177) 00010000:00010000:1.0:1780298642.595902:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e6303333b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d255b expref: 15 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.595911:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e6303333b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d255b expref: 15 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.595924:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298642.595940:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e6303333b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d255b expref: 15 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.595949:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2398:x1866777609393024:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 132us (242us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.595952:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b41180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777604887808:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298642.595956:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15009 00000100:00100000:1.0:1780298642.595997:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34a90d3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.596692:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b41180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777604887808:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298642.596715:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298642.596740:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609393152:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 934us (1010us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.596746:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15008 00000100:00100000:0.0:1780298642.596781:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.596813:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609393280 00000100:00100000:2.0:1780298642.596817:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609393280 00000100:00100000:2.0:1780298642.596849:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8945 00000100:00100000:2.0:1780298642.596858:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609393280:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.596869:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.596882:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63033342 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2562 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.596907:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000ced6368f/0xaf35132e630332ee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [290816->33845247] (req 290816->294911) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2515 expref: 6 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.596915:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(83 130) 00010000:00010000:2.0:1780298642.596918:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(84 130) 00010000:00010000:2.0:1780298642.596922:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63033342 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2562 expref: 10 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.596942:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000ced6368f/0xaf35132e630332ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [290816->33845247] (req 290816->294911) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2515 expref: 6 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.596955:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000ced6368f/0xaf35132e630332ee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [290816->33845247] (req 290816->294911) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2515 expref: 6 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.596965:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604887936:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298642.597007:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.597581:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604887936:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298642.597599:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63033342 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2562 expref: 10 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.597610:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157c00, rc 0) 00000100:00100000:2.0:1780298642.597638:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609393280:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 782us (856us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.597649:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8945 00000100:00100000:3.0:1780298642.597761:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.597792:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609393408 00000100:00100000:0.0:1780298642.597796:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609393408 00010000:00010000:0.0:1780298642.597812:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000ced6368f/0xaf35132e630332ee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [290816->33845247] (req 290816->294911) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2515 expref: 7 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.597833:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15010 00000100:00100000:0.0:1780298642.597840:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609393408:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298642.597848:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298642.597862:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000ced6368f/0xaf35132e630332ee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [290816->33845247] (req 290816->294911) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2515 expref: 7 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.597874:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000ced6368f/0xaf35132e630332ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [290816->33845247] (req 290816->294911) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2515 expref: 7 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.597889:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ced6368f/0xaf35132e630332ee lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [290816->33845247] (req 290816->294911) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2515 expref: 7 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.597901:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(84 130) 00010000:00010000:0.0:1780298642.597904:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(84 130) 00010000:00010000:0.0:1780298642.597910:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63033342 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2562 expref: 9 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.597918:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63033342 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2562 expref: 9 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.597940:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63033342 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2562 expref: 9 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.597951:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e56df80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777604888064:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298642.597988:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.598762:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e56df80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777604888064:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298642.598790:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.598829:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609393536 00000100:00100000:2.0:1780298642.598834:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609393536 00000100:00100000:0.0:1780298642.598850:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00002000:00010000:2.0:1780298642.598856:0:8362:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (1115924->18446744073709551615) 00002000:00100000:2.0:1780298642.598862:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf343965880 with x1866777609393536 ext(1115924->18446744073709551615) 00002000:00010000:2.0:1780298642.598869:0:8362:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf343965880. 00010000:00010000:3.0:1780298642.598883:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298642.598893:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609393664 00000100:00100000:2.0:1780298642.598898:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609393664 00000100:00100000:1.0:1780298642.598902:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6008 00000100:00100000:1.0:1780298642.598913:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343965880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2398:x1866777609393536:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:2.0:1780298642.598915:0:8362:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00000100:00100000:3.0:1780298642.598917:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609393408:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1075us (1155us total) trans 0 rc 0/0 00002000:00100000:2.0:1780298642.598923:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf343964380 with x1866777609393664 ext(0->18446744073709551615) 00002000:00010000:2.0:1780298642.598931:0:8362:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf343964380. 00000100:00100000:3.0:1780298642.598935:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15010 00000100:00100000:2.0:1780298642.598950:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6009 00000100:00100000:2.0:1780298642.598960:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343964380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609393664:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:2.0:1780298642.599305:0:8362:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 1502074 -> 0 00002000:00010000:1.0:1780298642.599307:0:19949:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 1115924 00002000:00010000:2.0:1780298642.599309:0:8362:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298642.599311:0:19949:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:2.0:1780298642.599315:0:8362:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298642.599315:0:19949:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298642.599319:0:19949:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:2.0:1780298642.599321:0:8362:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298642.599322:0:19949:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 464 -> 392 00002000:00010000:2.0:1780298642.599327:0:8362:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 16 -> 0 00000100:00100000:1.0:1780298642.599350:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343965880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2398:x1866777609393536:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 441us (560us total) trans 4294969631 rc 0/0 00000100:00100000:1.0:1780298642.599357:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6008 00000100:00100000:2.0:1780298642.599358:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343964380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609393664:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 398us (504us total) trans 4294969197 rc 0/0 00002000:00010000:1.0:1780298642.599360:0:19949:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (1115924->18446744073709551615) 00002000:00100000:1.0:1780298642.599365:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf343965880 with x1866777609393536 ext(1115924->18446744073709551615) 00002000:00010000:1.0:1780298642.599370:0:19949:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf343965880. 00000100:00100000:2.0:1780298642.599374:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6009 00002000:00010000:2.0:1780298642.599379:0:8362:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00002000:00100000:2.0:1780298642.599382:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf343964380 with x1866777609393664 ext(0->18446744073709551615) 00002000:00010000:2.0:1780298642.599388:0:8362:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf343964380. 00000100:00100000:0.0:1780298642.600118:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.600173:0:6476:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609393792 00000100:00100000:2.0:1780298642.600178:0:6476:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609393792 00000100:00100000:2.0:1780298642.600205:0:6476:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61943 00000100:00100000:2.0:1780298642.600210:0:6476:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343966680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+99:29360:x1866777609393792:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.600219:0:6476:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.600230:0:6476:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156800/0xaf35132e63033349 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 59 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2569 expref: -99 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.600287:0:6476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156e00/0xaf35132e63033350 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 60 type: IBT flags: 0x40000000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.600311:0:6476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156e00/0xaf35132e63033350 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 60 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.600318:0:6476:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156e00/0xaf35132e63033350 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 60 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298642.600336:0:6476:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 1115924 00000004:00200000:2.0:1780298642.600340:0:6476:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298642.600351:0:6476:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156e00/0xaf35132e63033350 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 60 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.600366:0:6476:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156800/0xaf35132e63033349 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 60 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2569 expref: 101 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.600376:0:6476:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156e00/0xaf35132e63033350 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 59 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2569 expref: 100 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.600381:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.600382:0:6476:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298642.600384:0:6476:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156e00, rc 0) 00000100:00100000:2.0:1780298642.600405:0:6476:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343966680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+100:29360:x1866777609393792:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 196us (288us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.600410:0:6476:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61943 00000100:00100000:0.0:1780298642.601155:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.601282:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.601284:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609393920 00000100:00100000:3.0:1780298642.601301:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609393920 00000100:00100000:2.0:1780298642.601321:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609394048 00000100:00100000:2.0:1780298642.601326:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609394048 00000100:00100000:2.0:1780298642.601363:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8946 00000100:00100000:3.0:1780298642.601372:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8947 00000100:00100000:2.0:1780298642.601377:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343967100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609394048:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298642.601387:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343964a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2398:x1866777609393920:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.601387:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.601398:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.601398:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eae09c9b/0xaf35132e63033357 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2577 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.601408:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e6303335e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2570 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.601441:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604888192:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.601447:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481ffb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604888320:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.601502:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.601502:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.602155:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604888192:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.602179:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000eae09c9b/0xaf35132e63033357 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2577 expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.602197:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eae09c9b/0xaf35132e63033357 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2577 expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.602207:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157a00, rc 0) 00000100:00100000:3.0:1780298642.602252:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481ffb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604888320:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.602255:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343967100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609394048:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 884us (972us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.602263:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8946 00010000:00010000:3.0:1780298642.602277:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e6303335e lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2570 expref: 15 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.602304:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e6303335e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2570 expref: 15 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.602320:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3200, rc 0) 00000100:00100000:3.0:1780298642.602429:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343964a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2398:x1866777609393920:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 965us (1191us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.602437:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8947 00000100:00100000:0.0:1780298642.603098:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.603140:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609394176 00000100:00100000:1.0:1780298642.603145:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609394176 00000100:00100000:1.0:1780298642.603180:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8948 00000100:00100000:0.0:1780298642.603185:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.603195:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343964000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2397:x1866777609394176:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.603204:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.603213:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e63033365 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d257e expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.603227:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609394304 00000100:00100000:3.0:1780298642.603233:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609394304 00000100:00100000:1.0:1780298642.603238:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b41180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604888448:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.603272:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8949 00000100:00100000:0.0:1780298642.603282:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343965c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609394304:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.603300:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298642.603302:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34a90d3c0 going to sleep for 11 seconds 00010000:00010000:0.0:1780298642.603309:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e6303336c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2585 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.603334:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343967480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604888576:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.603361:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.603851:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b41180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604888448:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.603864:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e63033365 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d257e expref: 15 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.603874:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e63033365 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d257e expref: 15 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.603880:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36c00, rc 0) 00000100:00100000:1.0:1780298642.603904:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343964000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2397:x1866777609394176:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 716us (807us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.603908:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8948 00000100:00100000:0.0:1780298642.604104:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343967480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604888576:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.604120:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e6303336c lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2585 expref: 7 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.604158:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e6303336c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2585 expref: 7 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.604170:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3800, rc 0) 00000100:00100000:0.0:1780298642.604198:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343965c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609394304:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 919us (1015us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.604206:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8949 00000100:00100000:0.0:1780298642.605606:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.605653:0:9976:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609394432 00000100:00100000:2.0:1780298642.605659:0:9976:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609394432 00000100:00100000:2.0:1780298642.605697:0:9976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61944 00000100:00100000:2.0:1780298642.605705:0:9976:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343967480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+109:29360:x1866777609394432:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.605721:0:9976:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.605735:0:9976:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156000/0xaf35132e63033373 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 60 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d258c expref: -99 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.605810:0:9976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156400/0xaf35132e6303337a lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 61 type: IBT flags: 0x40000000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.605825:0:9976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156400/0xaf35132e6303337a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 61 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.605837:0:9976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156400/0xaf35132e6303337a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 61 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298642.605860:0:9976:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 1115924 00000004:00200000:2.0:1780298642.605864:0:9976:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298642.605880:0:9976:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156400/0xaf35132e6303337a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 61 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.605900:0:9976:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156000/0xaf35132e63033373 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 61 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d258c expref: 111 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.605917:0:9976:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156400/0xaf35132e6303337a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 60 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d258c expref: 110 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.605925:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.605927:0:9976:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298642.605930:0:9976:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156400, rc 0) 00000100:00100000:2.0:1780298642.605960:0:9976:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343967480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+110:29360:x1866777609394432:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 257us (357us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.605965:0:9976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61944 00000100:00100000:2.0:1780298642.607209:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.607255:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609394560 00000100:00100000:0.0:1780298642.607260:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609394560 00000100:00100000:0.0:1780298642.607309:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6010 00000100:00100000:0.0:1780298642.607318:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609394560:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298642.607474:0:16972:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 0 -> 148373 00002000:00010000:0.0:1780298642.607478:0:16972:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:0.0:1780298642.607480:0:16972:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:0.0:1780298642.607483:0:16972:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:0.0:1780298642.607485:0:16972:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 0 -> 8 00000100:00100000:0.0:1780298642.607518:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609394560:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 203us (311us total) trans 4294969198 rc 0/0 00000100:00100000:0.0:1780298642.607525:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6010 00000100:00100000:3.0:1780298642.609546:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.609585:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609394688 00000100:00100000:0.0:1780298642.609592:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609394688 00000100:00100000:0.0:1780298642.609627:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8950 00000100:00100000:0.0:1780298642.609635:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481ffb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:29360:x1866777609394688:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.609646:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.609661:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e63033381 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2593 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.609683:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e6303333b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d255b expref: 15 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.609696:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(111 177) 00010000:00010000:0.0:1780298642.609701:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(112 177) 00010000:00010000:0.0:1780298642.609706:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e63033381 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [4325376->4337663] (req 4325376->4337663) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2593 expref: 15 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.609735:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e6303333b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d255b expref: 15 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.609745:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e6303333b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d255b expref: 15 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.609755:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343966a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604888704:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298642.609803:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417a80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.610397:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343966a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604888704:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298642.610419:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e63033381 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [4325376->4337663] (req 4325376->4337663) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2593 expref: 15 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.610433:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2a00, rc 0) 00000100:00100000:0.0:1780298642.610464:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481ffb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:29360:x1866777609394688:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 830us (919us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.610470:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8950 00000100:00100000:3.0:1780298642.610700:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.610738:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609394816 00000100:00100000:2.0:1780298642.610742:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609394816 00010000:00010000:2.0:1780298642.610758:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e6303333b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d255b expref: 16 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.610779:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15011 00000100:00100000:2.0:1780298642.610784:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481ff480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2399:x1866777609394816:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298642.610791:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298642.610802:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e6303333b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d255b expref: 16 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.610811:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e6303333b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d255b expref: 16 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.610824:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e6303333b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d255b expref: 16 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.610832:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(112 177) 00010000:00010000:2.0:1780298642.610834:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(112 177) 00010000:00010000:2.0:1780298642.610840:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e63033381 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [4325376->37879807] (req 4325376->4337663) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2593 expref: 14 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.610850:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e63033381 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [4325376->37879807] (req 4325376->4337663) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2593 expref: 14 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.610866:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e63033381 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [4325376->37879807] (req 4325376->4337663) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2593 expref: 14 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.610872:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244adaa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777604888832:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298642.610902:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.611595:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244adaa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777604888832:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298642.611606:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298642.611626:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481ff480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2399:x1866777609394816:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 842us (928us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298642.611631:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15011 00000100:00100000:0.0:1780298642.612200:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.612276:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609394944 00000100:00100000:3.0:1780298642.612282:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609394944 00000100:00100000:2.0:1780298642.612311:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.612365:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609395072 00000100:00100000:3.0:1780298642.612370:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609395072 00000100:00100000:0.0:1780298642.612374:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8951 00000100:00100000:0.0:1780298642.612387:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343966a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2397:x1866777609394944:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.612396:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298642.612403:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8952 00010000:00010000:0.0:1780298642.612406:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a269a887/0xaf35132e63033388 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d259a expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.612416:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609395072:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.612426:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.612436:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000039c5f58c/0xaf35132e6303338f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d25a1 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.612443:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343965f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604888960:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.612477:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604889088:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.612496:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417a80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.612509:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00002000:00010000:2.0:1780298642.613183:0:12101:0:(ofd_lvb.c:259:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size: 1115924 -> 4334242 00000100:00100000:2.0:1780298642.613196:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343965f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604888960:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.613210:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a269a887/0xaf35132e63033388 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d259a expref: 16 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.613221:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a269a887/0xaf35132e63033388 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d259a expref: 16 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.613228:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3e00, rc 0) 00000100:00100000:2.0:1780298642.613254:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343966a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2397:x1866777609394944:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 873us (1056us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.613259:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8951 00000100:00100000:0.0:1780298642.613282:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604889088:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.613319:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000039c5f58c/0xaf35132e6303338f lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d25a1 expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.613333:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000039c5f58c/0xaf35132e6303338f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d25a1 expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.613342:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3200, rc 0) 00000100:00100000:0.0:1780298642.613373:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609395072:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 962us (1061us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.613380:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8952 00000100:00100000:1.0:1780298642.614167:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.614191:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609395200 00000100:00100000:0.0:1780298642.614196:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609395200 00000100:00100000:1.0:1780298642.614204:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.614228:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609395328 00000100:00100000:0.0:1780298642.614231:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609395328 00000100:00100000:1.0:1780298642.614242:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8953 00000100:00100000:1.0:1780298642.614251:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b41180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2400:x1866777609395200:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298642.614254:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8954 00010000:00010000:1.0:1780298642.614265:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298642.614265:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b43800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609395328:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.614273:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e63033396 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d25a8 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.614273:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.614281:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d4afe0b/0xaf35132e6303339d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d25af expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.614316:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b41c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604889216:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.614321:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343966d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604889344:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.614353:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34a90d3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.614356:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417300 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.615051:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343966d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604889344:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.615069:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001d4afe0b/0xaf35132e6303339d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d25af expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.615079:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b41c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604889216:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.615085:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d4afe0b/0xaf35132e6303339d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d25af expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.615095:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3000, rc 0) 00010000:00010000:1.0:1780298642.615106:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e63033396 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d25a8 expref: 16 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.615117:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e63033396 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d25a8 expref: 16 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.615143:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37200, rc 0) 00000100:00100000:0.0:1780298642.615150:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609395328:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 886us (945us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.615158:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8954 00000100:00100000:1.0:1780298642.615170:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b41180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2400:x1866777609395200:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 920us (1003us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.615177:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8953 00000100:00100000:3.0:1780298642.616063:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.616088:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.616107:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609395456 00000100:00100000:1.0:1780298642.616112:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609395456 00000100:00100000:2.0:1780298642.616152:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609395584 00000100:00100000:2.0:1780298642.616157:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609395584 00000100:00100000:1.0:1780298642.616161:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8955 00000100:00100000:1.0:1780298642.616171:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2399:x1866777609395456:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.616178:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298642.616187:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8956 00010000:00010000:1.0:1780298642.616187:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e630333a4 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d25b6 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.616200:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609395584:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.616207:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.616215:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042c16ab0/0xaf35132e630333ab lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d25bd expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.616221:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b41c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604889472:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.616242:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604889600:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.616262:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34a90d3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.616268:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.617021:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b41c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604889472:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.617037:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e630333a4 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d25b6 expref: 16 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.617045:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604889600:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.617052:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e630333a4 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d25b6 expref: 16 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.617059:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37000, rc 0) 00010000:00010000:0.0:1780298642.617073:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000042c16ab0/0xaf35132e630333ab lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d25bd expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.617084:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2399:x1866777609395456:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 916us (1022us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.617091:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8955 00010000:00010000:0.0:1780298642.617091:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042c16ab0/0xaf35132e630333ab lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d25bd expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.617104:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156800, rc 0) 00000100:00100000:0.0:1780298642.617150:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609395584:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 953us (1060us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.617159:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8956 00000100:00100000:0.0:1780298642.618101:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.618139:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609395712 00000100:00100000:3.0:1780298642.618143:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609395712 00000100:00100000:0.0:1780298642.618194:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8957 00000100:00100000:0.0:1780298642.618204:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343966d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:29360:x1866777609395712:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.618212:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.618220:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e630333b2 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d25c4 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.618236:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(112 177) 00010000:00010000:0.0:1780298642.618240:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(112 177) 00010000:00010000:0.0:1780298642.618249:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e630333b2 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1732608->4325375] (req 1732608->1736703) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d25c4 expref: 16 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.618263:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e630333b2 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1732608->4325375] (req 1732608->1736703) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d25c4 expref: 16 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.618272:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3c00, rc 0) 00000100:00100000:0.0:1780298642.618311:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343966d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:29360:x1866777609395712:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 110us (210us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.618317:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8957 00000100:00100000:3.0:1780298642.619033:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.619071:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609395840 00000100:00100000:1.0:1780298642.619076:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609395840 00002000:00100000:1.0:1780298642.619104:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (1732608->1736703) req@ffff9cf3481fc700 x1866777609395840/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298642.619115:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fc700 with x1866777609395840 ext(1732608->1736703) 00002000:00010000:1.0:1780298642.619120:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481fc700 00000100:00100000:1.0:1780298642.619176:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6011 00000100:00100000:1.0:1780298642.619187:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2398:x1866777609395840:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298642.619331:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2398:x1866777609395840:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 147us (297us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.619339:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6011 00002000:00100000:1.0:1780298642.619345:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (1732608->1736703) req@ffff9cf3481fc700 x1866777609395840/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/440 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298642.619353:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fc700 with x1866777609395840 ext(1732608->1736703) 00002000:00010000:1.0:1780298642.619356:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481fc700 00000100:00100000:2.0:1780298642.620214:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.620259:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609395968 00000100:00100000:0.0:1780298642.620264:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609395968 00002000:00100000:0.0:1780298642.620285:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (1736704->1740799) req@ffff9cf244adad80 x1866777609395968/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.620307:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244adad80 with x1866777609395968 ext(1736704->1740799) 00002000:00010000:0.0:1780298642.620311:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf244adad80 00000100:00100000:0.0:1780298642.620332:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6012 00000100:00100000:0.0:1780298642.620340:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2397:x1866777609395968:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298642.620421:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2397:x1866777609395968:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 82us (205us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.620428:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6012 00002000:00100000:0.0:1780298642.620435:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (1736704->1740799) req@ffff9cf244adad80 x1866777609395968/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/440 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.620443:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244adad80 with x1866777609395968 ext(1736704->1740799) 00002000:00010000:0.0:1780298642.620447:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf244adad80 00000100:00100000:0.0:1780298642.621241:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.621305:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609396096 00000100:00100000:1.0:1780298642.621309:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609396096 00000100:00100000:1.0:1780298642.621334:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8958 00000100:00100000:1.0:1780298642.621338:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343966300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2399:x1866777609396096:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.621346:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.621353:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e630333b9 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d25cb expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.621379:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b41c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604889728:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.621415:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34a90d3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.622040:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b41c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604889728:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.622067:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e630333b9 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d25cb expref: 16 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.622080:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e630333b9 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d25cb expref: 16 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.622087:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36a00, rc 0) 00000100:00100000:1.0:1780298642.622110:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343966300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2399:x1866777609396096:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 772us (871us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.622115:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8958 00000100:00100000:2.0:1780298642.622986:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.623023:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609396224 00000100:00100000:3.0:1780298642.623028:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609396224 00000100:00100000:3.0:1780298642.623059:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8959 00000100:00100000:3.0:1780298642.623067:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2397:x1866777609396224:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.623076:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.623084:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e630333c0 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d25d2 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.623116:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604889856:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.623185:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.623753:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604889856:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.623767:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e630333c0 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d25d2 expref: 16 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.623776:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e630333c0 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d25d2 expref: 16 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.623782:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2000, rc 0) 00000100:00100000:3.0:1780298642.623806:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2397:x1866777609396224:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 740us (822us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.623812:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8959 00000100:00100000:0.0:1780298642.624844:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.624887:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609396352 00000100:00100000:2.0:1780298642.624891:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609396352 00000100:00100000:2.0:1780298642.624918:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8960 00000100:00100000:2.0:1780298642.624923:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343964e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2399:x1866777609396352:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.624929:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.624936:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009c278703/0xaf35132e630333c7 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d25d9 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.624960:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244adb800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604889984:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.624992:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4000 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.625676:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244adb800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604889984:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.625697:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009c278703/0xaf35132e630333c7 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d25d9 expref: 16 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.625710:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009c278703/0xaf35132e630333c7 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d25d9 expref: 16 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.625717:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156600, rc 0) 00000100:00100000:0.0:1780298642.625743:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343964e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2399:x1866777609396352:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 820us (899us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.625750:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8960 00000100:00100000:0.0:1780298642.626564:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.626597:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609396480 00000100:00100000:2.0:1780298642.626600:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609396480 00000100:00100000:2.0:1780298642.626622:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8961 00000100:00100000:2.0:1780298642.626626:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e582050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2397:x1866777609396480:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.626633:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.626639:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e630333ce lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d25e0 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.626661:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ada300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604890112:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.626691:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4900 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.627307:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ada300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604890112:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.627319:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e630333ce lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d25e0 expref: 16 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.627329:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e630333ce lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d25e0 expref: 16 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.627335:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157400, rc 0) 00000100:00100000:2.0:1780298642.627364:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e582050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2397:x1866777609396480:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 736us (800us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.627371:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8961 00000100:00100000:1.0:1780298642.628194:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.628221:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609396608 00000100:00100000:2.0:1780298642.628225:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609396608 00000100:00100000:2.0:1780298642.628245:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8962 00000100:00100000:2.0:1780298642.628250:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b41c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2399:x1866777609396608:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.628258:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.628266:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e630333d5 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d25e7 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.628299:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604890240:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.628328:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4900 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.628925:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604890240:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.628936:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e630333d5 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d25e7 expref: 16 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.628945:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e630333d5 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d25e7 expref: 16 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.628951:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157000, rc 0) 00000100:00100000:2.0:1780298642.628982:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b41c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2399:x1866777609396608:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 732us (791us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.628986:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8962 00000100:00100000:3.0:1780298642.629795:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.629829:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609396736 00000100:00100000:2.0:1780298642.629833:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609396736 00000100:00100000:2.0:1780298642.629854:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8963 00000100:00100000:2.0:1780298642.629858:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:29360:x1866777609396736:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.629864:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.629870:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e630333dc lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d25ee expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.629884:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e630333b2 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1732608->4325375] (req 1732608->1736703) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d25c4 expref: 15 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.629890:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(112 177) 00010000:00010000:2.0:1780298642.629893:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(113 177) 00010000:00010000:2.0:1780298642.629896:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e630333dc lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1957888->1961983] (req 1957888->1961983) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d25ee expref: 16 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.629913:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e630333b2 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1732608->4325375] (req 1732608->1736703) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d25c4 expref: 15 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.629923:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e630333b2 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1732608->4325375] (req 1732608->1736703) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d25c4 expref: 15 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.629931:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604890368:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298642.629957:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4900 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.630584:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604890368:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298642.630596:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e630333dc lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1957888->1961983] (req 1957888->1961983) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d25ee expref: 16 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.630604:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156a00, rc 0) 00000100:00100000:2.0:1780298642.630625:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:29360:x1866777609396736:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 767us (832us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.630630:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8963 00000100:00100000:3.0:1780298642.630819:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.630851:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609396864 00000100:00100000:2.0:1780298642.630854:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609396864 00010000:00010000:2.0:1780298642.630868:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e630333b2 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1732608->4325375] (req 1732608->1736703) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d25c4 expref: 16 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.630890:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15012 00000100:00100000:2.0:1780298642.630894:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609396864:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298642.630903:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298642.630911:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e630333b2 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1732608->4325375] (req 1732608->1736703) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d25c4 expref: 16 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.630918:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e630333b2 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1732608->4325375] (req 1732608->1736703) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d25c4 expref: 16 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.630932:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e630333b2 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1732608->4325375] (req 1732608->1736703) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d25c4 expref: 16 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.630939:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(113 177) 00010000:00010000:2.0:1780298642.630941:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(113 177) 00010000:00010000:2.0:1780298642.630947:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e630333dc lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1957888->4325375] (req 1957888->1961983) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d25ee expref: 15 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.630952:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e630333dc lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1957888->4325375] (req 1957888->1961983) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d25ee expref: 15 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.630966:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e630333dc lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1957888->4325375] (req 1957888->1961983) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d25ee expref: 15 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.630972:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad8000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777604890496:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298642.630998:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4900 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.631647:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad8000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777604890496:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298642.631655:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298642.631680:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2400:x1866777609396864:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 785us (863us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.631687:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15012 00000100:00100000:3.0:1780298642.631835:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.631868:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609396992 00000100:00100000:0.0:1780298642.631873:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609396992 00000100:00100000:0.0:1780298642.631910:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6013 00000100:00100000:0.0:1780298642.631918:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609396992:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298642.632068:0:8364:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4334242 -> 1115924 00002000:00010000:0.0:1780298642.632071:0:8364:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:0.0:1780298642.632072:0:8364:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:0.0:1780298642.632074:0:8364:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:0.0:1780298642.632075:0:8364:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 392 -> 400 00000100:00100000:0.0:1780298642.632099:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609396992:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 183us (265us total) trans 4294969632 rc 0/0 00000100:00100000:0.0:1780298642.632103:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6013 00000100:00100000:1.0:1780298642.632996:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.633036:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609397120 00000100:00100000:1.0:1780298642.633041:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.633042:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609397120 00000100:00100000:3.0:1780298642.633075:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609397248 00000100:00100000:0.0:1780298642.633081:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8964 00000100:00100000:0.0:1780298642.633089:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2399:x1866777609397120:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.633097:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.633106:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bde78cfa/0xaf35132e630333e3 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d25f5 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.633118:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609397248 00000100:00100000:0.0:1780298642.633164:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343967b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604890624:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.633197:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8965 00000100:00100000:1.0:1780298642.633211:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609397248:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298642.633212:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417300 going to sleep for 11 seconds 00010000:00010000:1.0:1780298642.633222:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.633231:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009cb6f854/0xaf35132e630333ea lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d25fc expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.633257:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604890752:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.633286:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34a90d3c0 going to sleep for 11 seconds 00002000:00010000:0.0:1780298642.633876:0:12101:0:(ofd_lvb.c:259:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size: 1115924 -> 4334242 00000100:00100000:0.0:1780298642.633887:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343967b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604890624:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.633907:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bde78cfa/0xaf35132e630333e3 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d25f5 expref: 16 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.633931:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bde78cfa/0xaf35132e630333e3 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d25f5 expref: 16 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.633943:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3a00, rc 0) 00000100:00100000:0.0:1780298642.633982:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2399:x1866777609397120:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 893us (987us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.633989:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8964 00000100:00100000:1.0:1780298642.633999:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604890752:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.634019:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009cb6f854/0xaf35132e630333ea lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d25fc expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.634032:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009cb6f854/0xaf35132e630333ea lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d25fc expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.634041:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36e00, rc 0) 00000100:00100000:1.0:1780298642.634071:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609397248:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 866us (1028us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.634078:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8965 00000100:00100000:2.0:1780298642.634893:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.634926:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609397376 00000100:00100000:1.0:1780298642.634930:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609397376 00000100:00100000:1.0:1780298642.634955:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8966 00000100:00100000:1.0:1780298642.634963:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2397:x1866777609397376:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.634971:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.634981:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e630333f1 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2603 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.635009:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.635013:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604890880:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.635063:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34a90d3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.635064:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609397504 00000100:00100000:3.0:1780298642.635070:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609397504 00000100:00100000:3.0:1780298642.635100:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8967 00000100:00100000:3.0:1780298642.635109:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609397504:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.635119:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.635153:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e630333f8 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d260a expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.635184:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fe680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604891008:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.635261:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.635649:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604890880:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.635676:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e630333f1 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2603 expref: 16 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.635688:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e630333f1 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2603 expref: 16 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.635700:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37200, rc 0) 00000100:00100000:1.0:1780298642.635725:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2397:x1866777609397376:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 765us (834us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.635730:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8966 00000100:00100000:3.0:1780298642.635938:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fe680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604891008:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.635955:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e630333f8 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d260a expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.635970:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e630333f8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d260a expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.635980:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2000, rc 0) 00000100:00100000:3.0:1780298642.636009:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609397504:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 902us (1001us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.636016:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8967 00000100:00100000:2.0:1780298642.636826:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.636862:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609397632 00000100:00100000:0.0:1780298642.636867:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609397632 00000100:00100000:0.0:1780298642.636897:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8968 00000100:00100000:0.0:1780298642.636906:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:29360:x1866777609397632:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.636914:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.636923:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e2cfab1/0xaf35132e630333ff lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2611 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.636940:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63033342 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2562 expref: 11 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.636948:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(84 130) 00010000:00010000:0.0:1780298642.636952:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(85 130) 00010000:00010000:0.0:1780298642.636957:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003e2cfab1/0xaf35132e630333ff lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2834432->2842623] (req 2834432->2842623) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2611 expref: 7 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.636977:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63033342 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2562 expref: 11 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.636992:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63033342 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2562 expref: 11 pid: 12101 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.637003:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343967b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604891136:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298642.637046:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417300 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.637656:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343967b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604891136:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298642.637671:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003e2cfab1/0xaf35132e630333ff lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2834432->2842623] (req 2834432->2842623) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2611 expref: 7 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.637682:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2e00, rc 0) 00000100:00100000:0.0:1780298642.637711:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609397632:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 807us (884us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.637717:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8968 00000100:00100000:2.0:1780298642.637825:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.637865:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609397760 00000100:00100000:0.0:1780298642.637870:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609397760 00010000:00010000:0.0:1780298642.637887:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63033342 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2562 expref: 12 pid: 12101 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.637913:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15013 00000100:00100000:0.0:1780298642.637919:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b03050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609397760:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298642.637929:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298642.637940:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63033342 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2562 expref: 12 pid: 12101 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.637956:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63033342 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2562 expref: 12 pid: 12101 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.637972:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63033342 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2562 expref: 12 pid: 12101 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.637982:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(85 130) 00010000:00010000:0.0:1780298642.637986:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(85 130) 00010000:00010000:0.0:1780298642.637993:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000003e2cfab1/0xaf35132e630333ff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2834432->36388863] (req 2834432->2842623) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2611 expref: 6 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.638001:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003e2cfab1/0xaf35132e630333ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2834432->36388863] (req 2834432->2842623) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2611 expref: 6 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.638021:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000003e2cfab1/0xaf35132e630333ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2834432->36388863] (req 2834432->2842623) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2611 expref: 6 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.638030:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343967b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:20253:1866777604891264:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298642.638066:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417300 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.638733:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343967b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:20253:1866777604891264:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298642.638744:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298642.638769:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b03050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609397760:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 850us (944us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.638777:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15013 00000100:00100000:2.0:1780298642.638854:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.638887:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609397888 00000100:00100000:0.0:1780298642.638891:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609397888 00000100:00100000:0.0:1780298642.638925:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6014 00000100:00100000:0.0:1780298642.638933:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609397888:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298642.639040:0:8362:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 148373 -> 148373 00002000:00010000:0.0:1780298642.639044:0:8362:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:0.0:1780298642.639047:0:8362:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:0.0:1780298642.639049:0:8362:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:0.0:1780298642.639051:0:8362:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 8 -> 8 00000100:00100000:0.0:1780298642.639078:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609397888:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 146us (223us total) trans 4294969199 rc 0/0 00000100:00100000:0.0:1780298642.639085:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6014 00000100:00100000:2.0:1780298642.639837:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.639882:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609398016 00000100:00100000:1.0:1780298642.639887:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609398016 00000100:00100000:1.0:1780298642.639923:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8969 00000100:00100000:1.0:1780298642.639933:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ada300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2399:x1866777609398016:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.639942:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298642.639949:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:1.0:1780298642.639954:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009cb6f854/0xaf35132e63033406 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2618 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.639973:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609398144 00000100:00100000:0.0:1780298642.639978:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609398144 00000100:00100000:1.0:1780298642.639989:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604891392:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.640012:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8970 00000100:00100000:0.0:1780298642.640023:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609398144:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.640032:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298642.640035:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34a90d3c0 going to sleep for 11 seconds 00010000:00010000:0.0:1780298642.640041:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000658b0d9c/0xaf35132e6303340d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d261f expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.640071:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343967b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604891520:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.640102:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417e40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.640648:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604891392:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.640666:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009cb6f854/0xaf35132e63033406 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2618 expref: 16 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.640682:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009cb6f854/0xaf35132e63033406 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2618 expref: 16 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.640693:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36e00, rc 0) 00000100:00100000:1.0:1780298642.640723:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ada300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2399:x1866777609398016:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 793us (884us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.640731:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8969 00000100:00100000:0.0:1780298642.640779:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343967b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604891520:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.640795:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000658b0d9c/0xaf35132e6303340d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d261f expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.640812:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000658b0d9c/0xaf35132e6303340d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d261f expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.640821:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2c00, rc 0) 00000100:00100000:0.0:1780298642.640848:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609398144:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 829us (900us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.640854:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8970 00000100:00100000:3.0:1780298642.641643:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.641673:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.641696:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609398272 00000100:00100000:2.0:1780298642.641701:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609398272 00000100:00100000:0.0:1780298642.641711:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609398400 00000100:00100000:0.0:1780298642.641717:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609398400 00000100:00100000:2.0:1780298642.641731:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8971 00000100:00100000:2.0:1780298642.641738:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2397:x1866777609398272:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.641744:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.641752:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009c841b52/0xaf35132e63033414 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2626 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.641758:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8972 00000100:00100000:1.0:1780298642.641771:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609398400:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.641778:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298642.641781:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604891648:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.641787:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a10929f0/0xaf35132e6303341b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d262d expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.641812:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604891776:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.641820:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4900 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.641839:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34a90d3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.642527:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604891648:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.642548:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009c841b52/0xaf35132e63033414 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2626 expref: 16 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.642564:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009c841b52/0xaf35132e63033414 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2626 expref: 16 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.642575:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157600, rc 0) 00000100:00100000:1.0:1780298642.642597:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604891776:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.642619:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a10929f0/0xaf35132e6303341b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d262d expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.642642:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2397:x1866777609398272:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 902us (997us total) trans 0 rc 301/301 00010000:00010000:1.0:1780298642.642642:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a10929f0/0xaf35132e6303341b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d262d expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.642654:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8971 00010000:00010000:1.0:1780298642.642664:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37200, rc 0) 00000100:00100000:1.0:1780298642.642693:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609398400:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 926us (1020us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.642698:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8972 00000100:00100000:2.0:1780298642.643698:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.643717:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.643738:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609398528 00000100:00100000:3.0:1780298642.643741:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609398656 00000100:00100000:1.0:1780298642.643744:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609398528 00000100:00100000:3.0:1780298642.643747:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609398656 00000100:00100000:1.0:1780298642.643781:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8973 00000100:00100000:3.0:1780298642.643789:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8974 00000100:00100000:1.0:1780298642.643793:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2397:x1866777609398528:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.643802:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298642.643803:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609398656:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.643812:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.643813:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63033422 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2634 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.643822:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e63033429 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d263b expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.643845:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604891904:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.643855:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604892032:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.643893:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.643895:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34a90d3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.644556:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604891904:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.644567:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63033422 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2634 expref: 16 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.644576:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63033422 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2634 expref: 16 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.644583:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36a00, rc 0) 00000100:00100000:1.0:1780298642.644605:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2397:x1866777609398528:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 818us (907us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.644610:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8973 00000100:00100000:1.0:1780298642.644695:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604892032:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.644707:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e63033429 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d263b expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.644715:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e63033429 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d263b expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.644720:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3800, rc 0) 00000100:00100000:1.0:1780298642.644740:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609398656:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 942us (1021us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.644745:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8974 00000100:00100000:3.0:1780298642.645594:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.645636:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609398784 00000100:00100000:0.0:1780298642.645642:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609398784 00000100:00100000:0.0:1780298642.645672:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8975 00000100:00100000:0.0:1780298642.645681:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2399:x1866777609398784:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.645689:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.645699:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bde78cfa/0xaf35132e63033430 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2642 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.645730:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343967b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604892160:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.645768:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.645782:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417e40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.645806:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609398912 00000100:00100000:1.0:1780298642.645812:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609398912 00000100:00100000:1.0:1780298642.645841:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8976 00000100:00100000:1.0:1780298642.645849:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609398912:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.645856:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.645863:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63033437 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2649 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.645887:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604892288:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.645918:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34a90d3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.646337:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343967b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604892160:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.646354:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bde78cfa/0xaf35132e63033430 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2642 expref: 16 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.646368:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bde78cfa/0xaf35132e63033430 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2642 expref: 16 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.646379:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3a00, rc 0) 00000100:00100000:0.0:1780298642.646410:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2399:x1866777609398784:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 730us (817us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.646419:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8975 00000100:00100000:1.0:1780298642.646503:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604892288:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.646520:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63033437 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2649 expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.646534:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63033437 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2649 expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.646543:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37000, rc 0) 00000100:00100000:1.0:1780298642.646571:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609398912:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 723us (805us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.646577:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8976 00000100:00100000:2.0:1780298642.647353:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.647391:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609399040 00000100:00100000:0.0:1780298642.647396:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609399040 00000100:00100000:0.0:1780298642.647422:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8977 00000100:00100000:0.0:1780298642.647427:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2398:x1866777609399040:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.647434:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.647442:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008361787d/0xaf35132e6303343e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2650 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.647463:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.647466:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343967b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604892416:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.647490:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609399168 00000100:00100000:1.0:1780298642.647495:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609399168 00000100:00100000:0.0:1780298642.647501:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417e40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.647529:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8978 00000100:00100000:1.0:1780298642.647536:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481ffb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609399168:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.647543:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.647552:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078c7f460/0xaf35132e63033445 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2657 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.647577:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604892544:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.647608:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34a90d3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.648177:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343967b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604892416:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.648191:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604892544:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.648198:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008361787d/0xaf35132e6303343e lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2650 expref: 16 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.648213:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008361787d/0xaf35132e6303343e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2650 expref: 16 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.648216:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000078c7f460/0xaf35132e63033445 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2657 expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.648236:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2800, rc 0) 00010000:00010000:1.0:1780298642.648249:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078c7f460/0xaf35132e63033445 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2657 expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.648258:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36c00, rc 0) 00000100:00100000:0.0:1780298642.648270:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2398:x1866777609399040:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 840us (915us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.648279:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8977 00000100:00100000:1.0:1780298642.648285:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481ffb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609399168:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 747us (820us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.648305:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8978 00000100:00100000:2.0:1780298642.649399:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.649429:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609399296 00000100:00100000:0.0:1780298642.649433:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609399296 00000100:00100000:2.0:1780298642.649438:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.649462:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609399424 00000100:00100000:0.0:1780298642.649465:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609399424 00000100:00100000:3.0:1780298642.649505:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8979 00000100:00100000:1.0:1780298642.649516:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8980 00000100:00100000:3.0:1780298642.649517:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2399:x1866777609399296:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.649525:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298642.649529:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609399424:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.649532:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e6303344c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d265e expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.649538:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.649548:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009cb6f854/0xaf35132e63033453 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2665 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.649557:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604892672:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.649582:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604892800:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.649596:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.649615:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34a90d3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.650231:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604892672:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.650256:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e6303344c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d265e expref: 16 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.650268:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604892800:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.650269:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e6303344c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d265e expref: 16 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.650282:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2000, rc 0) 00010000:00010000:1.0:1780298642.650309:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009cb6f854/0xaf35132e63033453 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2665 expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.650319:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2399:x1866777609399296:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 805us (919us total) trans 0 rc 301/301 00010000:00010000:1.0:1780298642.650324:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009cb6f854/0xaf35132e63033453 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2665 expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.650328:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8979 00010000:00010000:1.0:1780298642.650340:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36e00, rc 0) 00000100:00100000:1.0:1780298642.650366:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609399424:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 841us (927us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.650374:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8980 00000100:00100000:0.0:1780298642.651345:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.651376:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609399552 00000100:00100000:2.0:1780298642.651380:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609399552 00000100:00100000:2.0:1780298642.651403:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8981 00000100:00100000:2.0:1780298642.651409:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343967b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:29360:x1866777609399552:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.651415:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.651422:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000700ba1da/0xaf35132e6303345a lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d266c expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.651435:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e630333dc lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1957888->4325375] (req 1957888->1961983) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d25ee expref: 16 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.651442:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(113 177) 00010000:00010000:2.0:1780298642.651445:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(114 177) 00010000:00010000:2.0:1780298642.651448:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000700ba1da/0xaf35132e6303345a lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2969600->2977791] (req 2969600->2977791) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d266c expref: 16 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.651463:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e630333dc lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1957888->4325375] (req 1957888->1961983) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d25ee expref: 16 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.651472:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e630333dc lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1957888->4325375] (req 1957888->1961983) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d25ee expref: 16 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.651478:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bf480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604892928:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298642.651508:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4900 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.652104:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bf480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604892928:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298642.652118:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000700ba1da/0xaf35132e6303345a lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2969600->2977791] (req 2969600->2977791) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d266c expref: 16 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.652144:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156000, rc 0) 00000100:00100000:3.0:1780298642.652165:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343967b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:29360:x1866777609399552:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 757us (820us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298642.652170:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8981 00000100:00100000:3.0:1780298642.652299:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.652326:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609399680 00000100:00100000:0.0:1780298642.652330:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609399680 00010000:00010000:0.0:1780298642.652345:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e630333dc lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1957888->4325375] (req 1957888->1961983) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d25ee expref: 17 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.652365:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15014 00000100:00100000:0.0:1780298642.652371:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609399680:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298642.652378:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298642.652386:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e630333dc lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1957888->4325375] (req 1957888->1961983) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d25ee expref: 17 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.652394:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e630333dc lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1957888->4325375] (req 1957888->1961983) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d25ee expref: 17 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.652406:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e630333dc lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1957888->4325375] (req 1957888->1961983) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d25ee expref: 17 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.652413:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(114 177) 00010000:00010000:0.0:1780298642.652415:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(114 177) 00010000:00010000:0.0:1780298642.652421:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000700ba1da/0xaf35132e6303345a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2969600->4325375] (req 2969600->2977791) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d266c expref: 15 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.652427:0:6463:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000700ba1da/0xaf35132e6303345a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2969600->4325375] (req 2969600->2977791) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d266c expref: 15 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.652441:0:6463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000700ba1da/0xaf35132e6303345a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2969600->4325375] (req 2969600->2977791) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d266c expref: 15 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.652447:0:6463:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343964700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6463:1866777604893056:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298642.652478:0:6463:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417e40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.653172:0:6463:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343964700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6463:1866777604893056:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298642.653183:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298642.653209:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609399680:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 838us (966us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.653219:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15014 00000100:00100000:3.0:1780298642.653379:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.653413:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609399808 00000100:00100000:0.0:1780298642.653417:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609399808 00000100:00100000:0.0:1780298642.653445:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6015 00000100:00100000:0.0:1780298642.653451:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609399808:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298642.653549:0:16972:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4334242 -> 1115924 00002000:00010000:0.0:1780298642.653552:0:16972:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:0.0:1780298642.653554:0:16972:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:0.0:1780298642.653556:0:16972:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:0.0:1780298642.653557:0:16972:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 400 -> 400 00000100:00100000:0.0:1780298642.653579:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2400:x1866777609399808:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 128us (201us total) trans 4294969633 rc 0/0 00000100:00100000:0.0:1780298642.653585:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6015 00000100:00100000:0.0:1780298642.654466:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.654495:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.654497:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609399936 00000100:00100000:1.0:1780298642.654501:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609399936 00000100:00100000:1.0:1780298642.654527:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609400064 00000100:00100000:1.0:1780298642.654529:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609400064 00000100:00100000:1.0:1780298642.654549:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8982 00000100:00100000:1.0:1780298642.654557:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343964700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609399936:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298642.654561:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8983 00010000:00010000:1.0:1780298642.654566:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298642.654575:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343967100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609400064:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.654576:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63033461 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2673 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.654586:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.654597:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63033468 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d267a expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.654615:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604893184:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.654629:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604893312:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.654662:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.654662:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34a90d3c0 going to sleep for 11 seconds 00002000:00010000:1.0:1780298642.655448:0:20252:0:(ofd_lvb.c:259:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size: 1115924 -> 4334242 00000100:00100000:2.0:1780298642.655451:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604893312:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.655458:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604893184:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.655475:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63033468 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d267a expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.655481:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63033461 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2673 expref: 17 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.655492:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63033468 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d267a expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.655496:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63033461 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2673 expref: 17 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.655510:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36a00, rc 0) 00010000:00010000:2.0:1780298642.655515:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2800, rc 0) 00000100:00100000:1.0:1780298642.655534:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343964700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609399936:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 980us (1071us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.655544:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343967100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609400064:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 973us (1049us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.655545:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8982 00000100:00100000:2.0:1780298642.655555:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8983 00000100:00100000:2.0:1780298642.656609:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.656642:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609400192 00000100:00100000:1.0:1780298642.656645:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609400192 00000100:00100000:1.0:1780298642.656672:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8984 00000100:00100000:1.0:1780298642.656678:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609400192:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.656684:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.656691:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e6303346f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2681 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.656720:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604893440:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.656746:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.656752:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34a90d3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.656779:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609400320 00000100:00100000:2.0:1780298642.656783:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609400320 00000100:00100000:2.0:1780298642.656810:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8985 00000100:00100000:2.0:1780298642.656817:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609400320:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.656823:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.656830:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c841b52/0xaf35132e63033476 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2688 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.656849:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604893568:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.656873:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4000 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.657315:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604893440:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.657326:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e6303346f lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2681 expref: 17 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.657335:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a10929f0/0xaf35132e6303346f lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2681 expref: 17 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.657342:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37200, rc 0) 00000100:00100000:1.0:1780298642.657361:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609400192:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 684us (754us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.657365:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8984 00000100:00100000:2.0:1780298642.657536:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604893568:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.657552:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009c841b52/0xaf35132e63033476 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2688 expref: 12 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.657561:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c841b52/0xaf35132e63033476 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2688 expref: 12 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.657567:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157600, rc 0) 00000100:00100000:2.0:1780298642.657590:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609400320:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 773us (847us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.657596:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8985 00000100:00100000:2.0:1780298642.658530:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.658567:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609400448 00000100:00100000:0.0:1780298642.658572:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609400448 00000100:00100000:0.0:1780298642.658603:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8986 00000100:00100000:0.0:1780298642.658612:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:29360:x1866777609400448:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.658620:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.658629:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bde78cfa/0xaf35132e6303347d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d268f expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.658645:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(85 130) 00010000:00010000:0.0:1780298642.658649:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(85 130) 00010000:00010000:0.0:1780298642.658658:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bde78cfa/0xaf35132e6303347d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2424832->2834431] (req 2424832->2433023) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d268f expref: 9 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.658672:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bde78cfa/0xaf35132e6303347d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2424832->2834431] (req 2424832->2433023) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d268f expref: 9 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.658688:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3a00, rc 0) 00000100:00100000:0.0:1780298642.658717:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:29360:x1866777609400448:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 107us (188us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.658724:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8986 00000100:00100000:0.0:1780298642.659587:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.659622:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609400576 00000100:00100000:1.0:1780298642.659626:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609400576 00002000:00100000:1.0:1780298642.659647:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x12:0x0] (2424832->2428927) req@ffff9cf343964a80 x1866777609400576/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298642.659654:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf343964a80 with x1866777609400576 ext(2424832->2428927) 00002000:00010000:1.0:1780298642.659658:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf343964a80 00000100:00100000:1.0:1780298642.659674:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6016 00000100:00100000:1.0:1780298642.659680:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343964a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609400576:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298642.659809:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343964a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609400576:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 129us (222us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.659814:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6016 00002000:00100000:1.0:1780298642.659819:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x12:0x0] (2424832->2428927) req@ffff9cf343964a80 x1866777609400576/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/440 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298642.659825:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf343964a80 with x1866777609400576 ext(2424832->2428927) 00002000:00010000:1.0:1780298642.659828:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf343964a80 00000100:00100000:3.0:1780298642.660544:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.660579:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609400704 00000100:00100000:0.0:1780298642.660584:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609400704 00002000:00100000:0.0:1780298642.660604:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (2428928->2433023) req@ffff9cf3481ff480 x1866777609400704/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.660615:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481ff480 with x1866777609400704 ext(2428928->2433023) 00002000:00010000:0.0:1780298642.660618:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf3481ff480 00000100:00100000:0.0:1780298642.660638:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6017 00000100:00100000:0.0:1780298642.660645:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481ff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609400704:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298642.660723:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481ff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609400704:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 78us (178us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.660731:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6017 00002000:00100000:0.0:1780298642.660738:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (2428928->2433023) req@ffff9cf3481ff480 x1866777609400704/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/440 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.660747:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481ff480 with x1866777609400704 ext(2428928->2433023) 00002000:00010000:0.0:1780298642.660751:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf3481ff480 00000100:00100000:2.0:1780298642.661616:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.661646:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.661663:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609400832 00000100:00100000:3.0:1780298642.661667:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609400832 00000100:00100000:0.0:1780298642.661677:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609400960 00000100:00100000:0.0:1780298642.661682:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609400960 00000100:00100000:3.0:1780298642.661698:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8987 00000100:00100000:3.0:1780298642.661705:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7be300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609400832:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.661712:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.661721:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63033484 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2696 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.661721:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8988 00000100:00100000:0.0:1780298642.661732:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609400960:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.661739:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.661747:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d4afe0b/0xaf35132e6303348b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d269d expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.661755:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604893696:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.661776:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343964000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604893824:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.661800:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.661801:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484173c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.662530:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604893696:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.662549:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63033484 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2696 expref: 17 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.662563:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63033484 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2696 expref: 17 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.662570:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343964000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604893824:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.662573:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2600, rc 0) 00000100:00100000:3.0:1780298642.662600:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7be300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609400832:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 895us (984us total) trans 0 rc 301/301 00010000:00010000:0.0:1780298642.662603:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001d4afe0b/0xaf35132e6303348b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d269d expref: 12 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.662608:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8987 00010000:00010000:0.0:1780298642.662625:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d4afe0b/0xaf35132e6303348b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d269d expref: 12 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.662637:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3000, rc 0) 00000100:00100000:0.0:1780298642.662662:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609400960:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 932us (1015us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.662670:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8988 00000100:00100000:2.0:1780298642.663627:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.663707:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609401088 00000100:00100000:3.0:1780298642.663711:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609401088 00000100:00100000:3.0:1780298642.663731:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8989 00000100:00100000:3.0:1780298642.663736:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255491f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609401088:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.663743:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.663750:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e63033492 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d26a4 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.663775:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604893952:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.663800:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.663810:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.663838:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609401216 00000100:00100000:1.0:1780298642.663844:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609401216 00000100:00100000:1.0:1780298642.663878:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8990 00000100:00100000:1.0:1780298642.663888:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255490e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609401216:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.663897:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.663905:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009cb6f854/0xaf35132e63033499 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d26ab expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.663932:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604894080:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.663968:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34a90d3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.664406:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604893952:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.664423:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e63033492 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d26a4 expref: 17 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.664437:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e63033492 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d26a4 expref: 17 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.664447:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2e00, rc 0) 00000100:00100000:3.0:1780298642.664522:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255491f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609401088:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 783us (895us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.664531:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8989 00000100:00100000:1.0:1780298642.664674:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604894080:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.664692:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009cb6f854/0xaf35132e63033499 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d26ab expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.664706:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009cb6f854/0xaf35132e63033499 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d26ab expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.664715:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36e00, rc 0) 00000100:00100000:1.0:1780298642.664744:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255490e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609401216:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 858us (943us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.664751:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8990 00000100:00100000:2.0:1780298642.665567:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.665603:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609401344 00000100:00100000:1.0:1780298642.665608:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609401344 00000100:00100000:2.0:1780298642.665619:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.665646:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609401472 00000100:00100000:1.0:1780298642.665649:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609401472 00000100:00100000:3.0:1780298642.665658:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8991 00000100:00100000:0.0:1780298642.665669:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8992 00000100:00100000:0.0:1780298642.665679:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609401472:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.665687:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.665695:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a269a887/0xaf35132e630334a0 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d26b9 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.665706:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255493100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2397:x1866777609401344:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.665716:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298642.665721:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343964000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604894208:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.665726:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e630334a7 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d26b2 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.665758:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484173c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.665759:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604894336:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.665790:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.666596:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604894336:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.666601:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343964000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604894208:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.666622:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e630334a7 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d26b2 expref: 18 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.666622:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a269a887/0xaf35132e630334a0 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d26b9 expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.666642:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a269a887/0xaf35132e630334a0 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d26b9 expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.666650:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e630334a7 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d26b2 expref: 18 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.666652:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3e00, rc 0) 00010000:00010000:3.0:1780298642.666665:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3000, rc 0) 00000100:00100000:0.0:1780298642.666685:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609401472:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1007us (1066us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.666691:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255493100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2397:x1866777609401344:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1022us (1123us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.666704:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8992 00000100:00100000:3.0:1780298642.666714:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8991 00000100:00100000:2.0:1780298642.667624:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.667657:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609401600 00000100:00100000:1.0:1780298642.667661:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609401600 00000100:00100000:1.0:1780298642.667685:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8993 00000100:00100000:1.0:1780298642.667690:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:29360:x1866777609401600:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.667696:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.667706:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a10929f0/0xaf35132e630334ae lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d26c0 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.667722:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(85 130) 00010000:00010000:1.0:1780298642.667727:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(85 130) 00010000:00010000:1.0:1780298642.667738:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a10929f0/0xaf35132e630334ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1384448->2424831] (req 1384448->1388543) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d26c0 expref: 10 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.667753:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a10929f0/0xaf35132e630334ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1384448->2424831] (req 1384448->1388543) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d26c0 expref: 10 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.667762:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37200, rc 0) 00000100:00100000:1.0:1780298642.667785:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:29360:x1866777609401600:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 95us (162us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.667790:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8993 00000100:00100000:2.0:1780298642.668571:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.668609:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609401728 00000100:00100000:0.0:1780298642.668612:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609401728 00002000:00100000:0.0:1780298642.668635:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (1384448->1388543) req@ffff9cf37f747100 x1866777609401728/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.668641:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f747100 with x1866777609401728 ext(1384448->1388543) 00002000:00010000:0.0:1780298642.668645:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f747100 00000100:00100000:0.0:1780298642.668662:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6018 00000100:00100000:0.0:1780298642.668666:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f747100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609401728:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298642.668776:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f747100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609401728:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 109us (204us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.668781:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6018 00002000:00100000:0.0:1780298642.668785:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (1384448->1388543) req@ffff9cf37f747100 x1866777609401728/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/440 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.668792:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f747100 with x1866777609401728 ext(1384448->1388543) 00002000:00010000:0.0:1780298642.668794:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f747100 00000100:00100000:2.0:1780298642.669888:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.669921:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609401856 00000100:00100000:0.0:1780298642.669926:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609401856 00002000:00100000:0.0:1780298642.669946:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (1388544->1392639) req@ffff9cf37f744700 x1866777609401856/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.669953:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f744700 with x1866777609401856 ext(1388544->1392639) 00002000:00010000:0.0:1780298642.669956:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f744700 00000100:00100000:0.0:1780298642.669975:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6019 00000100:00100000:0.0:1780298642.669980:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f744700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609401856:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298642.670038:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f744700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609401856:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 59us (150us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.670043:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6019 00002000:00100000:0.0:1780298642.670048:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (1388544->1392639) req@ffff9cf37f744700 x1866777609401856/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/440 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.670053:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f744700 with x1866777609401856 ext(1388544->1392639) 00002000:00010000:0.0:1780298642.670056:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f744700 00000100:00100000:2.0:1780298642.671051:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.671091:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609401984 00000100:00100000:0.0:1780298642.671096:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609401984 00002000:00100000:0.0:1780298642.671116:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x12:0x0] (1384448->1392639) req@ffff9cf37f745c00 x1866777609401984/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 8680/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.671149:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f745c00 with x1866777609401984 ext(1384448->1392639) 00002000:00010000:0.0:1780298642.671153:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f745c00 00000100:00100000:0.0:1780298642.671184:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6020 00000100:00100000:0.0:1780298642.671195:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609401984:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:0.0:1780298642.671669:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609401984:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 477us (617us total) trans 4294969200 rc 0/0 00000100:00100000:0.0:1780298642.671677:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6020 00002000:00100000:0.0:1780298642.671684:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x12:0x0] (1384448->1392639) req@ffff9cf37f745c00 x1866777609401984/t4294969200(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 8680/448 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.671692:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f745c00 with x1866777609401984 ext(1384448->1392639) 00002000:00010000:0.0:1780298642.671695:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f745c00 00000100:00100000:2.0:1780298642.672418:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.672466:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609402112 00000100:00100000:1.0:1780298642.672471:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609402112 00000100:00100000:1.0:1780298642.672501:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8994 00000100:00100000:1.0:1780298642.672509:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f755500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609402112:12345-192.168.203.25@tcp:16:fsx.0 00000001:00080000:0.0:1780298642.673226:0:8351:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0000: transno 4294969200 is committed 00000001:00080000:0.0:1780298642.673234:0:8351:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0000: transno 4294969198 is committed 00000100:00100000:1.0:1780298642.673323:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f755500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609402112:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 814us (905us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.673331:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8994 00000100:00100000:1.0:1780298642.674259:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.674306:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609402240 00000100:00100000:2.0:1780298642.674310:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609402240 00000100:00100000:1.0:1780298642.674316:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.674336:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609402368 00000100:00100000:2.0:1780298642.674339:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609402368 00000100:00100000:3.0:1780298642.674356:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8995 00000100:00100000:2.0:1780298642.674363:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8996 00000100:00100000:3.0:1780298642.674368:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609402240:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298642.674375:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b41880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609402368:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.674378:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.674382:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.674394:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e630334bc lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d26c7 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.674394:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008746f5b4/0xaf35132e630334b5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d26ce expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.674456:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f756300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604894464:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.674457:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604894592:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.674519:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.674520:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.675251:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604894592:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.675279:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e630334bc lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d26c7 expref: 17 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1780298642.675279:0:20252:0:(ofd_lvb.c:259:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size: 148373 -> 1392640 00002000:00010000:1.0:1780298642.675287:0:20252:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks: 8 -> 24 00000100:00100000:1.0:1780298642.675307:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f756300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604894464:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.675328:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e630334bc lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d26c7 expref: 17 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.675335:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008746f5b4/0xaf35132e630334b5 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d26ce expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.675342:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2e00, rc 0) 00010000:00010000:1.0:1780298642.675355:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008746f5b4/0xaf35132e630334b5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d26ce expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.675367:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157200, rc 0) 00000100:00100000:3.0:1780298642.675376:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609402240:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1011us (1119us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.675383:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8995 00000100:00100000:1.0:1780298642.675394:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b41880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609402368:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1022us (1078us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.675403:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8996 00000100:00100000:0.0:1780298642.676259:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.676278:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.676301:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609402624 00000100:00100000:1.0:1780298642.676305:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609402624 00000100:00100000:2.0:1780298642.676334:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609402496 00000100:00100000:1.0:1780298642.676339:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8997 00000100:00100000:2.0:1780298642.676340:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609402496 00000100:00100000:1.0:1780298642.676350:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343964000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609402624:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.676360:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.676370:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009cb6f854/0xaf35132e630334c3 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d26dc expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.676375:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8998 00000100:00100000:2.0:1780298642.676385:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343965c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609402496:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.676391:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.676399:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009c841b52/0xaf35132e630334ca lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d26d5 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.676410:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b41500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604894720:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.676436:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f756680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604894848:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.676453:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34a90d3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.676471:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.677081:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b41500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604894720:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.677096:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009cb6f854/0xaf35132e630334c3 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d26dc expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.677107:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009cb6f854/0xaf35132e630334c3 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d26dc expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.677113:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36e00, rc 0) 00000100:00100000:1.0:1780298642.677168:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343964000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609402624:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 820us (908us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.677176:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8997 00000100:00100000:3.0:1780298642.677231:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f756680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604894848:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.677309:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009c841b52/0xaf35132e630334ca lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d26d5 expref: 17 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.677319:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009c841b52/0xaf35132e630334ca lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d26d5 expref: 17 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.677326:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157600, rc 0) 00000100:00100000:3.0:1780298642.677357:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343965c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609402496:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 975us (1079us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.677362:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8998 00000100:00100000:2.0:1780298642.678103:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.678156:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609402752 00000100:00100000:1.0:1780298642.678161:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609402752 00000100:00100000:1.0:1780298642.678188:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 8999 00000100:00100000:1.0:1780298642.678197:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f757b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:29360:x1866777609402752:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.678205:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.678213:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e630334d1 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d26e3 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.678230:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(114 177) 00010000:00010000:1.0:1780298642.678234:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(114 177) 00010000:00010000:1.0:1780298642.678243:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e630334d1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2514944->2969599] (req 2514944->2527231) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d26e3 expref: 17 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.678258:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e630334d1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2514944->2969599] (req 2514944->2527231) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d26e3 expref: 17 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.678276:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36a00, rc 0) 00000100:00100000:1.0:1780298642.678312:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f757b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:29360:x1866777609402752:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 117us (210us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.678319:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 8999 00000100:00100000:2.0:1780298642.678962:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.679000:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609402880 00000100:00100000:1.0:1780298642.679004:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609402880 00000100:00100000:1.0:1780298642.679032:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6021 00000100:00100000:1.0:1780298642.679037:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f755c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2399:x1866777609402880:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298642.679234:0:19943:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4334242 -> 2523814 00002000:00010000:1.0:1780298642.679237:0:19943:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298642.679240:0:19943:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298642.679243:0:19943:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298642.679245:0:19943:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 400 -> 424 00000100:00100000:1.0:1780298642.679273:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f755c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2399:x1866777609402880:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 235us (312us total) trans 4294969634 rc 0/0 00000100:00100000:1.0:1780298642.679281:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6021 00000100:00100000:0.0:1780298642.680175:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.680208:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609403008 00000100:00100000:1.0:1780298642.680211:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609403008 00000100:00100000:0.0:1780298642.680223:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.680249:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609403136 00000100:00100000:1.0:1780298642.680251:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609403136 00000100:00100000:1.0:1780298642.680270:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9000 00000100:00100000:1.0:1780298642.680278:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343966a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2400:x1866777609403008:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298642.680280:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9001 00010000:00010000:1.0:1780298642.680287:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298642.680307:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343966d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609403136:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.680307:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e630334d8 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d26ea expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.680321:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.680331:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042c16ab0/0xaf35132e630334df lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d26f1 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.680351:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b41500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604894976:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.680361:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f755880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604895104:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.680402:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34a90d3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.680403:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.681209:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f755880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604895104:192.168.203.25@tcp:106: 00002000:00010000:1.0:1780298642.681221:0:20252:0:(ofd_lvb.c:259:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size: 2523814 -> 4334242 00000100:00100000:1.0:1780298642.681233:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b41500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604894976:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.681235:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000042c16ab0/0xaf35132e630334df lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d26f1 expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.681254:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042c16ab0/0xaf35132e630334df lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d26f1 expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.681258:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e630334d8 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d26ea expref: 18 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.681273:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156800, rc 0) 00010000:00010000:1.0:1780298642.681281:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e630334d8 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d26ea expref: 18 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.681303:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36c00, rc 0) 00000100:00100000:0.0:1780298642.681319:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343966d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609403136:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1028us (1095us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.681330:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343966a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2400:x1866777609403008:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1051us (1153us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.681336:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9001 00000100:00100000:1.0:1780298642.681350:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9000 00000100:00100000:3.0:1780298642.682212:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.682231:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.682249:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609403264 00000100:00100000:2.0:1780298642.682254:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609403264 00000100:00100000:0.0:1780298642.682257:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609403392 00000100:00100000:0.0:1780298642.682260:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609403392 00000100:00100000:2.0:1780298642.682284:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9002 00000100:00100000:0.0:1780298642.682301:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9003 00000100:00100000:2.0:1780298642.682303:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2399:x1866777609403264:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298642.682310:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609403392:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.682311:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.682316:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.682320:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e630334e6 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d26f8 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.682322:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5a492f0/0xaf35132e630334ed lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d26ff expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.682348:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f755880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604895232:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.682348:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343966300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604895360:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.682387:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.682387:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484173c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.683081:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f755880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604895232:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.683101:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e630334e6 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d26f8 expref: 18 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.683103:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343966300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604895360:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.683143:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a5a492f0/0xaf35132e630334ed lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d26ff expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.683152:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e630334e6 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d26f8 expref: 18 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.683164:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5a492f0/0xaf35132e630334ed lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d26ff expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.683167:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157c00, rc 0) 00010000:00010000:0.0:1780298642.683180:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3600, rc 0) 00000100:00100000:0.0:1780298642.683215:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609403392:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 906us (981us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.683223:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2399:x1866777609403264:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 929us (1008us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.683226:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9003 00000100:00100000:2.0:1780298642.683236:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9002 00000100:00100000:3.0:1780298642.684317:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.684356:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609403520 00000100:00100000:0.0:1780298642.684362:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609403520 00000100:00100000:0.0:1780298642.684392:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9004 00000100:00100000:0.0:1780298642.684402:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2398:x1866777609403520:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.684408:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.684416:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e630334f4 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2706 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.684440:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343966300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604895488:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.684455:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.684478:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484173c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.684501:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609403648 00000100:00100000:1.0:1780298642.684506:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609403648 00000100:00100000:1.0:1780298642.684533:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9005 00000100:00100000:1.0:1780298642.684541:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609403648:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.684549:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.684555:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e630334fb lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d270d expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.684579:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b41500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604895616:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.684608:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34a90d3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.685160:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343966300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604895488:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.685172:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e630334f4 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2706 expref: 18 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.685183:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e630334f4 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2706 expref: 18 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.685191:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3800, rc 0) 00000100:00100000:0.0:1780298642.685218:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2398:x1866777609403520:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 819us (902us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.685225:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9004 00000100:00100000:1.0:1780298642.685339:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b41500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604895616:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.685356:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e630334fb lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d270d expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.685365:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e630334fb lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d270d expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.685372:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37000, rc 0) 00000100:00100000:1.0:1780298642.685391:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609403648:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 853us (939us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.685396:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9005 00000100:00100000:3.0:1780298642.686263:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.686298:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609403776 00000100:00100000:1.0:1780298642.686303:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609403776 00000100:00100000:3.0:1780298642.686321:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.686331:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609403904 00000100:00100000:1.0:1780298642.686333:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609403904 00000100:00100000:0.0:1780298642.686345:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9006 00000100:00100000:0.0:1780298642.686357:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2399:x1866777609403776:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298642.686365:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9007 00010000:00010000:0.0:1780298642.686365:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298642.686374:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609403904:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.686376:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e63033502 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2714 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.686382:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.686391:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009cb6f854/0xaf35132e63033509 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d271b expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.686406:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343966300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604895744:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.686417:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b41500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604895872:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.686442:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484173c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.686445:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34a90d3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.687095:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b41500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604895872:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.687108:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009cb6f854/0xaf35132e63033509 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d271b expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.687115:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343966300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604895744:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.687156:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009cb6f854/0xaf35132e63033509 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d271b expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.687161:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e63033502 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2714 expref: 18 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.687177:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36e00, rc 0) 00010000:00010000:0.0:1780298642.687181:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e63033502 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2714 expref: 18 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.687190:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2000, rc 0) 00000100:00100000:1.0:1780298642.687206:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609403904:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 834us (885us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.687211:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9007 00000100:00100000:0.0:1780298642.687220:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2399:x1866777609403776:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 865us (955us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.687229:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9006 00000100:00100000:2.0:1780298642.688169:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.688193:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609404032 00000100:00100000:0.0:1780298642.688197:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609404032 00000100:00100000:0.0:1780298642.688229:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9008 00000100:00100000:0.0:1780298642.688238:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e581850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:29360:x1866777609404032:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.688246:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.688253:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a269a887/0xaf35132e63033510 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2722 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.688265:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000700ba1da/0xaf35132e6303345a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2969600->4325375] (req 2969600->2977791) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d266c expref: 16 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.688271:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(114 177) 00010000:00010000:0.0:1780298642.688274:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(115 177) 00010000:00010000:0.0:1780298642.688278:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a269a887/0xaf35132e63033510 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3764224->3776511] (req 3764224->3776511) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2722 expref: 19 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.688302:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000700ba1da/0xaf35132e6303345a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2969600->4325375] (req 2969600->2977791) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d266c expref: 16 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.688313:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000700ba1da/0xaf35132e6303345a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2969600->4325375] (req 2969600->2977791) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d266c expref: 16 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.688320:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343966300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604896000:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298642.688349:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484173c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.688848:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343966300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604896000:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298642.688860:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a269a887/0xaf35132e63033510 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3764224->3776511] (req 3764224->3776511) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2722 expref: 19 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.688869:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3e00, rc 0) 00000100:00100000:0.0:1780298642.688890:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e581850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:29360:x1866777609404032:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 655us (721us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.688894:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9008 00000100:00100000:2.0:1780298642.689076:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.689110:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609404160 00000100:00100000:1.0:1780298642.689113:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609404160 00010000:00010000:1.0:1780298642.689154:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000700ba1da/0xaf35132e6303345a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2969600->4325375] (req 2969600->2977791) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d266c expref: 17 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.689188:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15015 00000100:00100000:1.0:1780298642.689195:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f756680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2399:x1866777609404160:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298642.689201:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298642.689208:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000700ba1da/0xaf35132e6303345a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [2969600->4325375] (req 2969600->2977791) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d266c expref: 17 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.689216:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000700ba1da/0xaf35132e6303345a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [2969600->4325375] (req 2969600->2977791) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d266c expref: 17 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.689229:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000700ba1da/0xaf35132e6303345a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [2969600->4325375] (req 2969600->2977791) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d266c expref: 17 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.689239:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(115 177) 00010000:00010000:1.0:1780298642.689248:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(115 177) 00010000:00010000:1.0:1780298642.689256:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a269a887/0xaf35132e63033510 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3764224->4325375] (req 3764224->3776511) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2722 expref: 18 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.689266:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a269a887/0xaf35132e63033510 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3764224->4325375] (req 3764224->3776511) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2722 expref: 18 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.689300:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000a269a887/0xaf35132e63033510 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3764224->4325375] (req 3764224->3776511) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2722 expref: 18 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.689311:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b41500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777604896128:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298642.689342:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34a90d3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.689887:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b41500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777604896128:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298642.689894:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298642.689914:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f756680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2399:x1866777609404160:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 722us (839us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.689919:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15015 00000100:00100000:3.0:1780298642.690167:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.690190:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609404288 00000100:00100000:0.0:1780298642.690194:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609404288 00002000:00100000:0.0:1780298642.690220:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (3764224->3768319) req@ffff9cf36e56fb80 x1866777609404288/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.690230:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e56fb80 with x1866777609404288 ext(3764224->3768319) 00002000:00010000:0.0:1780298642.690234:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf36e56fb80 00000100:00100000:0.0:1780298642.690254:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6022 00000100:00100000:0.0:1780298642.690263:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2400:x1866777609404288:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298642.690387:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2400:x1866777609404288:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 126us (220us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.690391:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6022 00002000:00100000:0.0:1780298642.690396:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (3764224->3768319) req@ffff9cf36e56fb80 x1866777609404288/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/440 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.690401:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e56fb80 with x1866777609404288 ext(3764224->3768319) 00002000:00010000:0.0:1780298642.690404:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf36e56fb80 00000100:00100000:2.0:1780298642.691104:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.691158:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609404416 00000100:00100000:0.0:1780298642.691163:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609404416 00002000:00100000:0.0:1780298642.691187:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (3772416->3776511) req@ffff9cf37f756300 x1866777609404416/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.691196:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f756300 with x1866777609404416 ext(3772416->3776511) 00002000:00010000:0.0:1780298642.691199:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f756300 00000100:00100000:0.0:1780298642.691218:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6023 00000100:00100000:0.0:1780298642.691227:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f756300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2398:x1866777609404416:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298642.691315:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f756300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2398:x1866777609404416:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 90us (212us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.691321:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6023 00002000:00100000:0.0:1780298642.691326:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (3772416->3776511) req@ffff9cf37f756300 x1866777609404416/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/440 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.691332:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f756300 with x1866777609404416 ext(3772416->3776511) 00002000:00010000:0.0:1780298642.691334:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f756300 00000100:00100000:2.0:1780298642.692244:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.692274:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609404544 00000100:00100000:1.0:1780298642.692278:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609404544 00000100:00100000:1.0:1780298642.692309:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9009 00000100:00100000:1.0:1780298642.692315:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f755500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2400:x1866777609404544:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.692322:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.692329:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e63033517 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2729 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.692354:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b41500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604896256:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.692386:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34a90d3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.692387:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.692417:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609404672 00000100:00100000:2.0:1780298642.692421:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609404672 00000100:00100000:2.0:1780298642.692450:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9010 00000100:00100000:2.0:1780298642.692457:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609404672:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.692463:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.692470:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bed20494/0xaf35132e6303351e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2730 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.692495:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f757b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604896384:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.692526:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.692977:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b41500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604896256:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.692991:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e63033517 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2729 expref: 20 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.693001:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e63033517 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2729 expref: 20 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.693008:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36c00, rc 0) 00000100:00100000:1.0:1780298642.693031:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f755500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2400:x1866777609404544:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 716us (787us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.693039:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9009 00000100:00100000:2.0:1780298642.693227:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f757b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604896384:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.693245:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bed20494/0xaf35132e6303351e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2730 expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.693255:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bed20494/0xaf35132e6303351e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2730 expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.693261:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157000, rc 0) 00000100:00100000:2.0:1780298642.693284:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609404672:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 829us (941us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.693298:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9010 00000100:00100000:2.0:1780298642.694038:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.694075:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609404800 00000100:00100000:0.0:1780298642.694079:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609404800 00000100:00100000:0.0:1780298642.694102:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9011 00000100:00100000:0.0:1780298642.694108:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f757b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2399:x1866777609404800:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.694114:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.694120:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008361787d/0xaf35132e63033525 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2737 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.694177:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.694179:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343967b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604896512:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.694195:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609404928 00000100:00100000:3.0:1780298642.694199:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609404928 00000100:00100000:0.0:1780298642.694220:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484173c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.694226:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9012 00000100:00100000:3.0:1780298642.694234:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b41500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609404928:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.694240:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.694248:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e6303352c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d273e expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.694269:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e56df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604896640:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.694333:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.694730:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343967b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604896512:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.694744:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008361787d/0xaf35132e63033525 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2737 expref: 20 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.694755:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008361787d/0xaf35132e63033525 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2737 expref: 20 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.694762:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2800, rc 0) 00000100:00100000:0.0:1780298642.694787:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f757b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2399:x1866777609404800:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 679us (749us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.694792:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9011 00000100:00100000:1.0:1780298642.694849:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e56df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604896640:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.694871:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e6303352c lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d273e expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.694885:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e6303352c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d273e expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.694895:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3000, rc 0) 00000100:00100000:1.0:1780298642.694920:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b41500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609404928:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 688us (747us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.694925:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9012 00000100:00100000:2.0:1780298642.695674:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.695698:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.695712:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609405056 00000100:00100000:1.0:1780298642.695717:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609405056 00000100:00100000:3.0:1780298642.695731:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609405184 00000100:00100000:3.0:1780298642.695735:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609405184 00000100:00100000:1.0:1780298642.695746:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9013 00000100:00100000:1.0:1780298642.695756:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f755880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2399:x1866777609405056:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.695764:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298642.695767:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9014 00010000:00010000:1.0:1780298642.695774:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e63033533 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2745 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.695775:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609405184:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.695791:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.695799:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e6303353a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d274c expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.695812:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b42680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604896768:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.695822:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e56df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604896896:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.695848:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.695848:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34a90d3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.696510:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b42680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604896768:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.696526:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e63033533 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2745 expref: 17 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.696535:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e56df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604896896:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.696539:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e63033533 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2745 expref: 17 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.696549:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37000, rc 0) 00010000:00010000:0.0:1780298642.696560:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e6303353a lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d274c expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.696576:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f755880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2399:x1866777609405056:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 821us (899us total) trans 0 rc 301/301 00010000:00010000:0.0:1780298642.696584:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e6303353a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d274c expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.696596:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2e00, rc 0) 00000100:00100000:1.0:1780298642.696600:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9013 00000100:00100000:0.0:1780298642.696618:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609405184:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 844us (919us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.696623:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9014 00000100:00100000:0.0:1780298642.697470:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.697503:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609405312 00000100:00100000:2.0:1780298642.697506:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609405312 00000100:00100000:2.0:1780298642.697537:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9015 00000100:00100000:2.0:1780298642.697551:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343967b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2398:x1866777609405312:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298642.697557:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:2.0:1780298642.697564:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.697575:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e63033541 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2753 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.697592:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609405440 00000100:00100000:1.0:1780298642.697596:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609405440 00000100:00100000:2.0:1780298642.697607:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604897024:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.697621:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9016 00000100:00100000:1.0:1780298642.697630:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609405440:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.697638:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.697648:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ac4408d/0xaf35132e63033548 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d275a expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.697652:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.697680:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b42680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604897152:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.697715:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34a90d3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.698267:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24310fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604897024:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.698272:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b42680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604897152:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.698283:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e63033541 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2753 expref: 17 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.698299:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000ac4408d/0xaf35132e63033548 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d275a expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.698306:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e63033541 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2753 expref: 17 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.698314:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156a00, rc 0) 00010000:00010000:1.0:1780298642.698314:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ac4408d/0xaf35132e63033548 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d275a expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.698325:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341499200, rc 0) 00000100:00100000:2.0:1780298642.698341:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343967b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2398:x1866777609405312:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 794us (872us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.698347:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609405440:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 719us (793us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.698351:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9015 00000100:00100000:1.0:1780298642.698358:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9016 00000100:00100000:0.0:1780298642.699173:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.699208:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609405696 00000100:00100000:2.0:1780298642.699212:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609405696 00000100:00100000:2.0:1780298642.699247:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61945 00000100:00100000:2.0:1780298642.699254:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343964700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+110:29360:x1866777609405696:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:2.0:1780298642.699302:0:6477:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298642.699317:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156400/0xaf35132e6303337a lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 61 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d258c expref: 110 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.699328:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.699331:0:6477:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298642.699451:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156600/0xaf35132e6303354f lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 60 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.699463:0:6477:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156e00/0xaf35132e63033350 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 60 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2569 expref: 99 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.699479:0:6477:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156600/0xaf35132e6303354f lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 60 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.699500:0:6477:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156e00/0xaf35132e63033350 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 60 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2569 expref: 99 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.699512:0:6477:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156e00/0xaf35132e63033350 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 60 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2569 expref: 99 pid: 6476 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.699520:0:6477:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310fb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6477:1866777604897280:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298642.699555:0:6477:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.700083:0:6477:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24310fb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6477:1866777604897280:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298642.700091:0:6477:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156600/0xaf35132e6303354f lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 60 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.700254:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.700320:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609405824 00000100:00100000:3.0:1780298642.700324:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609405824 00010000:00010000:3.0:1780298642.700338:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156e00/0xaf35132e63033350 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 60 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2569 expref: 100 pid: 6476 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.700359:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15016 00000100:00100000:3.0:1780298642.700367:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343967100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+100:2400:x1866777609405824:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298642.700376:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298642.700381:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156e00/0xaf35132e63033350 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 61 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2569 expref: 100 pid: 6476 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.700392:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156e00/0xaf35132e63033350 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 61 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2569 expref: 100 pid: 6476 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.700404:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156e00/0xaf35132e63033350 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 61 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2569 expref: 100 pid: 6476 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.700412:0:6462:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298642.700414:0:6462:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156600/0xaf35132e6303354f lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 60 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.700422:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156600/0xaf35132e6303354f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 60 type: IBT flags: 0x40290000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.700428:0:6462:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156600/0xaf35132e6303354f lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 60 type: IBT flags: 0x40290000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.700441:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298642.700461:0:6477:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156600/0xaf35132e6303354f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 59 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.700464:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343967100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+99:2400:x1866777609405824:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 98us (211us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298642.700477:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15016 00010000:00010000:2.0:1780298642.700479:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156600/0xaf35132e6303354f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 59 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1780298642.700526:0:6477:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991022 00000004:00010000:2.0:1780298642.700548:0:6477:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000308b7bfd reply state 00000000fb5c8c7d transno 4294991023 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156600/0xaf35132e6303354f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 59 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.700563:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156600/0xaf35132e6303354f lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 59 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.700569:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.700572:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1780298642.700574:0:6477:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1780298642.700627:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343964700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+112:29360:x1866777609405696:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1373us (1454us total) trans 4294991023 rc 0/0 00000100:00100000:2.0:1780298642.700639:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61945 00000020:00080000:2.0:1780298642.700643:0:6477:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1780298642.701166:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.701189:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609405952 00000100:00100000:2.0:1780298642.701193:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609405952 00000100:00100000:2.0:1780298642.701214:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9017 00000100:00100000:2.0:1780298642.701219:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343964000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:29360:x1866777609405952:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.701225:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.701230:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e63033556 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2761 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.701242:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e630334d1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2514944->2969599] (req 2514944->2527231) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d26e3 expref: 20 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.701248:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000a269a887/0xaf35132e63033510 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3764224->4325375] (req 3764224->3776511) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2722 expref: 20 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.701253:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e63033381 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [4325376->37879807] (req 4325376->4337663) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2593 expref: 15 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.701261:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 3 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(115 177) 00010000:00010000:2.0:1780298642.701263:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(118 177) 00010000:00010000:2.0:1780298642.701266:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e63033556 lrc: 6/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2761 expref: 20 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.701277:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e63033381 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [4325376->37879807] (req 4325376->4337663) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2593 expref: 15 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.701283:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e63033381 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [4325376->37879807] (req 4325376->4337663) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2593 expref: 15 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.701300:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604897408:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298642.701334:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000a269a887/0xaf35132e63033510 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3764224->4325375] (req 3764224->3776511) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2722 expref: 20 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.701341:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000a269a887/0xaf35132e63033510 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3764224->4325375] (req 3764224->3776511) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2722 expref: 20 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.701347:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25491a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604897536:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298642.701367:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e630334d1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2514944->2969599] (req 2514944->2527231) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d26e3 expref: 20 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.701373:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e630334d1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2514944->2969599] (req 2514944->2527231) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d26e3 expref: 20 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.701381:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254918a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604897664:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298642.701395:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.701937:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24310fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604897408:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298642.701946:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254918a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604897664:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298642.702011:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25491a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604897536:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298642.702032:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e63033556 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2761 expref: 20 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.702049:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157400, rc 0) 00000100:00100000:2.0:1780298642.702078:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343964000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:29360:x1866777609405952:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 858us (912us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.702085:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9017 00000100:00100000:3.0:1780298642.702338:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.702350:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.702376:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609406080 00000100:00100000:2.0:1780298642.702380:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609406080 00002000:00100000:2.0:1780298642.702400:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (4325376->4334241) req@ffff9cf343965c00 x1866777609406080/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 9360/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00000100:00100000:0.0:1780298642.702401:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00002000:00100000:2.0:1780298642.702410:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf343965c00 with x1866777609406080 ext(4325376->4334241) 00010000:00010000:2.0:1780298642.702416:0:8362:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e63033381 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [4325376->37879807] (req 4325376->4337663) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2593 expref: 16 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.702425:0:8362:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e63033381 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [4325376->37879807] (req 4325376->4337663) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2593 expref: 16 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.702428:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609406208 00000100:00100000:3.0:1780298642.702434:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609406208 00002000:00010000:2.0:1780298642.702436:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf343965c00 00000100:00100000:1.0:1780298642.702436:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609406336 00000100:00100000:1.0:1780298642.702442:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609406336 00010000:00010000:3.0:1780298642.702450:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e630334d1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2514944->2969599] (req 2514944->2527231) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d26e3 expref: 21 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00100000:1.0:1780298642.702461:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x2c0000401:0x11:0x0] (3764224->3776511) req@ffff9cf343966d80 x1866777609406336/t0(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 12776/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00000100:00100000:2.0:1780298642.702463:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 187 00002000:00100000:1.0:1780298642.702472:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf343966d80 with x1866777609406336 ext(3764224->3776511) 00000100:00100000:3.0:1780298642.702479:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15017 00010000:00010000:1.0:1780298642.702479:0:19951:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000a269a887/0xaf35132e63033510 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3764224->4325375] (req 3764224->3776511) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2722 expref: 21 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.702480:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343965c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609406080:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:3.0:1780298642.702492:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2400:x1866777609406208:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298642.702499:0:19951:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000a269a887/0xaf35132e63033510 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3764224->4325375] (req 3764224->3776511) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2722 expref: 21 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.702501:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:1.0:1780298642.702506:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf343966d80 00010000:00010000:3.0:1780298642.702513:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e630334d1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [2514944->2969599] (req 2514944->2527231) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d26e3 expref: 21 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.702521:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e630334d1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [2514944->2969599] (req 2514944->2527231) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d26e3 expref: 21 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.702529:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 188 00010000:00010000:3.0:1780298642.702535:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e630334d1 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [2514944->2969599] (req 2514944->2527231) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d26e3 expref: 21 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.702540:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343966d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2399:x1866777609406336:12345-192.168.203.25@tcp:4:fsx.0 00010000:00010000:3.0:1780298642.702554:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298642.702578:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56d180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2400:x1866777609406208:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 90us (240us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298642.702585:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15017 00000100:00100000:2.0:1780298642.702913:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343965c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2400:x1866777609406080:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 441us (565us total) trans 4294969635 rc 0/0 00000100:00100000:1.0:1780298642.702920:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343966d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2399:x1866777609406336:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 380us (519us total) trans 4294969636 rc 0/0 00000100:00100000:2.0:1780298642.702924:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 187 00002000:00100000:2.0:1780298642.702931:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (4325376->4334241) req@ffff9cf343965c00 x1866777609406080/t4294969635(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 9360/448 e 0 to 0 dl 1780298653 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00000100:00100000:1.0:1780298642.702937:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 188 00002000:00100000:2.0:1780298642.702944:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf343965c00 with x1866777609406080 ext(4325376->4334241) 00010000:00010000:2.0:1780298642.702949:0:8362:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e63033381 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [4325376->37879807] (req 4325376->4337663) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2593 expref: 18 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00100000:1.0:1780298642.702949:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x2c0000401:0x11:0x0] (3764224->3776511) req@ffff9cf343966d80 x1866777609406336/t4294969636(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 12776/448 e 0 to 0 dl 1780298653 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00010000:00010000:2.0:1780298642.702965:0:8362:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e63033381 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [4325376->37879807] (req 4325376->4337663) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2593 expref: 18 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1780298642.702972:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf343965c00 00002000:00100000:1.0:1780298642.702986:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf343966d80 with x1866777609406336 ext(3764224->3776511) 00010000:00010000:1.0:1780298642.702991:0:19951:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000a269a887/0xaf35132e63033510 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3764224->4325375] (req 3764224->3776511) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2722 expref: 21 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.702999:0:19951:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000a269a887/0xaf35132e63033510 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3764224->4325375] (req 3764224->3776511) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2722 expref: 21 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1780298642.703004:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf343966d80 00000100:00100000:3.0:1780298642.703628:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.703649:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.703666:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609406464 00000100:00100000:0.0:1780298642.703670:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609406464 00010000:00010000:0.0:1780298642.703686:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e63033381 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [4325376->37879807] (req 4325376->4337663) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2593 expref: 18 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.703687:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609406592 00000100:00100000:1.0:1780298642.703693:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609406592 00000100:00100000:0.0:1780298642.703709:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15018 00010000:00010000:1.0:1780298642.703714:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000a269a887/0xaf35132e63033510 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3764224->4325375] (req 3764224->3776511) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2722 expref: 21 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.703720:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2397:x1866777609406464:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298642.703730:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:0.0:1780298642.703736:0:6461:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 424 -> 472 00010000:00010000:0.0:1780298642.703740:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e63033381 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [4325376->37879807] (req 4325376->4337663) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2593 expref: 18 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.703752:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15019 00010000:00010000:0.0:1780298642.703756:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e63033381 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [4325376->37879807] (req 4325376->4337663) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2593 expref: 18 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.703763:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2397:x1866777609406592:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298642.703769:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e63033381 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [4325376->37879807] (req 4325376->4337663) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2593 expref: 18 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.703774:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298642.703777:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298642.703785:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000a269a887/0xaf35132e63033510 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3764224->4325375] (req 3764224->3776511) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2722 expref: 21 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.703794:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000a269a887/0xaf35132e63033510 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3764224->4325375] (req 3764224->3776511) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2722 expref: 21 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.703801:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2397:x1866777609406464:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 84us (174us total) trans 0 rc 0/0 00010000:00010000:1.0:1780298642.703808:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a269a887/0xaf35132e63033510 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3764224->4325375] (req 3764224->3776511) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2722 expref: 21 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.703809:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15018 00010000:00010000:1.0:1780298642.703824:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(118 177) 00010000:00010000:1.0:1780298642.703827:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(118 177) 00010000:00010000:1.0:1780298642.703832:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e63033556 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2761 expref: 20 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.703837:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e63033556 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2761 expref: 20 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.703854:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e63033556 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2761 expref: 20 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.703861:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b42680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777604897792:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298642.703892:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34a90d3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.704356:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b42680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777604897792:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298642.704363:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298642.704383:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2397:x1866777609406592:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 625us (734us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.704388:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15019 00000100:00100000:0.0:1780298642.704589:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.704612:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609406720 00000100:00100000:3.0:1780298642.704617:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609406720 00000100:00100000:3.0:1780298642.704646:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9018 00000100:00100000:3.0:1780298642.704654:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343966a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609406720:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.704662:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.704670:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e6303355d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2768 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.704685:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000a10929f0/0xaf35132e630334ae lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [1384448->2424831] (req 1384448->1388543) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d26c0 expref: 8 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.704693:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003e2cfab1/0xaf35132e630333ff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [2834432->36388863] (req 2834432->2842623) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2611 expref: 8 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.704701:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000bde78cfa/0xaf35132e6303347d lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [2424832->2834431] (req 2424832->2433023) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d268f expref: 8 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.704707:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 3 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(85 130) 00010000:00010000:3.0:1780298642.704711:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(88 130) 00010000:00010000:3.0:1780298642.704716:0:20252:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e6303355d lrc: 6/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2768 expref: 10 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.704734:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000bde78cfa/0xaf35132e6303347d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [2424832->2834431] (req 2424832->2433023) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d268f expref: 8 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.704745:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000bde78cfa/0xaf35132e6303347d lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [2424832->2834431] (req 2424832->2433023) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d268f expref: 8 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.704755:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e56df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604897920:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298642.704803:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003e2cfab1/0xaf35132e630333ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [2834432->36388863] (req 2834432->2842623) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2611 expref: 8 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.704813:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003e2cfab1/0xaf35132e630333ff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [2834432->36388863] (req 2834432->2842623) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2611 expref: 8 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.704825:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255093b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604898048:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298642.704902:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000a10929f0/0xaf35132e630334ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [1384448->2424831] (req 1384448->1388543) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d26c0 expref: 8 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.704913:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000a10929f0/0xaf35132e630334ae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [1384448->2424831] (req 1384448->1388543) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d26c0 expref: 8 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.704924:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255092d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604898176:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298642.704989:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.705246:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e56df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604897920:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298642.705421:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255093b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604898048:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298642.705576:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.705630:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609406848 00000100:00100000:3.0:1780298642.705634:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609406848 00000100:00100000:0.0:1780298642.705652:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.705672:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255092d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604898176:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298642.705678:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000bde78cfa/0xaf35132e6303347d lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [2424832->2834431] (req 2424832->2433023) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d268f expref: 9 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.705692:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609406976 00010000:00010000:1.0:1780298642.705696:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e6303355d lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2768 expref: 10 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.705697:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609406976 00000100:00100000:3.0:1780298642.705704:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15020 00010000:00010000:1.0:1780298642.705713:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2600, rc 0) 00010000:00010000:2.0:1780298642.705717:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000003e2cfab1/0xaf35132e630333ff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [2834432->36388863] (req 2834432->2842623) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2611 expref: 10 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.705719:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b01850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609406848:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298642.705734:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1780298642.705739:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343966a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609406720:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1085us (1148us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.705747:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9018 00010000:00010000:3.0:1780298642.705748:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000bde78cfa/0xaf35132e6303347d lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [2424832->2834431] (req 2424832->2433023) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d268f expref: 10 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.705757:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15021 00010000:00010000:3.0:1780298642.705765:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000bde78cfa/0xaf35132e6303347d lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [2424832->2834431] (req 2424832->2433023) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d268f expref: 10 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.705772:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343966300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609406976:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298642.705787:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298642.705789:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bde78cfa/0xaf35132e6303347d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [2424832->2834431] (req 2424832->2433023) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d268f expref: 10 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.705798:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000003e2cfab1/0xaf35132e630333ff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 8 type: EXT [2834432->36388863] (req 2834432->2842623) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2611 expref: 10 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.705810:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000003e2cfab1/0xaf35132e630333ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 8 type: EXT [2834432->36388863] (req 2834432->2842623) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2611 expref: 10 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.705823:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e2cfab1/0xaf35132e630333ff lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 8 type: EXT [2834432->36388863] (req 2834432->2842623) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2611 expref: 10 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.705832:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298642.705853:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343966300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609406976:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 89us (201us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.705859:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15021 00010000:00010000:3.0:1780298642.705878:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298642.705904:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.705926:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b01850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609406848:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 212us (349us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.705932:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609407104 00000100:00100000:1.0:1780298642.705935:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15020 00000100:00100000:2.0:1780298642.705937:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609407104 00010000:00010000:2.0:1780298642.705950:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000a10929f0/0xaf35132e630334ae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1384448->2424831] (req 1384448->1388543) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d26c0 expref: 7 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.705970:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15022 00000100:00100000:2.0:1780298642.705977:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343964e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609407104:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298642.705983:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298642.705989:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000a10929f0/0xaf35132e630334ae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1384448->2424831] (req 1384448->1388543) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d26c0 expref: 7 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.705999:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000a10929f0/0xaf35132e630334ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1384448->2424831] (req 1384448->1388543) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d26c0 expref: 7 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.706012:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a10929f0/0xaf35132e630334ae lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1384448->2424831] (req 1384448->1388543) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d26c0 expref: 7 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.706022:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(88 130) 00010000:00010000:2.0:1780298642.706026:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(88 130) 00010000:00010000:2.0:1780298642.706031:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e6303355d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2768 expref: 9 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.706040:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e6303355d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2768 expref: 9 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.706060:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e6303355d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2768 expref: 9 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.706070:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25491a680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6461:1866777604898304:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298642.706101:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.706795:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.706814:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25491a680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6461:1866777604898304:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298642.706822:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298642.706844:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343964e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609407104:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 868us (940us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.706849:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15022 00000100:00100000:3.0:1780298642.706875:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609407232 00000100:00100000:3.0:1780298642.706879:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609407232 00000100:00100000:0.0:1780298642.706888:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00002000:00010000:3.0:1780298642.706895:0:19943:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (282257->18446744073709551615) 00002000:00100000:3.0:1780298642.706899:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b42680 with x1866777609407232 ext(282257->18446744073709551615) 00002000:00010000:3.0:1780298642.706904:0:19943:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf242b42680. 00000100:00100000:2.0:1780298642.706926:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609407360 00000100:00100000:2.0:1780298642.706930:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609407360 00002000:00010000:2.0:1780298642.706943:0:19928:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00002000:00100000:2.0:1780298642.706946:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf343965f80 with x1866777609407360 ext(0->18446744073709551615) 00002000:00010000:2.0:1780298642.706950:0:19928:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf343965f80. 00000100:00100000:3.0:1780298642.706959:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6024 00000100:00100000:3.0:1780298642.706968:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b42680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2397:x1866777609407232:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:1.0:1780298642.706973:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6025 00000100:00100000:1.0:1780298642.706982:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343965f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609407360:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:1.0:1780298642.707144:0:19950:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 1392640 -> 0 00002000:00010000:1.0:1780298642.707147:0:19950:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298642.707150:0:19950:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298642.707151:0:19950:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298642.707153:0:19950:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 24 -> 0 00000100:00100000:1.0:1780298642.707174:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343965f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609407360:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 194us (288us total) trans 4294969201 rc 0/0 00000100:00100000:1.0:1780298642.707179:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6025 00002000:00010000:1.0:1780298642.707182:0:19950:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00002000:00100000:1.0:1780298642.707184:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf343965f80 with x1866777609407360 ext(0->18446744073709551615) 00002000:00010000:1.0:1780298642.707188:0:19950:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf343965f80. 00002000:00010000:3.0:1780298642.707808:0:19943:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4334242 -> 282257 00002000:00010000:3.0:1780298642.707812:0:19943:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:3.0:1780298642.707815:0:19943:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:3.0:1780298642.707817:0:19943:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:3.0:1780298642.707819:0:19943:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 472 -> 296 00000100:00100000:3.0:1780298642.707850:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b42680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2397:x1866777609407232:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 883us (1054us total) trans 4294969637 rc 0/0 00000100:00100000:3.0:1780298642.707858:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6024 00002000:00010000:3.0:1780298642.707862:0:19943:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (282257->18446744073709551615) 00002000:00100000:3.0:1780298642.707865:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b42680 with x1866777609407232 ext(282257->18446744073709551615) 00002000:00010000:3.0:1780298642.707870:0:19943:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf242b42680. 00000100:00100000:0.0:1780298642.708465:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.708496:0:7411:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609407488 00000100:00100000:1.0:1780298642.708500:0:7411:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609407488 00000100:00100000:1.0:1780298642.708523:0:7411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61946 00000100:00100000:1.0:1780298642.708528:0:7411:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343967100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+112:29360:x1866777609407488:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.708536:0:7411:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.708543:0:7411:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499c00/0xaf35132e63033564 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 60 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d276f expref: -99 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.708584:0:7411:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341498c00/0xaf35132e6303356b lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 61 type: IBT flags: 0x40000000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.708591:0:7411:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341498c00/0xaf35132e6303356b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 61 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.708597:0:7411:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341498c00/0xaf35132e6303356b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 61 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1780298642.708609:0:7411:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 282257 00000004:00200000:1.0:1780298642.708612:0:7411:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1780298642.708621:0:7411:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341498c00/0xaf35132e6303356b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 61 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.708633:0:7411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499c00/0xaf35132e63033564 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 61 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d276f expref: 114 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.708642:0:7411:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341498c00/0xaf35132e6303356b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 60 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d276f expref: 113 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.708646:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298642.708648:0:7411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1780298642.708650:0:7411:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341498c00, rc 0) 00000100:00100000:1.0:1780298642.708671:0:7411:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343967100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+113:29360:x1866777609407488:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 144us (208us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.708675:0:7411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61946 00000100:00100000:1.0:1780298642.710214:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.710256:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609407616 00000100:00100000:0.0:1780298642.710260:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609407616 00002000:00100000:0.0:1780298642.710274:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (0->282623) req@ffff9cf242b41c00 x1866777609407616/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.710280:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b41c00 with x1866777609407616 ext(0->282623) 00002000:00010000:0.0:1780298642.710283:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf242b41c00 00000100:00100000:0.0:1780298642.710307:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6026 00000100:00100000:0.0:1780298642.710313:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b41c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2398:x1866777609407616:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298642.711582:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b41c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2398:x1866777609407616:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 1267us (1368us total) trans 0 rc 282624/282624 00000100:00100000:0.0:1780298642.711589:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6026 00002000:00100000:0.0:1780298642.711597:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (0->282623) req@ffff9cf242b41c00 x1866777609407616/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/440 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 282624/282624 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.711606:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b41c00 with x1866777609407616 ext(0->282623) 00002000:00010000:0.0:1780298642.711610:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf242b41c00 00000100:00100000:0.0:1780298642.712232:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.712258:0:6476:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609407744 00000100:00100000:2.0:1780298642.712262:0:6476:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609407744 00000100:00100000:2.0:1780298642.712296:0:6476:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61947 00000100:00100000:2.0:1780298642.712304:0:6476:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+99:29360:x1866777609407744:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.712310:0:6476:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.712319:0:6476:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156400/0xaf35132e63033572 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 61 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2776 expref: -99 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.712352:0:6476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156a00/0xaf35132e63033579 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 62 type: IBT flags: 0x40000000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.712362:0:6476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156a00/0xaf35132e63033579 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 62 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.712370:0:6476:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156a00/0xaf35132e63033579 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 62 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298642.712386:0:6476:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 282257 00000004:00200000:2.0:1780298642.712389:0:6476:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298642.712399:0:6476:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156a00/0xaf35132e63033579 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 62 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.712412:0:6476:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156400/0xaf35132e63033572 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 62 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2776 expref: 101 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.712421:0:6476:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156a00/0xaf35132e63033579 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 61 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2776 expref: 100 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.712429:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.712431:0:6476:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298642.712433:0:6476:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156a00, rc 0) 00000100:00100000:2.0:1780298642.712456:0:6476:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+100:29360:x1866777609407744:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 154us (224us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.712460:0:6476:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61947 00000100:00100000:3.0:1780298642.713095:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.713145:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609407872 00000100:00100000:2.0:1780298642.713150:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609407872 00000100:00100000:2.0:1780298642.713184:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9019 00000100:00100000:2.0:1780298642.713193:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255091500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2397:x1866777609407872:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.713200:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.713207:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63033580 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d277d expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.713226:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.713235:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25491a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604898432:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.713253:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609408000 00000100:00100000:1.0:1780298642.713258:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609408000 00000100:00100000:2.0:1780298642.713276:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.713300:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9020 00000100:00100000:1.0:1780298642.713310:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609408000:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.713316:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.713324:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8a7e8eb/0xaf35132e63033587 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2784 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.713346:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b42a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604898560:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.713369:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3602870c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.713937:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25491a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604898432:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.713955:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63033580 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d277d expref: 18 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.713966:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63033580 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d277d expref: 18 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.713973:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157c00, rc 0) 00000100:00100000:1.0:1780298642.713974:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b42a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604898560:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.713993:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e8a7e8eb/0xaf35132e63033587 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2784 expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.714000:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255091500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2397:x1866777609407872:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 808us (904us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.714012:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9019 00010000:00010000:1.0:1780298642.714017:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8a7e8eb/0xaf35132e63033587 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2784 expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.714025:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341499e00, rc 0) 00000100:00100000:1.0:1780298642.714043:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609408000:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 737us (818us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.714048:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9020 00000100:00100000:3.0:1780298642.714806:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.714834:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609408128 00000100:00100000:2.0:1780298642.714838:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609408128 00000100:00100000:2.0:1780298642.714859:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9021 00000100:00100000:2.0:1780298642.714865:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255090a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2397:x1866777609408128:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.714873:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.714882:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009c841b52/0xaf35132e6303358e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d278b expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.714912:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25491a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604898688:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.714923:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.714954:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609408256 00000100:00100000:1.0:1780298642.714959:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609408256 00000100:00100000:2.0:1780298642.714963:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.714992:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9022 00000100:00100000:1.0:1780298642.715002:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609408256:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.715014:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.715021:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ac4408d/0xaf35132e63033595 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2792 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.715048:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b42a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604898816:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.715082:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3602870c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.715668:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25491a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604898688:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.715682:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009c841b52/0xaf35132e6303358e lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d278b expref: 18 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.715693:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009c841b52/0xaf35132e6303358e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d278b expref: 18 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.715699:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157600, rc 0) 00000100:00100000:1.0:1780298642.715710:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b42a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604898816:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.715724:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255090a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2397:x1866777609408128:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 859us (917us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.715729:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9021 00010000:00010000:1.0:1780298642.715733:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000ac4408d/0xaf35132e63033595 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2792 expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.715750:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ac4408d/0xaf35132e63033595 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2792 expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.715761:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341499200, rc 0) 00000100:00100000:1.0:1780298642.715792:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609408256:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 792us (870us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.715800:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9022 00000100:00100000:3.0:1780298642.716479:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.716509:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609408384 00000100:00100000:1.0:1780298642.716513:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609408384 00000100:00100000:1.0:1780298642.716536:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9023 00000100:00100000:1.0:1780298642.716541:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255090700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:29360:x1866777609408384:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.716547:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.716554:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e6303359c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2799 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.716568:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e63033556 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2761 expref: 20 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.716573:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(118 177) 00010000:00010000:1.0:1780298642.716576:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(119 177) 00010000:00010000:1.0:1780298642.716578:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e6303359c lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [110592->122879] (req 110592->122879) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2799 expref: 18 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.716590:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e63033556 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2761 expref: 20 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.716597:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e63033556 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2761 expref: 20 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.716603:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b42a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604898944:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298642.716632:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3602870c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.717097:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b42a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604898944:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298642.717106:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e6303359c lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [110592->122879] (req 110592->122879) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2799 expref: 18 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.717113:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36a00, rc 0) 00000100:00100000:1.0:1780298642.717149:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255090700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:29360:x1866777609408384:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 607us (669us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.717156:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9023 00000100:00100000:0.0:1780298642.717590:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.717622:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609408512 00000100:00100000:2.0:1780298642.717626:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609408512 00010000:00010000:2.0:1780298642.717645:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e63033556 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2761 expref: 21 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.717662:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15023 00000100:00100000:2.0:1780298642.717667:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ada300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2400:x1866777609408512:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298642.717673:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298642.717680:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e63033556 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2761 expref: 21 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.717687:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e63033556 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2761 expref: 21 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.717697:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e63033556 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2761 expref: 21 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.717704:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(119 177) 00010000:00010000:2.0:1780298642.717706:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(119 177) 00010000:00010000:2.0:1780298642.717711:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e6303359c lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 110592->122879) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2799 expref: 17 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.717716:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e6303359c lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 110592->122879) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2799 expref: 17 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.717729:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e6303359c lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 110592->122879) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2799 expref: 17 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.717739:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25491a680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777604899072:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298642.717765:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.718378:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25491a680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777604899072:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298642.718388:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298642.718409:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ada300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2400:x1866777609408512:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 743us (820us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.718415:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15023 00000100:00100000:3.0:1780298642.719098:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.719147:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609408640 00000100:00100000:0.0:1780298642.719152:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609408640 00002000:00100000:0.0:1780298642.719172:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (0->282623) req@ffff9cf255093480 x1866777609408640/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.719179:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255093480 with x1866777609408640 ext(0->282623) 00002000:00010000:0.0:1780298642.719181:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255093480 00000100:00100000:0.0:1780298642.719198:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6027 00000100:00100000:0.0:1780298642.719203:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255093480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2397:x1866777609408640:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298642.720105:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255093480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2397:x1866777609408640:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 901us (1012us total) trans 0 rc 282624/282624 00000100:00100000:2.0:1780298642.720110:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6027 00002000:00100000:2.0:1780298642.720115:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (0->282623) req@ffff9cf255093480 x1866777609408640/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/440 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 282624/282624 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.720122:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255093480 with x1866777609408640 ext(0->282623) 00002000:00010000:2.0:1780298642.720145:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255093480 00000100:00100000:3.0:1780298642.720905:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.720944:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609408768 00000100:00100000:1.0:1780298642.720949:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609408768 00000100:00100000:1.0:1780298642.720977:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9024 00000100:00100000:1.0:1780298642.720985:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255092680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2398:x1866777609408768:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.720993:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.721001:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e630335a3 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d27a0 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.721014:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(119 177) 00010000:00010000:1.0:1780298642.721018:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(119 177) 00010000:00010000:1.0:1780298642.721029:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e630335a3 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27a0 expref: 21 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.721041:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e630335a3 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27a0 expref: 21 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.721051:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37000, rc 0) 00000100:00100000:1.0:1780298642.721077:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255092680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2398:x1866777609408768:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 94us (173us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.721084:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9024 00000100:00100000:0.0:1780298642.721990:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.722022:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609408896 00000100:00100000:3.0:1780298642.722026:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609408896 00000100:00100000:3.0:1780298642.722047:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9025 00000100:00100000:3.0:1780298642.722052:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:29360:x1866777609408896:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.722057:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.722063:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e630335aa lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d27a7 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.722074:0:19992:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e6303359c lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 110592->122879) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2799 expref: 17 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.722080:0:19992:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e630335a3 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27a0 expref: 22 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.722084:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(119 177) 00010000:00010000:3.0:1780298642.722086:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(119 177) 00010000:00010000:3.0:1780298642.722089:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e630335aa lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2207744->2211839] (req 2207744->2211839) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27a7 expref: 22 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.722100:0:19992:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e630335a3 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27a0 expref: 22 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.722107:0:19992:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e630335a3 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27a0 expref: 22 pid: 12101 timeout: 932 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.722112:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255093100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604899200:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298642.722166:0:19992:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e6303359c lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 110592->122879) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2799 expref: 17 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.722173:0:19992:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e6303359c lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 110592->122879) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2799 expref: 17 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.722180:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255092d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604899328:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298642.722196:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.722756:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255093100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604899200:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298642.722769:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255092d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604899328:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298642.722858:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e630335aa lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2207744->2211839] (req 2207744->2211839) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27a7 expref: 22 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.722869:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:3.0:1780298642.722870:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2400, rc 0) 00000100:00100000:2.0:1780298642.722898:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609409024 00000100:00100000:2.0:1780298642.722902:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609409024 00000100:00100000:3.0:1780298642.722903:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:29360:x1866777609408896:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 849us (912us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298642.722912:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9025 00010000:00010000:2.0:1780298642.722919:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e630335a3 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27a0 expref: 23 pid: 12101 timeout: 932 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.722945:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15024 00000100:00100000:2.0:1780298642.722952:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adaa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:2399:x1866777609409024:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298642.722959:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298642.722969:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e630335a3 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27a0 expref: 22 pid: 12101 timeout: 932 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.722979:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e630335a3 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27a0 expref: 22 pid: 12101 timeout: 932 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.722994:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e630335a3 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27a0 expref: 22 pid: 12101 timeout: 932 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.723005:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298642.723031:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adaa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2399:x1866777609409024:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 80us (162us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.723036:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15024 00000100:00100000:0.0:1780298642.723307:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.723333:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609409152 00000100:00100000:1.0:1780298642.723337:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609409152 00010000:00010000:1.0:1780298642.723348:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e6303359c lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 110592->122879) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2799 expref: 18 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.723371:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15025 00000100:00100000:1.0:1780298642.723375:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2399:x1866777609409152:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298642.723381:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298642.723387:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e6303359c lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 110592->122879) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2799 expref: 18 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.723394:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e6303359c lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 110592->122879) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2799 expref: 18 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.723403:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e6303359c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 110592->122879) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2799 expref: 18 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.723410:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(119 177) 00010000:00010000:1.0:1780298642.723412:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(119 177) 00010000:00010000:1.0:1780298642.723416:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e630335aa lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2207744->35762175] (req 2207744->2211839) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27a7 expref: 20 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.723422:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e630335aa lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2207744->35762175] (req 2207744->2211839) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27a7 expref: 20 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.723435:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e630335aa lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2207744->35762175] (req 2207744->2211839) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27a7 expref: 20 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.723441:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b42a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777604899456:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298642.723468:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3602870c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.724072:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b42a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777604899456:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298642.724080:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298642.724100:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2399:x1866777609409152:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 725us (795us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.724107:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15025 00000100:00100000:0.0:1780298642.724161:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.724179:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609409280 00000100:00100000:1.0:1780298642.724183:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609409280 00000100:00100000:1.0:1780298642.724207:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6028 00000100:00100000:1.0:1780298642.724214:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2400:x1866777609409280:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298642.724328:0:19951:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 282257 -> 282257 00002000:00010000:1.0:1780298642.724331:0:19951:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298642.724332:0:19951:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298642.724334:0:19951:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298642.724335:0:19951:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 296 -> 304 00000100:00100000:1.0:1780298642.724355:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:2400:x1866777609409280:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 144us (198us total) trans 4294969638 rc 0/0 00000100:00100000:1.0:1780298642.724359:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6028 00000100:00100000:0.0:1780298642.725035:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.725058:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.725092:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609409408 00000100:00100000:3.0:1780298642.725097:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609409408 00000100:00100000:2.0:1780298642.725097:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609409536 00000100:00100000:2.0:1780298642.725104:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609409536 00000100:00100000:3.0:1780298642.725150:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9026 00000100:00100000:3.0:1780298642.725159:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2398:x1866777609409408:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298642.725162:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9027 00010000:00010000:3.0:1780298642.725170:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298642.725175:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609409536:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.725178:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e630335b1 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d27ae expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.725182:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.725192:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d1b192d6/0xaf35132e630335b8 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d27b5 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.725202:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255092d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604899584:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.725219:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25491a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604899712:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.725243:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.725245:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.725861:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255092d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604899584:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.725873:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25491a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604899712:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.725880:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e630335b1 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27ae expref: 18 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.725890:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d1b192d6/0xaf35132e630335b8 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27b5 expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.725893:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e630335b1 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27ae expref: 18 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.725922:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3600, rc 0) 00010000:00010000:2.0:1780298642.725925:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d1b192d6/0xaf35132e630335b8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27b5 expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.725937:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341692400, rc 0) 00000100:00100000:3.0:1780298642.725947:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2398:x1866777609409408:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 790us (912us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.725953:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9026 00000100:00100000:2.0:1780298642.725964:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609409536:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 792us (904us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.725973:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9027 00000100:00100000:0.0:1780298642.726770:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.726798:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609409664 00000100:00100000:1.0:1780298642.726802:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609409664 00000100:00100000:0.0:1780298642.726806:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.726825:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609409792 00000100:00100000:1.0:1780298642.726827:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609409792 00000100:00100000:1.0:1780298642.726846:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9028 00000100:00100000:1.0:1780298642.726853:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2397:x1866777609409664:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298642.726857:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9029 00010000:00010000:1.0:1780298642.726863:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298642.726869:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609409792:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.726871:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e630335bf lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d27bc expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.726876:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.726884:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e06fa3fe/0xaf35132e630335c6 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d27c3 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.726894:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b42a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604899840:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.726909:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25491a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604899968:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.726932:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3602870c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.726935:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.727610:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b42a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604899840:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.727618:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25491a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604899968:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.727631:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e630335bf lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27bc expref: 18 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.727636:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e06fa3fe/0xaf35132e630335c6 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27c3 expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.727648:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e630335bf lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27bc expref: 18 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.727651:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e06fa3fe/0xaf35132e630335c6 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27c3 expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.727665:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36c00, rc 0) 00010000:00010000:2.0:1780298642.727668:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341693e00, rc 0) 00000100:00100000:1.0:1780298642.727692:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2397:x1866777609409664:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 840us (922us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.727698:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609409792:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 832us (891us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.727701:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9028 00000100:00100000:2.0:1780298642.727708:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9029 00000100:00100000:0.0:1780298642.730258:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.730301:0:9976:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609410048 00000100:00100000:2.0:1780298642.730308:0:9976:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609410048 00000100:00100000:2.0:1780298642.730339:0:9976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61948 00000100:00100000:2.0:1780298642.730346:0:9976:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+113:29360:x1866777609410048:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:2.0:1780298642.730380:0:9976:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298642.730394:0:9976:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341498c00/0xaf35132e6303356b lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 62 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d276f expref: 113 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.730405:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.730408:0:9976:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298642.730541:0:9976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693400/0xaf35132e630335cd lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 61 type: IBT flags: 0x40000000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.730551:0:9976:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156a00/0xaf35132e63033579 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 61 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2776 expref: 99 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.730564:0:9976:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693400/0xaf35132e630335cd lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 61 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.730580:0:9976:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156a00/0xaf35132e63033579 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 61 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2776 expref: 99 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.730591:0:9976:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156a00/0xaf35132e63033579 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 61 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2776 expref: 99 pid: 6476 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.730602:0:9976:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25491a680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:9976:1866777604900096:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298642.730640:0:9976:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4000 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.731221:0:9976:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25491a680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:9976:1866777604900096:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298642.731234:0:9976:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693400/0xaf35132e630335cd lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 61 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.731374:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.731401:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609410176 00000100:00100000:1.0:1780298642.731404:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609410176 00010000:00010000:1.0:1780298642.731419:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156a00/0xaf35132e63033579 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 61 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2776 expref: 100 pid: 6476 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.731444:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15026 00000100:00100000:1.0:1780298642.731452:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7be300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+100:2400:x1866777609410176:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298642.731461:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298642.731466:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156a00/0xaf35132e63033579 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 62 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2776 expref: 100 pid: 6476 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.731481:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156a00/0xaf35132e63033579 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 62 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2776 expref: 100 pid: 6476 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.731494:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156a00/0xaf35132e63033579 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 62 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2776 expref: 100 pid: 6476 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.731506:0:6462:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298642.731510:0:6462:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693400/0xaf35132e630335cd lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 61 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.731519:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693400/0xaf35132e630335cd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 61 type: IBT flags: 0x40290000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.731524:0:6462:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693400/0xaf35132e630335cd lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 61 type: IBT flags: 0x40290000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.731539:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298642.731554:0:9976:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693400/0xaf35132e630335cd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 60 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.731561:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7be300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+99:2400:x1866777609410176:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 110us (187us total) trans 0 rc 0/0 00010000:00010000:2.0:1780298642.731562:0:9976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693400/0xaf35132e630335cd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 60 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.731572:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15026 00000001:00080000:2.0:1780298642.731609:0:9976:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991024 00000004:00010000:2.0:1780298642.731628:0:9976:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 000000004b8c4872 reply state 000000003dc38b34 transno 4294991025 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693400/0xaf35132e630335cd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 60 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.731636:0:9976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693400/0xaf35132e630335cd lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 60 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.731641:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.731642:0:9976:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1780298642.731645:0:9976:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1780298642.731689:0:9976:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+115:29360:x1866777609410048:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1343us (1435us total) trans 4294991025 rc 0/0 00000100:00100000:2.0:1780298642.731694:0:9976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61948 00000020:00080000:2.0:1780298642.731698:0:9976:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1780298642.732243:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.732265:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609410304 00000100:00100000:3.0:1780298642.732269:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609410304 00000100:00100000:3.0:1780298642.732303:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9030 00000100:00100000:3.0:1780298642.732309:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:29360:x1866777609410304:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.732316:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.732323:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e630335d4 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d27ca expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.732338:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e630335aa lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2207744->35762175] (req 2207744->2211839) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27a7 expref: 23 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.732344:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(119 177) 00010000:00010000:3.0:1780298642.732348:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(120 177) 00010000:00010000:3.0:1780298642.732350:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e630335d4 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27ca expref: 23 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.732362:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e630335aa lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2207744->35762175] (req 2207744->2211839) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27a7 expref: 23 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.732369:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e630335aa lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2207744->35762175] (req 2207744->2211839) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27a7 expref: 23 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.732375:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255092d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604900224:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298642.732405:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.732948:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255092d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604900224:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298642.732958:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e630335d4 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27ca expref: 23 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.732966:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3a00, rc 0) 00000100:00100000:3.0:1780298642.732987:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:29360:x1866777609410304:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 679us (746us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298642.732991:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9030 00000100:00100000:0.0:1780298642.733190:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.733220:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609410432 00000100:00100000:1.0:1780298642.733225:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609410432 00010000:00010000:1.0:1780298642.733239:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e630335aa lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2207744->35762175] (req 2207744->2211839) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27a7 expref: 23 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.733259:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15027 00000100:00100000:1.0:1780298642.733265:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b02050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2399:x1866777609410432:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298642.733273:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298642.733282:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e630335aa lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2207744->35762175] (req 2207744->2211839) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27a7 expref: 23 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.733301:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e630335aa lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2207744->35762175] (req 2207744->2211839) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27a7 expref: 23 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.733313:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e630335aa lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2207744->35762175] (req 2207744->2211839) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27a7 expref: 23 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.733321:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(120 177) 00010000:00010000:1.0:1780298642.733323:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(120 177) 00010000:00010000:1.0:1780298642.733328:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e630335d4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27ca expref: 22 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.733336:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e630335d4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27ca expref: 22 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.733359:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e630335d4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27ca expref: 22 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.733370:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b42a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777604900352:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298642.733406:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3602870c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.734057:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b42a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777604900352:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298642.734065:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298642.734087:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b02050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:2399:x1866777609410432:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 822us (899us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.734095:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15027 00000100:00100000:0.0:1780298642.734198:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.734230:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609410560 00000100:00100000:2.0:1780298642.734235:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609410560 00002000:00010000:2.0:1780298642.734254:0:19928:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (5010679->18446744073709551615) 00002000:00100000:2.0:1780298642.734258:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f7be300 with x1866777609410560 ext(5010679->18446744073709551615) 00002000:00010000:2.0:1780298642.734265:0:19928:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f7be300. 00000100:00100000:2.0:1780298642.734282:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6029 00000100:00100000:2.0:1780298642.734300:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7be300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:2399:x1866777609410560:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:0.0:1780298642.734308:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.734363:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609410688 00000100:00100000:3.0:1780298642.734367:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609410688 00002000:00010000:3.0:1780298642.734380:0:19943:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4194304->18446744073709551615) 00002000:00100000:3.0:1780298642.734384:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255491f80 with x1866777609410688 ext(4194304->18446744073709551615) 00002000:00010000:3.0:1780298642.734389:0:19943:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf255491f80. 00000100:00100000:3.0:1780298642.734412:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6030 00000100:00100000:3.0:1780298642.734419:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255491f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609410688:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:2.0:1780298642.734427:0:19928:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 282257 -> 5010679 00002000:00010000:2.0:1780298642.734430:0:19928:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:2.0:1780298642.734432:0:19928:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:2.0:1780298642.734434:0:19928:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:2.0:1780298642.734437:0:19928:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 304 -> 304 00000100:00100000:2.0:1780298642.734464:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7be300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2399:x1866777609410560:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 174us (265us total) trans 4294969639 rc 0/0 00000100:00100000:2.0:1780298642.734471:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6029 00002000:00010000:2.0:1780298642.734475:0:19928:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (5010679->18446744073709551615) 00002000:00100000:2.0:1780298642.734478:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f7be300 with x1866777609410560 ext(5010679->18446744073709551615) 00002000:00010000:2.0:1780298642.734482:0:19928:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f7be300. 00002000:00010000:3.0:1780298642.734486:0:19943:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 0 -> 4194304 00002000:00010000:3.0:1780298642.734489:0:19943:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:3.0:1780298642.734491:0:19943:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:3.0:1780298642.734493:0:19943:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:3.0:1780298642.734495:0:19943:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:3.0:1780298642.734524:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255491f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609410688:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 106us (219us total) trans 4294969202 rc 0/0 00000100:00100000:3.0:1780298642.734532:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6030 00002000:00010000:3.0:1780298642.734535:0:19943:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4194304->18446744073709551615) 00002000:00100000:3.0:1780298642.734537:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255491f80 with x1866777609410688 ext(4194304->18446744073709551615) 00002000:00010000:3.0:1780298642.734541:0:19943:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf255491f80. 00000100:00100000:0.0:1780298642.741805:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.741878:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609410816 00000100:00100000:3.0:1780298642.741885:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609410816 00000100:00100000:0.0:1780298642.741889:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.741924:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609410944 00000100:00100000:2.0:1780298642.741929:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609410944 00002000:00100000:3.0:1780298642.741940:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (282624->5013503) req@ffff9cf255490e00 x1866777609410816/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 504/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298642.741948:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255490e00 with x1866777609410816 ext(282624->5013503) 00002000:00100000:2.0:1780298642.741949:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (0->282623) req@ffff9cf255493100 x1866777609410944/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00010000:3.0:1780298642.741953:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255490e00 00002000:00100000:2.0:1780298642.741967:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255493100 with x1866777609410944 ext(0->282623) 00002000:00010000:2.0:1780298642.741971:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf255493100 00000100:00100000:3.0:1780298642.741982:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6031 00000100:00100000:3.0:1780298642.741992:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255490e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2398:x1866777609410816:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298642.742004:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6032 00000100:00100000:2.0:1780298642.742016:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255493100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609410944:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298642.743670:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255493100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609410944:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 1657us (1779us total) trans 0 rc 282624/282624 00000100:00100000:1.0:1780298642.743677:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6032 00002000:00100000:1.0:1780298642.743682:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (0->282623) req@ffff9cf255493100 x1866777609410944/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/440 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 282624/282624 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298642.743688:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255493100 with x1866777609410944 ext(0->282623) 00002000:00010000:1.0:1780298642.743691:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf255493100 00000100:00100000:2.0:1780298642.751959:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255490e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2398:x1866777609410816:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 9960us (10150us total) trans 0 rc 3915776/3915776 00000100:00100000:2.0:1780298642.751968:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6031 00002000:00100000:2.0:1780298642.751981:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (282624->5013503) req@ffff9cf255490e00 x1866777609410816/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 504/440 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 3915776/3915776 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.751988:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255490e00 with x1866777609410816 ext(282624->5013503) 00002000:00010000:2.0:1780298642.751994:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255490e00 00000100:00100000:2.0:1780298642.759227:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.759262:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.759284:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609411072 00000100:00100000:1.0:1780298642.759302:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609411072 00000100:00100000:0.0:1780298642.759331:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609411200 00000100:00100000:0.0:1780298642.759337:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609411200 00002000:00100000:1.0:1780298642.759341:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x2c0000401:0x11:0x0] (4194304->5009407) req@ffff9cf25491a680 x1866777609411072/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 504/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298642.759349:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25491a680 with x1866777609411072 ext(4194304->5009407) 00002000:00010000:1.0:1780298642.759353:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf25491a680 00002000:00100000:0.0:1780298642.759365:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (282624->4194303) req@ffff9cf254918a80 x1866777609411200/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00000100:00100000:1.0:1780298642.759377:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6033 00002000:00100000:0.0:1780298642.759378:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf254918a80 with x1866777609411200 ext(282624->4194303) 00002000:00010000:0.0:1780298642.759384:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf254918a80 00000100:00100000:1.0:1780298642.759388:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2397:x1866777609411072:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298642.759410:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6034 00000100:00100000:0.0:1780298642.759420:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254918a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609411200:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298642.761147:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2397:x1866777609411072:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 1760us (1928us total) trans 0 rc 286720/286720 00000100:00100000:1.0:1780298642.761154:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6033 00002000:00100000:1.0:1780298642.761160:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x2c0000401:0x11:0x0] (4194304->5009407) req@ffff9cf25491a680 x1866777609411072/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 504/440 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 286720/286720 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298642.761166:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25491a680 with x1866777609411072 ext(4194304->5009407) 00002000:00010000:1.0:1780298642.761170:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf25491a680 00000100:00100000:2.0:1780298642.762191:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.762229:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609411328 00000100:00100000:3.0:1780298642.762233:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609411328 00002000:00100000:3.0:1780298642.762313:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (5005312->5010678) req@ffff9cf25491b480 x1866777609411328/t0(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 5856/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298642.762322:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25491b480 with x1866777609411328 ext(5005312->5010678) 00002000:00010000:3.0:1780298642.762327:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf25491b480 00000100:00100000:3.0:1780298642.762350:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6035 00000100:00100000:3.0:1780298642.762359:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2399:x1866777609411328:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:3.0:1780298642.762809:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+25:2399:x1866777609411328:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 451us (620us total) trans 4294969640 rc 0/0 00000100:00100000:3.0:1780298642.762817:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6035 00002000:00100000:3.0:1780298642.762825:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (5005312->5010678) req@ffff9cf25491b480 x1866777609411328/t4294969640(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 5856/448 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298642.762834:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25491b480 with x1866777609411328 ext(5005312->5010678) 00002000:00010000:3.0:1780298642.762838:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf25491b480 00000100:00100000:2.0:1780298642.763612:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.763647:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609411456 00000100:00100000:3.0:1780298642.763651:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609411456 00000100:00100000:3.0:1780298642.763682:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9031 00000100:00100000:3.0:1780298642.763692:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+25:2400:x1866777609411456:12345-192.168.203.25@tcp:16:fsx.0 00000001:00080000:1.0:1780298642.764386:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969640 is committed 00000001:00080000:1.0:1780298642.764404:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969635 is committed 00000100:00100000:3.0:1780298642.764519:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609411456:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 829us (908us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298642.764527:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9031 00000100:00100000:2.0:1780298642.765401:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.765454:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609411584 00000100:00100000:1.0:1780298642.765460:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609411584 00000100:00100000:1.0:1780298642.765505:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61949 00000100:00100000:1.0:1780298642.765514:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+115:29360:x1866777609411584:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.765531:0:6477:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.765551:0:6477:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36c00/0xaf35132e630335db lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 61 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d27d1 expref: -99 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.765647:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36a00/0xaf35132e630335e2 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 62 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.765663:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36a00/0xaf35132e630335e2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 62 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.765676:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36a00/0xaf35132e630335e2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 62 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1780298642.765701:0:6477:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 9204983 00000004:00200000:1.0:1780298642.765705:0:6477:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1780298642.765722:0:6477:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36a00/0xaf35132e630335e2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 62 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.765746:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36c00/0xaf35132e630335db lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 62 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27d1 expref: 117 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.765763:0:6477:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36a00/0xaf35132e630335e2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 61 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27d1 expref: 116 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.765777:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298642.765780:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1780298642.765784:0:6477:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36a00, rc 0) 00000100:00100000:1.0:1780298642.765816:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+116:29360:x1866777609411584:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 304us (415us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.765823:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61949 00000100:00100000:2.0:1780298642.767541:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.767595:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609411712 00000100:00100000:0.0:1780298642.767601:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609411712 00000100:00100000:0.0:1780298642.767633:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9032 00000100:00100000:0.0:1780298642.767639:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254919880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:29360:x1866777609411712:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.767649:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.767661:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e630335e9 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d27d8 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.767683:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e630335d4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27ca expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.767689:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(120 177) 00010000:00010000:0.0:1780298642.767698:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(121 177) 00010000:00010000:0.0:1780298642.767701:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e630335e9 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2736128->2740223] (req 2736128->2740223) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27d8 expref: 7 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.767729:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e630335d4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27ca expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.767739:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e630335d4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27ca expref: 8 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.767750:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f747800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604900480:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298642.767808:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417540 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.768810:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f747800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604900480:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298642.768827:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e630335e9 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2736128->2740223] (req 2736128->2740223) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27d8 expref: 7 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.768838:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2600, rc 0) 00000100:00100000:0.0:1780298642.768863:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254919880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609411712:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1225us (1328us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.768870:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9032 00000100:00100000:0.0:1780298642.770501:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254918a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609411200:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 11082us (11237us total) trans 0 rc 3911680/3911680 00000100:00100000:0.0:1780298642.770519:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6034 00002000:00100000:0.0:1780298642.770532:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (282624->4194303) req@ffff9cf254918a80 x1866777609411200/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/440 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 3911680/3911680 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.770542:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf254918a80 with x1866777609411200 ext(282624->4194303) 00002000:00010000:0.0:1780298642.770547:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf254918a80 00000100:00100000:3.0:1780298642.772391:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.772444:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609411840 00000100:00100000:2.0:1780298642.772451:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609411840 00010000:00010000:2.0:1780298642.772477:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e630335d4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27ca expref: 9 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.772505:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15028 00000100:00100000:2.0:1780298642.772512:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255092d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609411840:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298642.772523:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:2.0:1780298642.772537:0:6461:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 304 -> 320 00010000:00010000:2.0:1780298642.772541:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e630335d4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27ca expref: 9 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.772551:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e630335d4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27ca expref: 9 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.772573:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e630335d4 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27ca expref: 9 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.772585:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(121 177) 00010000:00010000:2.0:1780298642.772588:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(121 177) 00010000:00010000:2.0:1780298642.772594:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e630335e9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2736128->36290559] (req 2736128->2740223) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27d8 expref: 6 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.772600:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e630335e9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2736128->36290559] (req 2736128->2740223) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27d8 expref: 6 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.772625:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e630335e9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2736128->36290559] (req 2736128->2740223) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27d8 expref: 6 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.772635:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254918000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777604900608:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298642.772685:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.773459:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254918000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777604900608:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298642.773474:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298642.773504:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255092d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2400:x1866777609411840:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 992us (1118us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.773510:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15028 00000100:00100000:3.0:1780298642.773569:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.773599:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609411968 00000100:00100000:0.0:1780298642.773602:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609411968 00000100:00100000:0.0:1780298642.773634:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6036 00000100:00100000:0.0:1780298642.773638:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255093100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609411968:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298642.773796:0:19949:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 5010679 -> 5010679 00002000:00010000:0.0:1780298642.773803:0:19949:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:0.0:1780298642.773805:0:19949:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:0.0:1780298642.773806:0:19949:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:0.0:1780298642.773807:0:19949:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 320 -> 328 00000100:00100000:0.0:1780298642.773827:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255093100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609411968:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 189us (260us total) trans 4294969641 rc 0/0 00000100:00100000:0.0:1780298642.773834:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6036 00000100:00100000:2.0:1780298642.774546:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.774583:0:7411:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609412096 00000100:00100000:1.0:1780298642.774587:0:7411:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609412096 00000100:00100000:1.0:1780298642.774615:0:7411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61950 00000100:00100000:1.0:1780298642.774621:0:7411:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254918e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+99:29360:x1866777609412096:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.774630:0:7411:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.774643:0:7411:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36c00/0xaf35132e630335f0 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 62 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d27df expref: -99 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.774706:0:7411:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36e00/0xaf35132e630335f7 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 63 type: IBT flags: 0x40000000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.774717:0:7411:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36e00/0xaf35132e630335f7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 63 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.774725:0:7411:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36e00/0xaf35132e630335f7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 63 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1780298642.774741:0:7411:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 9204983 00000004:00200000:1.0:1780298642.774743:0:7411:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1780298642.774754:0:7411:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36e00/0xaf35132e630335f7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 63 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.774771:0:7411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36c00/0xaf35132e630335f0 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 63 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27df expref: 101 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.774780:0:7411:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36e00/0xaf35132e630335f7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 62 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27df expref: 100 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.774785:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298642.774787:0:7411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1780298642.774789:0:7411:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36e00, rc 0) 00000100:00100000:1.0:1780298642.774811:0:7411:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254918e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+100:29360:x1866777609412096:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 192us (267us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.774816:0:7411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61950 00000100:00100000:2.0:1780298642.775587:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.775626:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609412224 00000100:00100000:0.0:1780298642.775630:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609412224 00000100:00100000:0.0:1780298642.775651:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9033 00000100:00100000:0.0:1780298642.775658:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609412224:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.775666:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.775676:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e630335fe lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d27e6 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.775714:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f747800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604900736:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.775742:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.775755:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417540 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.775776:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609412352 00000100:00100000:1.0:1780298642.775780:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609412352 00000100:00100000:1.0:1780298642.775810:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9034 00000100:00100000:1.0:1780298642.775817:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609412352:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.775824:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.775832:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63033605 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d27ed expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.775855:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b42a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604900864:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.775880:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.776365:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f747800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604900736:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.776379:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e630335fe lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27e6 expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.776390:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e630335fe lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27e6 expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.776400:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2a00, rc 0) 00000100:00100000:0.0:1780298642.776427:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609412224:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 769us (839us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.776435:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9033 00000100:00100000:1.0:1780298642.776619:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b42a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604900864:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.776633:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63033605 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27ed expref: 7 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.776646:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63033605 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27ed expref: 7 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.776653:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37000, rc 0) 00000100:00100000:1.0:1780298642.776674:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609412352:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 860us (934us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.776679:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9034 00000100:00100000:3.0:1780298642.777522:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.777542:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.777566:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609412480 00000100:00100000:2.0:1780298642.777571:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609412480 00000100:00100000:1.0:1780298642.777583:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609412608 00000100:00100000:1.0:1780298642.777589:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609412608 00000100:00100000:2.0:1780298642.777609:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9035 00000100:00100000:2.0:1780298642.777619:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255093b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609412480:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.777627:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298642.777629:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9036 00010000:00010000:2.0:1780298642.777637:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002ee949fd/0xaf35132e6303360c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d27f4 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.777642:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b6bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609412608:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.777654:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.777664:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a10929f0/0xaf35132e63033613 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d27fb expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.777668:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25491ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604900992:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.777698:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b42a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604901120:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.777715:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4900 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.777725:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.778391:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25491ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604900992:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.778407:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b42a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604901120:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.778418:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002ee949fd/0xaf35132e6303360c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27f4 expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.778423:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a10929f0/0xaf35132e63033613 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27fb expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.778446:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a10929f0/0xaf35132e63033613 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27fb expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.778447:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002ee949fd/0xaf35132e6303360c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27f4 expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.778475:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bafc00, rc 0) 00010000:00010000:1.0:1780298642.778475:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37200, rc 0) 00000100:00100000:1.0:1780298642.778506:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b6bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609412608:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 869us (963us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.778511:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255093b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609412480:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 892us (986us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.778516:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9036 00000100:00100000:3.0:1780298642.778527:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9035 00000100:00100000:3.0:1780298642.779523:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.779558:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609412736 00000100:00100000:0.0:1780298642.779562:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609412736 00000100:00100000:0.0:1780298642.779599:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9037 00000100:00100000:0.0:1780298642.779608:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b68700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2398:x1866777609412736:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.779616:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.779625:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a269a887/0xaf35132e6303361a lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2802 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.779652:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f747800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604901248:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.779692:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417540 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.780371:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f747800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604901248:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.780383:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a269a887/0xaf35132e6303361a lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2802 expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.780393:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a269a887/0xaf35132e6303361a lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2802 expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.780400:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3e00, rc 0) 00000100:00100000:0.0:1780298642.780423:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b68700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2398:x1866777609412736:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 818us (902us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.780427:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9037 00000100:00100000:2.0:1780298642.781232:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.781257:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609412864 00000100:00100000:1.0:1780298642.781261:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609412864 00000100:00100000:1.0:1780298642.781284:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9038 00000100:00100000:1.0:1780298642.781302:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254919180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:29360:x1866777609412864:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.781309:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.781317:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bb478d6d/0xaf35132e63033621 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2809 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.781330:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(121 177) 00010000:00010000:1.0:1780298642.781333:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(121 177) 00010000:00010000:1.0:1780298642.781341:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000bb478d6d/0xaf35132e63033621 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [962560->2736127] (req 962560->974847) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2809 expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.781351:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bb478d6d/0xaf35132e63033621 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [962560->2736127] (req 962560->974847) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2809 expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.781357:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341498c00, rc 0) 00000100:00100000:1.0:1780298642.781379:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254919180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609412864:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 90us (148us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.781383:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9038 00000100:00100000:3.0:1780298642.781960:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.782000:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609412992 00000100:00100000:2.0:1780298642.782005:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609412992 00000100:00100000:2.0:1780298642.782034:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6037 00000100:00100000:2.0:1780298642.782040:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b69880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609412992:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298642.782206:0:8364:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 5010679 -> 5010679 00002000:00010000:2.0:1780298642.782209:0:8364:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:2.0:1780298642.782211:0:8364:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:2.0:1780298642.782212:0:8364:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:2.0:1780298642.782214:0:8364:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 328 -> 352 00000100:00100000:2.0:1780298642.782237:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b69880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609412992:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 197us (279us total) trans 4294969642 rc 0/0 00000100:00100000:2.0:1780298642.782243:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6037 00000100:00100000:2.0:1780298642.782987:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.783020:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609413120 00000100:00100000:3.0:1780298642.783025:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609413120 00000100:00100000:3.0:1780298642.783056:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9039 00000100:00100000:3.0:1780298642.783065:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609413120:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.783073:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.783082:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a6833b51/0xaf35132e63033628 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2810 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.783116:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf348005c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604901376:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.783181:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.783717:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348005c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604901376:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.783732:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a6833b51/0xaf35132e63033628 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2810 expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.783742:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a6833b51/0xaf35132e63033628 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2810 expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.783748:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf36d5a8c00, rc 0) 00000100:00100000:1.0:1780298642.783771:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609413120:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 709us (786us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.783775:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9039 00000100:00100000:2.0:1780298642.784641:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.784673:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609413248 00000100:00100000:3.0:1780298642.784678:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609413248 00000100:00100000:3.0:1780298642.784709:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9040 00000100:00100000:3.0:1780298642.784717:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609413248:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.784725:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.784733:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000063c74321/0xaf35132e6303362f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2817 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.784762:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf348007b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604901504:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.784804:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.785403:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348007b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604901504:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.785419:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000063c74321/0xaf35132e6303362f lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2817 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.785433:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000063c74321/0xaf35132e6303362f lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2817 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.785442:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf36d5a9000, rc 0) 00000100:00100000:3.0:1780298642.785470:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609413248:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 754us (829us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.785478:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9040 00000100:00100000:2.0:1780298642.786179:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.786210:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609413376 00000100:00100000:0.0:1780298642.786215:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609413376 00000100:00100000:0.0:1780298642.786245:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9041 00000100:00100000:0.0:1780298642.786252:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254918000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:29360:x1866777609413376:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.786260:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.786268:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e63033636 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d281e expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.786302:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(121 177) 00010000:00010000:0.0:1780298642.786307:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(121 177) 00010000:00010000:0.0:1780298642.786314:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e63033636 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [405504->962559] (req 405504->413695) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d281e expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.786328:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e63033636 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [405504->962559] (req 405504->413695) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d281e expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.786336:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2a00, rc 0) 00000100:00100000:0.0:1780298642.786364:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254918000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:29360:x1866777609413376:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 113us (186us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.786371:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9041 00000100:00100000:2.0:1780298642.786910:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.786939:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609413504 00000100:00100000:3.0:1780298642.786944:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609413504 00000100:00100000:3.0:1780298642.786976:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6038 00000100:00100000:3.0:1780298642.786983:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609413504:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:3.0:1780298642.787110:0:19928:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 5010679 -> 5010679 00002000:00010000:3.0:1780298642.787114:0:19928:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:3.0:1780298642.787116:0:19928:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:3.0:1780298642.787118:0:19928:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:3.0:1780298642.787120:0:19928:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 352 -> 368 00000100:00100000:3.0:1780298642.787169:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609413504:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 186us (258us total) trans 4294969643 rc 0/0 00000100:00100000:3.0:1780298642.787179:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6038 00000100:00100000:2.0:1780298642.787971:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.788015:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609413632 00000100:00100000:1.0:1780298642.788020:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609413632 00000100:00100000:1.0:1780298642.788053:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9042 00000100:00100000:1.0:1780298642.788061:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f734700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609413632:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.788070:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.788082:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000ac4408d/0xaf35132e6303363d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2825 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.788110:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b42a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604901632:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.788168:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.788881:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b42a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604901632:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.788899:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000ac4408d/0xaf35132e6303363d lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2825 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.788919:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000ac4408d/0xaf35132e6303363d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2825 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.788929:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341499200, rc 0) 00000100:00100000:1.0:1780298642.788956:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f734700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609413632:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 897us (985us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.788960:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9042 00000100:00100000:2.0:1780298642.789794:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.789822:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609413760 00000100:00100000:1.0:1780298642.789826:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609413760 00000100:00100000:1.0:1780298642.789848:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9043 00000100:00100000:1.0:1780298642.789854:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f734e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609413760:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.789860:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.789866:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e8a7e8eb/0xaf35132e63033644 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d282c expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.789885:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b42a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604901760:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.789912:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.790533:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b42a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604901760:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.790544:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e8a7e8eb/0xaf35132e63033644 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d282c expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.790552:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e8a7e8eb/0xaf35132e63033644 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d282c expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.790561:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341499e00, rc 0) 00000100:00100000:1.0:1780298642.790583:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f734e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609413760:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 731us (791us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.790588:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9043 00000100:00100000:2.0:1780298642.791410:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.791453:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609413888 00000100:00100000:3.0:1780298642.791457:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609413888 00000100:00100000:3.0:1780298642.791484:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9044 00000100:00100000:2.0:1780298642.791491:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.791494:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f737800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609413888:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.791502:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.791510:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005dffef48/0xaf35132e6303364b lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2833 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.791515:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609414016 00000100:00100000:1.0:1780298642.791521:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609414016 00000100:00100000:1.0:1780298642.791550:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9045 00000100:00100000:1.0:1780298642.791557:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f735f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609414016:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298642.791562:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf348007b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604901888:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.791563:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.791573:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008be5bdd7/0xaf35132e63033652 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d283a expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.791605:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.791605:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b42a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604902016:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.791636:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.792235:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348007b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604901888:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.792253:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005dffef48/0xaf35132e6303364b lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2833 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.792263:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005dffef48/0xaf35132e6303364b lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2833 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.792270:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf36d5a8000, rc 0) 00000100:00100000:3.0:1780298642.792304:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f737800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609413888:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 814us (894us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.792317:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b42a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604902016:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.792333:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008be5bdd7/0xaf35132e63033652 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d283a expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.792344:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008be5bdd7/0xaf35132e63033652 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d283a expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.792349:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9044 00010000:00010000:1.0:1780298642.792358:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341499c00, rc 0) 00000100:00100000:1.0:1780298642.792387:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f735f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609414016:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 829us (897us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.792399:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9045 00000100:00100000:2.0:1780298642.793208:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.793226:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.793241:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609414144 00000100:00100000:1.0:1780298642.793246:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609414144 00000100:00100000:1.0:1780298642.793277:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609414272 00000100:00100000:1.0:1780298642.793281:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609414272 00000100:00100000:0.0:1780298642.793286:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9046 00000100:00100000:0.0:1780298642.793310:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f735c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609414144:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298642.793313:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9047 00010000:00010000:0.0:1780298642.793322:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298642.793325:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f736d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609414272:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.793332:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.793332:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a269a887/0xaf35132e63033659 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2841 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.793341:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000019d620bb/0xaf35132e63033660 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2848 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.793374:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f747800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604902144:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.793378:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf348007b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604902272:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.793429:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.793430:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417540 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.794072:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f747800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604902144:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.794084:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348007b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604902272:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.794088:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a269a887/0xaf35132e63033659 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2841 expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.794101:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000019d620bb/0xaf35132e63033660 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2848 expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.794105:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a269a887/0xaf35132e63033659 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2841 expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.794144:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000019d620bb/0xaf35132e63033660 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2848 expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.794144:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3e00, rc 0) 00010000:00010000:3.0:1780298642.794155:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf36d5a8200, rc 0) 00000100:00100000:0.0:1780298642.794180:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f735c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609414144:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 874us (970us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.794188:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9046 00000100:00100000:3.0:1780298642.794201:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f736d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609414272:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 878us (972us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.794208:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9047 00000100:00100000:2.0:1780298642.795063:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.795082:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.795105:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609414400 00000100:00100000:3.0:1780298642.795107:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609414528 00000100:00100000:0.0:1780298642.795109:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609414400 00000100:00100000:3.0:1780298642.795112:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609414528 00000100:00100000:0.0:1780298642.795167:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9048 00000100:00100000:3.0:1780298642.795175:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9049 00000100:00100000:0.0:1780298642.795179:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f735880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609414400:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.795186:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298642.795188:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f737b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609414528:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.795194:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e63033667 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d284f expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.795195:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.795204:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b6050429/0xaf35132e6303366e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2856 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.795219:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f747800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604902400:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.795253:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417540 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.795266:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b42a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604902528:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.795313:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.795856:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f747800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604902400:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.795869:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e63033667 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d284f expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.795880:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e63033667 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d284f expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.795891:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3800, rc 0) 00000100:00100000:0.0:1780298642.795915:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f735880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609414400:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 741us (852us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.795921:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b42a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604902528:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.795922:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9048 00010000:00010000:1.0:1780298642.795947:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b6050429/0xaf35132e6303366e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2856 expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.795962:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b6050429/0xaf35132e6303366e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2856 expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.795972:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf36d5a9a00, rc 0) 00000100:00100000:1.0:1780298642.796003:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f737b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609414528:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 818us (917us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.796011:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9049 00000100:00100000:2.0:1780298642.796807:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.796837:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609414656 00000100:00100000:0.0:1780298642.796840:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609414656 00000100:00100000:2.0:1780298642.796853:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.796863:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609414784 00000100:00100000:0.0:1780298642.796866:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609414784 00000100:00100000:3.0:1780298642.796883:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9050 00000100:00100000:3.0:1780298642.796894:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f736300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609414656:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298642.796894:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9051 00010000:00010000:3.0:1780298642.796903:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298642.796910:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f736680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609414784:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.796918:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.796926:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5a492f0/0xaf35132e63033675 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2864 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.796950:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f747800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604902656:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.796951:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000034be4f60/0xaf35132e6303367c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d285d expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.796988:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf348007b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604902784:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.796992:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417540 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.797016:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.797687:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f747800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604902656:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.797700:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a5a492f0/0xaf35132e63033675 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2864 expref: 7 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.797703:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348007b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604902784:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.797718:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5a492f0/0xaf35132e63033675 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2864 expref: 7 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.797724:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000034be4f60/0xaf35132e6303367c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d285d expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.797726:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3600, rc 0) 00010000:00010000:3.0:1780298642.797739:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000034be4f60/0xaf35132e6303367c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d285d expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.797746:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf36d5a8400, rc 0) 00000100:00100000:0.0:1780298642.797754:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f736680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609414784:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 850us (902us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.797760:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9051 00000100:00100000:3.0:1780298642.797768:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f736300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609414656:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 877us (961us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.797774:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9050 00000100:00100000:2.0:1780298642.798846:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.798878:0:6476:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609415040 00000100:00100000:1.0:1780298642.798882:0:6476:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609415040 00000100:00100000:1.0:1780298642.798909:0:6476:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61951 00000100:00100000:1.0:1780298642.798915:0:6476:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f734380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+100:29360:x1866777609415040:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:1.0:1780298642.798948:0:6476:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298642.798962:0:6476:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36e00/0xaf35132e630335f7 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 63 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27df expref: 100 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.798975:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298642.798977:0:6476:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298642.799102:0:6476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37200/0xaf35132e63033683 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 62 type: IBT flags: 0x40000000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.799112:0:6476:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36a00/0xaf35132e630335e2 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 62 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27d1 expref: 115 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.799149:0:6476:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37200/0xaf35132e63033683 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 62 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.799167:0:6476:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36a00/0xaf35132e630335e2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 62 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27d1 expref: 115 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.799178:0:6476:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36a00/0xaf35132e630335e2 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 62 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27d1 expref: 115 pid: 6477 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.799187:0:6476:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b42a00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6476:1866777604902912:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298642.799218:0:6476:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.799745:0:6476:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b42a00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6476:1866777604902912:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298642.799755:0:6476:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37200/0xaf35132e63033683 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 62 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.799892:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.799928:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609415168 00000100:00100000:0.0:1780298642.799932:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609415168 00010000:00010000:0.0:1780298642.799949:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36a00/0xaf35132e630335e2 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 62 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27d1 expref: 116 pid: 6477 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.799975:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15029 00000100:00100000:0.0:1780298642.799981:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f734a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+116:2399:x1866777609415168:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298642.799988:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298642.799992:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36a00/0xaf35132e630335e2 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 63 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27d1 expref: 116 pid: 6477 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.800000:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36a00/0xaf35132e630335e2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 63 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27d1 expref: 116 pid: 6477 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.800010:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36a00/0xaf35132e630335e2 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 63 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27d1 expref: 116 pid: 6477 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.800019:0:20253:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298642.800023:0:20253:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37200/0xaf35132e63033683 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 62 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.800034:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37200/0xaf35132e63033683 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 62 type: IBT flags: 0x40290000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.800042:0:20253:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37200/0xaf35132e63033683 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 62 type: IBT flags: 0x40290000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.800065:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298642.800085:0:6476:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37200/0xaf35132e63033683 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 61 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.800091:0:6476:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37200/0xaf35132e63033683 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 61 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.800095:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f734a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+115:2399:x1866777609415168:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 113us (201us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.800103:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15029 00000001:00080000:1.0:1780298642.800160:0:6476:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991026 00000004:00010000:1.0:1780298642.800186:0:6476:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000c233160e reply state 00000000b1121406 transno 4294991027 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37200/0xaf35132e63033683 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 61 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.800197:0:6476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37200/0xaf35132e63033683 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 61 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.800204:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298642.800206:0:6476:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1780298642.800209:0:6476:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1780298642.800264:0:6476:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f734380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+102:29360:x1866777609415040:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1348us (1417us total) trans 4294991027 rc 0/0 00000100:00100000:1.0:1780298642.800272:0:6476:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61951 00000020:00080000:1.0:1780298642.800276:0:6476:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1780298642.800895:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.800928:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609415296 00000100:00100000:1.0:1780298642.800932:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609415296 00000100:00100000:1.0:1780298642.800954:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9052 00000100:00100000:1.0:1780298642.800959:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f736a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:29360:x1866777609415296:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.800965:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.800971:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e6303368a lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d286b expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.800984:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e63033636 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [405504->962559] (req 405504->413695) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d281e expref: 11 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.800990:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000bb478d6d/0xaf35132e63033621 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [962560->2736127] (req 962560->974847) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2809 expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.800998:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e630335e9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2736128->36290559] (req 2736128->2740223) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27d8 expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.801002:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 3 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(121 177) 00010000:00010000:1.0:1780298642.801005:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(124 177) 00010000:00010000:1.0:1780298642.801008:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e6303368a lrc: 6/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d286b expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.801029:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e630335e9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2736128->36290559] (req 2736128->2740223) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27d8 expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.801035:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e630335e9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2736128->36290559] (req 2736128->2740223) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27d8 expref: 11 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.801041:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b42a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604903040:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298642.801076:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000bb478d6d/0xaf35132e63033621 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [962560->2736127] (req 962560->974847) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2809 expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.801083:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000bb478d6d/0xaf35132e63033621 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [962560->2736127] (req 962560->974847) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2809 expref: 9 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.801089:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b40380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604903168:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298642.801111:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e63033636 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [405504->962559] (req 405504->413695) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d281e expref: 11 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.801117:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e63033636 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [405504->962559] (req 405504->413695) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d281e expref: 11 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.801149:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604903296:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298642.801175:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.801714:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b42a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604903040:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298642.801722:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b40380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604903168:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298642.801748:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604903296:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298642.801766:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e6303368a lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d286b expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.801778:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37000, rc 0) 00000100:00100000:1.0:1780298642.801807:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f736a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:29360:x1866777609415296:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 847us (911us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.801814:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9052 00000100:00100000:2.0:1780298642.801863:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.801901:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609415424 00000100:00100000:0.0:1780298642.801907:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609415424 00010000:00010000:0.0:1780298642.801925:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e630335e9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2736128->36290559] (req 2736128->2740223) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27d8 expref: 11 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.801949:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15030 00000100:00100000:0.0:1780298642.801955:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f734000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609415424:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298642.801961:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298642.801969:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e630335e9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [2736128->36290559] (req 2736128->2740223) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27d8 expref: 11 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.801976:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e630335e9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [2736128->36290559] (req 2736128->2740223) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27d8 expref: 11 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.801990:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e630335e9 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [2736128->36290559] (req 2736128->2740223) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d27d8 expref: 11 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.802001:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:0.0:1780298642.802002:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298642.802032:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f734000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609415424:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 76us (167us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.802041:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15030 00000100:00100000:0.0:1780298642.802057:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609415552 00000100:00100000:0.0:1780298642.802061:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609415552 00010000:00010000:0.0:1780298642.802076:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e63033636 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [405504->962559] (req 405504->413695) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d281e expref: 10 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.802102:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15031 00000100:00100000:0.0:1780298642.802110:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348007b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609415552:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298642.802118:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298642.802146:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e63033636 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [405504->962559] (req 405504->413695) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d281e expref: 10 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.802153:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e63033636 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [405504->962559] (req 405504->413695) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d281e expref: 10 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.802163:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004e9b43ec/0xaf35132e63033636 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [405504->962559] (req 405504->413695) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d281e expref: 10 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.802172:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298642.802179:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.802199:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609415680 00000100:00100000:1.0:1780298642.802204:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609415680 00000100:00100000:0.0:1780298642.802204:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348007b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609415552:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 92us (201us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.802214:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15031 00010000:00010000:1.0:1780298642.802218:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000bb478d6d/0xaf35132e63033621 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [962560->2736127] (req 962560->974847) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2809 expref: 10 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.802234:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15032 00000100:00100000:0.0:1780298642.802243:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348005180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609415680:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298642.802253:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298642.802263:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000bb478d6d/0xaf35132e63033621 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [962560->2736127] (req 962560->974847) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2809 expref: 10 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.802281:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000bb478d6d/0xaf35132e63033621 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [962560->2736127] (req 962560->974847) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2809 expref: 10 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.802304:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bb478d6d/0xaf35132e63033621 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [962560->2736127] (req 962560->974847) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2809 expref: 10 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.802315:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(124 177) 00010000:00010000:0.0:1780298642.802319:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(124 177) 00010000:00010000:0.0:1780298642.802324:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e6303368a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d286b expref: 8 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.802333:0:6463:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e6303368a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d286b expref: 8 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.802355:0:6463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e6303368a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d286b expref: 8 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.802366:0:6463:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f747800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6463:1866777604903424:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298642.802404:0:6463:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417540 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.802917:0:6463:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f747800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6463:1866777604903424:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298642.802927:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298642.802949:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348005180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609415680:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 708us (775us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.802957:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15032 00000100:00100000:2.0:1780298642.803024:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.803062:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609415808 00000100:00100000:1.0:1780298642.803066:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609415808 00000100:00100000:1.0:1780298642.803090:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9053 00000100:00100000:1.0:1780298642.803096:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f737480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:29360:x1866777609415808:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.803102:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.803109:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078c7f460/0xaf35132e63033691 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2872 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.803120:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e6303355d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2768 expref: 11 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.803160:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(88 130) 00010000:00010000:1.0:1780298642.803164:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(89 130) 00010000:00010000:1.0:1780298642.803169:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000078c7f460/0xaf35132e63033691 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2872 expref: 7 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.803188:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e6303355d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2768 expref: 11 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.803200:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e6303355d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2768 expref: 11 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.803210:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604903552:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298642.803242:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.803715:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604903552:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298642.803727:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000078c7f460/0xaf35132e63033691 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2872 expref: 7 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.803741:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36c00, rc 0) 00000100:00100000:1.0:1780298642.803762:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f737480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609415808:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 666us (739us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.803766:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9053 00000100:00100000:2.0:1780298642.806686:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.806716:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609415936 00000100:00100000:1.0:1780298642.806720:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609415936 00010000:00010000:1.0:1780298642.806732:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e6303355d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2768 expref: 12 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.806750:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15033 00000100:00100000:1.0:1780298642.806755:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b01880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609415936:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298642.806761:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298642.806768:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e6303355d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2768 expref: 12 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.806775:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e6303355d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2768 expref: 12 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.806785:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e6303355d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2768 expref: 12 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.806793:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(89 130) 00010000:00010000:1.0:1780298642.806795:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(89 130) 00010000:00010000:1.0:1780298642.806799:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000078c7f460/0xaf35132e63033691 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2872 expref: 6 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.806804:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000078c7f460/0xaf35132e63033691 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2872 expref: 6 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.806817:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000078c7f460/0xaf35132e63033691 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2872 expref: 6 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.806823:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:8374:1866777604903680:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298642.806849:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.807402:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:8374:1866777604903680:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298642.807409:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298642.807429:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b01880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609415936:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 674us (745us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.807433:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15033 00000100:00100000:3.0:1780298642.807572:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.807600:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.807619:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609416064 00000100:00100000:1.0:1780298642.807623:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609416064 00000100:00100000:0.0:1780298642.807631:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609416192 00002000:00010000:1.0:1780298642.807637:0:8363:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (5831865->18446744073709551615) 00002000:00100000:1.0:1780298642.807641:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf348004380 with x1866777609416064 ext(5831865->18446744073709551615) 00000100:00100000:0.0:1780298642.807641:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609416192 00002000:00010000:1.0:1780298642.807648:0:8363:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf348004380. 00002000:00010000:0.0:1780298642.807659:0:19943:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4194304->18446744073709551615) 00002000:00100000:0.0:1780298642.807664:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf348005880 with x1866777609416192 ext(4194304->18446744073709551615) 00000100:00100000:1.0:1780298642.807666:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6039 00002000:00010000:0.0:1780298642.807673:0:19943:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf348005880. 00000100:00100000:1.0:1780298642.807676:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348004380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609416064:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:2.0:1780298642.807701:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6040 00000100:00100000:2.0:1780298642.807710:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348005880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609416192:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:2.0:1780298642.807779:0:19950:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:2.0:1780298642.807782:0:19950:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:2.0:1780298642.807785:0:19950:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:2.0:1780298642.807786:0:19950:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:2.0:1780298642.807788:0:19950:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 0 -> 0 00002000:00010000:1.0:1780298642.807799:0:8363:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 5010679 -> 5831865 00002000:00010000:1.0:1780298642.807802:0:8363:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298642.807804:0:8363:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298642.807806:0:8363:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298642.807807:0:8363:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 368 -> 368 00000100:00100000:2.0:1780298642.807814:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348005880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609416192:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 106us (214us total) trans 4294969203 rc 0/0 00000100:00100000:2.0:1780298642.807820:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6040 00002000:00010000:2.0:1780298642.807823:0:19950:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4194304->18446744073709551615) 00002000:00100000:2.0:1780298642.807825:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf348005880 with x1866777609416192 ext(4194304->18446744073709551615) 00002000:00010000:2.0:1780298642.807830:0:19950:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf348005880. 00000100:00100000:1.0:1780298642.807830:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348004380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609416064:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 154us (258us total) trans 4294969644 rc 0/0 00000100:00100000:1.0:1780298642.807838:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6039 00002000:00010000:1.0:1780298642.807841:0:8363:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (5831865->18446744073709551615) 00002000:00100000:1.0:1780298642.807843:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf348004380 with x1866777609416064 ext(5831865->18446744073709551615) 00002000:00010000:1.0:1780298642.807848:0:8363:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf348004380. 00000100:00100000:2.0:1780298642.808569:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.808605:0:9976:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609416320 00000100:00100000:0.0:1780298642.808610:0:9976:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609416320 00000100:00100000:0.0:1780298642.808642:0:9976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61952 00000100:00100000:0.0:1780298642.808651:0:9976:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b01180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+115:29360:x1866777609416320:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.808659:0:9976:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.808667:0:9976:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3600/0xaf35132e63033698 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 62 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2879 expref: -99 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.808713:0:9976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3800/0xaf35132e6303369f lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 63 type: IBT flags: 0x40000000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.808721:0:9976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3800/0xaf35132e6303369f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 63 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.808726:0:9976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3800/0xaf35132e6303369f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 63 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1780298642.808739:0:9976:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 10026169 00000004:00200000:0.0:1780298642.808742:0:9976:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1780298642.808752:0:9976:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3800/0xaf35132e6303369f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 63 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.808765:0:9976:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3600/0xaf35132e63033698 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 63 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2879 expref: 117 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.808777:0:9976:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3800/0xaf35132e6303369f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 62 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2879 expref: 116 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.808782:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298642.808784:0:9976:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1780298642.808785:0:9976:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3800, rc 0) 00000100:00100000:0.0:1780298642.808807:0:9976:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b01180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+116:29360:x1866777609416320:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 159us (238us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.808811:0:9976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61952 00000100:00100000:3.0:1780298642.809608:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.809635:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.809651:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609416576 00000100:00100000:1.0:1780298642.809655:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609416576 00000100:00100000:0.0:1780298642.809677:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609416448 00000100:00100000:1.0:1780298642.809680:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9054 00000100:00100000:0.0:1780298642.809682:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609416448 00000100:00100000:1.0:1780298642.809687:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348004e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609416576:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.809693:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.809700:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009cb6f854/0xaf35132e630336a6 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2887 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.809717:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9055 00000100:00100000:1.0:1780298642.809723:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604903808:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.809728:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e583050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609416448:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.809735:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.809743:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a269a887/0xaf35132e630336ad lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2880 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.809759:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.809767:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f747800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604903936:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.809791:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417540 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.810373:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604903808:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.810378:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f747800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604903936:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.810391:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009cb6f854/0xaf35132e630336a6 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2887 expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.810396:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a269a887/0xaf35132e630336ad lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2880 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.810409:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009cb6f854/0xaf35132e630336a6 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2887 expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.810413:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a269a887/0xaf35132e630336ad lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2880 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.810436:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3e00, rc 0) 00010000:00010000:1.0:1780298642.810437:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36e00, rc 0) 00000100:00100000:1.0:1780298642.810471:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348004e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609416576:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 782us (862us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.810473:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e583050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609416448:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 745us (832us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.810479:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9054 00000100:00100000:0.0:1780298642.810487:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9055 00000100:00100000:2.0:1780298642.811280:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.811314:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609416704 00000100:00100000:1.0:1780298642.811318:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609416704 00000100:00100000:1.0:1780298642.811341:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9056 00000100:00100000:1.0:1780298642.811347:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b00000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609416704:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.811353:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.811360:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e630336b4 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d288e expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.811379:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604904064:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.811386:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.811417:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.811430:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609416832 00000100:00100000:2.0:1780298642.811436:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609416832 00000100:00100000:2.0:1780298642.811461:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9057 00000100:00100000:2.0:1780298642.811470:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348007480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609416832:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.811479:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.811487:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e630336bb lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2895 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.811515:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b01500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604904192:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.811549:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.811978:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604904064:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.811991:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e630336b4 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d288e expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.812001:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e630336b4 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d288e expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.812007:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36a00, rc 0) 00000100:00100000:1.0:1780298642.812032:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b00000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609416704:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 685us (753us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.812040:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9056 00000100:00100000:2.0:1780298642.812103:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b01500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604904192:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.812119:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e630336bb lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2895 expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.812151:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e630336bb lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2895 expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.812158:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baea00, rc 0) 00000100:00100000:2.0:1780298642.812183:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348007480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609416832:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 716us (802us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.812188:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9057 00000100:00100000:0.0:1780298642.812884:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.812910:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609416960 00000100:00100000:0.0:1780298642.812914:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.812915:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609416960 00000100:00100000:3.0:1780298642.812940:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609417088 00000100:00100000:3.0:1780298642.812942:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609417088 00000100:00100000:1.0:1780298642.812947:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9058 00000100:00100000:1.0:1780298642.812955:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609416960:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.812961:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298642.812966:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9059 00010000:00010000:1.0:1780298642.812968:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bb478d6d/0xaf35132e630336c2 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d289c expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.812975:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609417088:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.812982:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298642.812990:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604904320:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.812993:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000034be4f60/0xaf35132e630336c9 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d28a3 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.813017:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf348004a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604904448:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.813036:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.813042:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.813669:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604904320:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.813685:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bb478d6d/0xaf35132e630336c2 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d289c expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.813695:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bb478d6d/0xaf35132e630336c2 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d289c expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.813703:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341498c00, rc 0) 00000100:00100000:3.0:1780298642.813704:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348004a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604904448:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.813719:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000034be4f60/0xaf35132e630336c9 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28a3 expref: 12 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.813728:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609416960:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 774us (844us total) trans 0 rc 301/301 00010000:00010000:3.0:1780298642.813732:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000034be4f60/0xaf35132e630336c9 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28a3 expref: 12 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.813734:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9058 00010000:00010000:3.0:1780298642.813745:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf36d5a8400, rc 0) 00000100:00100000:3.0:1780298642.813764:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609417088:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 792us (853us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.813768:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9059 00000100:00100000:0.0:1780298642.814515:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.814544:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609417216 00000100:00100000:1.0:1780298642.814548:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609417216 00000100:00100000:1.0:1780298642.814570:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9060 00000100:00100000:1.0:1780298642.814576:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609417216:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.814582:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.814589:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008be5bdd7/0xaf35132e630336d0 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d28aa expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.814608:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604904576:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.814617:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.814646:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.814655:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609417344 00000100:00100000:2.0:1780298642.814660:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609417344 00000100:00100000:2.0:1780298642.814696:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9061 00000100:00100000:2.0:1780298642.814706:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609417344:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.814717:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.814725:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e630336d7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d28b1 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.814751:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b01500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604904704:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.814780:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.815121:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604904576:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.815166:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008be5bdd7/0xaf35132e630336d0 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28aa expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.815181:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008be5bdd7/0xaf35132e630336d0 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28aa expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.815191:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341499c00, rc 0) 00000100:00100000:1.0:1780298642.815224:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609417216:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 649us (710us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.815231:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9060 00000100:00100000:2.0:1780298642.815286:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b01500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604904704:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.815309:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e630336d7 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28b1 expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.815318:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e630336d7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28b1 expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.815325:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf400, rc 0) 00000100:00100000:2.0:1780298642.815346:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609417344:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 642us (729us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.815353:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9061 00000100:00100000:3.0:1780298642.815898:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.815927:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609417472 00000100:00100000:1.0:1780298642.815931:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609417472 00000100:00100000:1.0:1780298642.815955:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61953 00000100:00100000:1.0:1780298642.815960:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348004a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+102:29360:x1866777609417472:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.815970:0:6477:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.815977:0:6477:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499e00/0xaf35132e630336de lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 63 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d28b8 expref: -99 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.816014:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499200/0xaf35132e630336e5 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 64 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.816021:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499200/0xaf35132e630336e5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 64 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.816026:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499200/0xaf35132e630336e5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 64 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1780298642.816037:0:6477:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 10026169 00000004:00200000:1.0:1780298642.816039:0:6477:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1780298642.816046:0:6477:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499200/0xaf35132e630336e5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 64 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.816057:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499e00/0xaf35132e630336de lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 64 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28b8 expref: 104 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.816065:0:6477:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499200/0xaf35132e630336e5 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 63 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28b8 expref: 103 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.816070:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298642.816072:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1780298642.816074:0:6477:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341499200, rc 0) 00000100:00100000:1.0:1780298642.816095:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348004a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+103:29360:x1866777609417472:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 135us (198us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.816098:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61953 00000100:00100000:3.0:1780298642.816980:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.817010:0:7411:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609417728 00000100:00100000:1.0:1780298642.817014:0:7411:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609417728 00000100:00100000:1.0:1780298642.817036:0:7411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61954 00000100:00100000:1.0:1780298642.817040:0:7411:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348005f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+116:29360:x1866777609417728:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:1.0:1780298642.817059:0:7411:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298642.817067:0:7411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3800/0xaf35132e6303369f lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 64 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2879 expref: 116 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.817074:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298642.817077:0:7411:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298642.817163:0:7411:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841c600/0xaf35132e630336ec lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 63 type: IBT flags: 0x40000000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.817170:0:7411:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499200/0xaf35132e630336e5 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 63 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28b8 expref: 102 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.817180:0:7411:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841c600/0xaf35132e630336ec lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 63 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.817194:0:7411:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499200/0xaf35132e630336e5 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 63 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28b8 expref: 102 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.817201:0:7411:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499200/0xaf35132e630336e5 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 63 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28b8 expref: 102 pid: 6477 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.817208:0:7411:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7411:1866777604904832:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298642.817233:0:7411:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439480 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.817741:0:7411:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7411:1866777604904832:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298642.817751:0:7411:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841c600/0xaf35132e630336ec lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 63 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.817810:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.817832:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609417856 00000100:00100000:1.0:1780298642.817836:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609417856 00010000:00010000:1.0:1780298642.817847:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499200/0xaf35132e630336e5 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 63 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28b8 expref: 103 pid: 6477 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.817865:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15034 00000100:00100000:1.0:1780298642.817870:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b02450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+103:2399:x1866777609417856:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298642.817877:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298642.817880:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499200/0xaf35132e630336e5 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 64 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28b8 expref: 103 pid: 6477 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.817886:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499200/0xaf35132e630336e5 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 64 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28b8 expref: 103 pid: 6477 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.817894:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499200/0xaf35132e630336e5 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 64 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28b8 expref: 103 pid: 6477 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.817901:0:6461:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298642.817904:0:6461:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841c600/0xaf35132e630336ec lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 63 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.817910:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841c600/0xaf35132e630336ec lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 63 type: IBT flags: 0x40290000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.817914:0:6461:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841c600/0xaf35132e630336ec lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 63 type: IBT flags: 0x40290000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.817926:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298642.817943:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b02450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+102:2399:x1866777609417856:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 73us (133us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.817948:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15034 00010000:00010000:2.0:1780298642.817957:0:7411:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841c600/0xaf35132e630336ec lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 62 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.817967:0:7411:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841c600/0xaf35132e630336ec lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 62 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1780298642.818008:0:7411:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991028 00000004:00010000:2.0:1780298642.818033:0:7411:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 000000005fcc78c6 reply state 00000000771840de transno 4294991029 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841c600/0xaf35132e630336ec lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 62 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.818049:0:7411:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841c600/0xaf35132e630336ec lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 62 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.818056:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.818058:0:7411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1780298642.818060:0:7411:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1780298642.818109:0:7411:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348005f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+118:29360:x1866777609417728:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1067us (1128us total) trans 4294991029 rc 0/0 00000100:00100000:2.0:1780298642.818118:0:7411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61954 00000020:00080000:2.0:1780298642.818122:0:7411:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1780298642.818609:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.818637:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609417984 00000100:00100000:2.0:1780298642.818640:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609417984 00000100:00100000:2.0:1780298642.818662:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9062 00000100:00100000:2.0:1780298642.818666:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348006680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609417984:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.818673:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.818678:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e630336f3 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d28bf expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.818690:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e6303368a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d286b expref: 9 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.818695:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(124 177) 00010000:00010000:2.0:1780298642.818697:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(125 177) 00010000:00010000:2.0:1780298642.818700:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e630336f3 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28bf expref: 10 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.818710:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e6303368a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d286b expref: 9 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.818718:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e6303368a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d286b expref: 9 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.818725:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b01500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604904960:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298642.818752:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.819220:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b01500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604904960:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298642.819234:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e630336f3 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28bf expref: 10 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.819246:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae000, rc 0) 00000100:00100000:2.0:1780298642.819272:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348006680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609417984:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 605us (663us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.819277:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9062 00000100:00100000:3.0:1780298642.819364:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.819386:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609418112 00000100:00100000:0.0:1780298642.819390:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609418112 00010000:00010000:0.0:1780298642.819404:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e6303368a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d286b expref: 10 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.819424:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15035 00000100:00100000:0.0:1780298642.819430:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348007100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609418112:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298642.819437:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298642.819444:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e6303368a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d286b expref: 10 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.819451:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e6303368a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d286b expref: 10 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.819460:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e6303368a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d286b expref: 10 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.819471:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(125 177) 00010000:00010000:0.0:1780298642.819473:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(125 177) 00010000:00010000:0.0:1780298642.819476:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e630336f3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28bf expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.819482:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e630336f3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28bf expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.819495:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e630336f3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28bf expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.819500:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bdf80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777604905088:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298642.819528:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417540 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.820033:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bdf80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777604905088:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298642.820044:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298642.820067:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348007100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609418112:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 638us (706us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298642.820073:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15035 00000100:00100000:0.0:1780298642.820078:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.820112:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609418240 00000100:00100000:2.0:1780298642.820115:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609418240 00000100:00100000:2.0:1780298642.820159:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9063 00000100:00100000:2.0:1780298642.820166:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:29360:x1866777609418240:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.820171:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.820177:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e630336fa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d28c6 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.820189:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000078c7f460/0xaf35132e63033691 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2872 expref: 7 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.820199:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(89 130) 00010000:00010000:2.0:1780298642.820202:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(90 130) 00010000:00010000:2.0:1780298642.820205:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e630336fa lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28c6 expref: 12 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.820217:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000078c7f460/0xaf35132e63033691 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2872 expref: 7 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.820224:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000078c7f460/0xaf35132e63033691 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2872 expref: 7 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.820231:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b01500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604905216:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298642.820257:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.820683:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b01500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604905216:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298642.820692:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e630336fa lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28c6 expref: 12 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.820699:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baea00, rc 0) 00000100:00100000:2.0:1780298642.820720:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:29360:x1866777609418240:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 556us (645us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.820724:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9063 00000100:00100000:0.0:1780298642.820813:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.820836:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609418368 00000100:00100000:3.0:1780298642.820840:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609418368 00010000:00010000:3.0:1780298642.820852:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000078c7f460/0xaf35132e63033691 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2872 expref: 8 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.820871:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15036 00000100:00100000:3.0:1780298642.820875:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bdf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609418368:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298642.820881:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298642.820888:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000078c7f460/0xaf35132e63033691 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2872 expref: 8 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.820895:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000078c7f460/0xaf35132e63033691 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2872 expref: 8 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.820905:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078c7f460/0xaf35132e63033691 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2872 expref: 8 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.820911:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(90 130) 00010000:00010000:3.0:1780298642.820914:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(90 130) 00010000:00010000:3.0:1780298642.820917:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e630336fa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28c6 expref: 11 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.820926:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e630336fa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28c6 expref: 11 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.820938:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e630336fa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28c6 expref: 11 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.820944:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf348007b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:7400:1866777604905344:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298642.820967:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.821492:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348007b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:7400:1866777604905344:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298642.821502:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298642.821526:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bdf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609418368:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 650us (714us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298642.821534:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15036 00000100:00100000:1.0:1780298642.821628:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.821654:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609418624 00000100:00100000:0.0:1780298642.821658:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609418624 00002000:00010000:0.0:1780298642.821672:0:19949:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (2855056->18446744073709551615) 00002000:00100000:0.0:1780298642.821675:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b43b80 with x1866777609418624 ext(2855056->18446744073709551615) 00002000:00010000:0.0:1780298642.821679:0:19949:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf242b43b80. 00000100:00100000:0.0:1780298642.821699:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6041 00000100:00100000:0.0:1780298642.821709:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2399:x1866777609418624:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:3.0:1780298642.821737:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.821769:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609418496 00000100:00100000:2.0:1780298642.821773:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609418496 00002000:00010000:2.0:1780298642.821785:0:8364:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:2.0:1780298642.821788:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf348007b80 with x1866777609418496 ext(4194304->18446744073709551615) 00002000:00010000:2.0:1780298642.821792:0:8364:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf348007b80. 00000100:00100000:2.0:1780298642.821810:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6042 00000100:00100000:2.0:1780298642.821816:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348007b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609418496:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:2.0:1780298642.822211:0:8364:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 5831865 -> 4194304 00002000:00010000:2.0:1780298642.822214:0:8364:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:2.0:1780298642.822216:0:8364:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:2.0:1780298642.822218:0:8364:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:2.0:1780298642.822220:0:8364:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 368 -> 352 00000100:00100000:2.0:1780298642.822244:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348007b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609418496:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 428us (544us total) trans 4294969645 rc 0/0 00000100:00100000:2.0:1780298642.822249:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6042 00002000:00010000:2.0:1780298642.822251:0:8364:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:2.0:1780298642.822254:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf348007b80 with x1866777609418496 ext(4194304->18446744073709551615) 00002000:00010000:2.0:1780298642.822258:0:8364:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf348007b80. 00002000:00010000:0.0:1780298642.822509:0:19949:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4194304 -> 2855056 00002000:00010000:0.0:1780298642.822512:0:19949:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:0.0:1780298642.822514:0:19949:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:0.0:1780298642.822516:0:19949:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:0.0:1780298642.822518:0:19949:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:0.0:1780298642.822540:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609418624:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 834us (914us total) trans 4294969204 rc 0/0 00000100:00100000:0.0:1780298642.822545:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6041 00002000:00010000:0.0:1780298642.822547:0:19949:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (2855056->18446744073709551615) 00002000:00100000:0.0:1780298642.822549:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b43b80 with x1866777609418624 ext(2855056->18446744073709551615) 00002000:00010000:0.0:1780298642.822553:0:19949:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf242b43b80. 00000100:00100000:3.0:1780298642.823073:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.823100:0:6476:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609418752 00000100:00100000:1.0:1780298642.823104:0:6476:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609418752 00000100:00100000:1.0:1780298642.823148:0:6476:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61955 00000100:00100000:1.0:1780298642.823156:0:6476:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348005180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+118:29360:x1866777609418752:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.823164:0:6476:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.823171:0:6476:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841ca00/0xaf35132e63033701 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 63 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d28cd expref: -99 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.823200:0:6476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841c800/0xaf35132e63033708 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 64 type: IBT flags: 0x40000000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.823207:0:6476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841c800/0xaf35132e63033708 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 64 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.823212:0:6476:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841c800/0xaf35132e63033708 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 64 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1780298642.823221:0:6476:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 7049360 00000004:00200000:1.0:1780298642.823223:0:6476:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1780298642.823229:0:6476:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841c800/0xaf35132e63033708 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 64 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.823240:0:6476:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841ca00/0xaf35132e63033701 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 64 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28cd expref: 120 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.823248:0:6476:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841c800/0xaf35132e63033708 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 63 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28cd expref: 119 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.823253:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298642.823255:0:6476:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1780298642.823257:0:6476:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34841c800, rc 0) 00000100:00100000:1.0:1780298642.823276:0:6476:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348005180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+119:29360:x1866777609418752:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 123us (204us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.823280:0:6476:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61955 00000100:00100000:1.0:1780298642.824109:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.824158:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609418880 00000100:00100000:2.0:1780298642.824162:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609418880 00000100:00100000:2.0:1780298642.824182:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9064 00000100:00100000:2.0:1780298642.824187:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609418880:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.824192:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.824198:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e6303370f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d28d4 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.824208:0:19992:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e630336fa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28c6 expref: 12 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.824217:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(90 130) 00010000:00010000:2.0:1780298642.824219:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(91 130) 00010000:00010000:2.0:1780298642.824222:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e6303370f lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [733184->745471] (req 733184->745471) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28d4 expref: 8 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.824232:0:19992:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e630336fa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28c6 expref: 12 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.824239:0:19992:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e630336fa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28c6 expref: 12 pid: 12101 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.824244:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b01500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604905472:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298642.824272:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.824804:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b01500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604905472:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298642.824814:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e6303370f lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [733184->745471] (req 733184->745471) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28d4 expref: 8 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.824822:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf400, rc 0) 00000100:00100000:2.0:1780298642.824845:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:29360:x1866777609418880:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 658us (736us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.824852:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9064 00000100:00100000:1.0:1780298642.824904:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.824953:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609419008 00000100:00100000:3.0:1780298642.824957:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609419008 00010000:00010000:3.0:1780298642.824969:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e630336fa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28c6 expref: 13 pid: 12101 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.824988:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15037 00000100:00100000:3.0:1780298642.824992:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b42a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609419008:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298642.824998:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298642.825005:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e630336fa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28c6 expref: 13 pid: 12101 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.825012:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e630336fa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28c6 expref: 13 pid: 12101 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.825021:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e630336fa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28c6 expref: 13 pid: 12101 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.825028:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(91 130) 00010000:00010000:3.0:1780298642.825030:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(91 130) 00010000:00010000:3.0:1780298642.825035:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e6303370f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [733184->34287615] (req 733184->745471) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28d4 expref: 7 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.825040:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e6303370f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [733184->34287615] (req 733184->745471) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28d4 expref: 7 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.825052:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e6303370f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [733184->34287615] (req 733184->745471) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28d4 expref: 7 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.825059:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf348004e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777604905600:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298642.825082:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.825697:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348004e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777604905600:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298642.825705:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298642.825726:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b42a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609419008:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 734us (822us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.825732:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15037 00000100:00100000:3.0:1780298642.825859:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.825892:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609419136 00000100:00100000:2.0:1780298642.825896:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609419136 00000100:00100000:2.0:1780298642.825919:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6043 00000100:00100000:2.0:1780298642.825924:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348005c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609419136:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298642.826000:0:19928:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 2855056 -> 2855056 00002000:00010000:2.0:1780298642.826003:0:19928:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:2.0:1780298642.826006:0:19928:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:2.0:1780298642.826008:0:19928:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:2.0:1780298642.826010:0:19928:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:2.0:1780298642.826034:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348005c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609419136:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 110us (176us total) trans 4294969205 rc 0/0 00000100:00100000:2.0:1780298642.826039:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6043 00000100:00100000:0.0:1780298642.826624:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.826665:0:9976:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609419264 00000100:00100000:2.0:1780298642.826669:0:9976:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609419264 00000100:00100000:2.0:1780298642.826691:0:9976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61956 00000100:00100000:2.0:1780298642.826697:0:9976:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255092d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+102:29360:x1866777609419264:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.826705:0:9976:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.826714:0:9976:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baec00/0xaf35132e63033716 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 64 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d28db expref: -99 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.826753:0:9976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baee00/0xaf35132e6303371d lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 65 type: IBT flags: 0x40000000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.826764:0:9976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baee00/0xaf35132e6303371d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 65 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.826772:0:9976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baee00/0xaf35132e6303371d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 65 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298642.826786:0:9976:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 7049360 00000004:00200000:2.0:1780298642.826789:0:9976:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298642.826795:0:9976:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baee00/0xaf35132e6303371d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 65 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.826807:0:9976:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baec00/0xaf35132e63033716 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 65 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28db expref: 104 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.826817:0:9976:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baee00/0xaf35132e6303371d lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 64 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28db expref: 103 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.826821:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.826823:0:9976:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298642.826825:0:9976:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baee00, rc 0) 00000100:00100000:2.0:1780298642.826847:0:9976:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255092d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+103:29360:x1866777609419264:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 151us (223us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.826851:0:9976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61956 00000100:00100000:3.0:1780298642.827548:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.827577:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609419392 00000100:00100000:2.0:1780298642.827581:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609419392 00000100:00100000:2.0:1780298642.827601:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9065 00000100:00100000:2.0:1780298642.827606:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348004e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609419392:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.827612:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.827624:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002ee949fd/0xaf35132e63033724 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d28e2 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.827644:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b01500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604905728:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.827678:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.827683:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.827711:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609419520 00000100:00100000:1.0:1780298642.827716:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609419520 00000100:00100000:1.0:1780298642.827743:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9066 00000100:00100000:1.0:1780298642.827749:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255091500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609419520:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.827755:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.827761:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006c79d5c1/0xaf35132e6303372b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d28e9 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.827781:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b41880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604905856:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.827806:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439c00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.828304:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b01500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604905728:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.828312:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b41880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604905856:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.828329:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002ee949fd/0xaf35132e63033724 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28e2 expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.828332:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000006c79d5c1/0xaf35132e6303372b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28e9 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.828359:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002ee949fd/0xaf35132e63033724 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28e2 expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.828359:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006c79d5c1/0xaf35132e6303372b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28e9 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.828377:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34841c000, rc 0) 00010000:00010000:2.0:1780298642.828379:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bafc00, rc 0) 00000100:00100000:2.0:1780298642.828402:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348004e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609419392:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 794us (854us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.828402:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255091500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609419520:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 652us (722us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.828418:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9066 00000100:00100000:2.0:1780298642.828424:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9065 00000100:00100000:3.0:1780298642.829181:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.829205:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609419648 00000100:00100000:2.0:1780298642.829209:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609419648 00000100:00100000:3.0:1780298642.829213:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.829240:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609419776 00000100:00100000:2.0:1780298642.829242:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609419776 00000100:00100000:1.0:1780298642.829256:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9067 00000100:00100000:2.0:1780298642.829264:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9068 00000100:00100000:1.0:1780298642.829269:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609419648:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298642.829276:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609419776:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.829276:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.829285:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000004503c2f/0xaf35132e63033732 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d28f0 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.829295:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.829306:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e63033739 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d28f7 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.829320:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b41880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604905984:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.829338:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b01500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604906112:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.829361:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439c00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.829369:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.829916:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b41880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604905984:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.829932:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000004503c2f/0xaf35132e63033732 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28f0 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.829935:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b01500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604906112:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.829949:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000004503c2f/0xaf35132e63033732 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28f0 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.829955:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e63033739 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28f7 expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.829977:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34841ca00, rc 0) 00010000:00010000:2.0:1780298642.829983:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e63033739 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28f7 expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.829991:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf200, rc 0) 00000100:00100000:2.0:1780298642.830010:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609419776:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 737us (799us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.830010:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609419648:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 744us (827us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.830024:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9068 00000100:00100000:1.0:1780298642.830031:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9067 00000100:00100000:0.0:1780298642.831085:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.831120:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609419904 00000100:00100000:3.0:1780298642.831150:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609419904 00002000:00100000:3.0:1780298642.831175:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (2519040->2523135) req@ffff9cf255090a80 x1866777609419904/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298642.831186:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255090a80 with x1866777609419904 ext(2519040->2523135) 00002000:00010000:3.0:1780298642.831190:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf255090a80 00000100:00100000:3.0:1780298642.831210:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6044 00000100:00100000:3.0:1780298642.831220:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255090a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609419904:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298642.831360:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255090a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609419904:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 142us (275us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1780298642.831368:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6044 00002000:00100000:3.0:1780298642.831375:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (2519040->2523135) req@ffff9cf255090a80 x1866777609419904/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/4536 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298642.831383:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255090a80 with x1866777609419904 ext(2519040->2523135) 00002000:00010000:3.0:1780298642.831387:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf255090a80 00000100:00100000:3.0:1780298642.832208:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.832254:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609420160 00000100:00100000:2.0:1780298642.832259:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609420160 00000100:00100000:3.0:1780298642.832276:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.832305:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609420032 00000100:00100000:2.0:1780298642.832309:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609420032 00000100:00100000:1.0:1780298642.832312:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9069 00000100:00100000:1.0:1780298642.832321:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609420160:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.832327:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.832334:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c18c87d/0xaf35132e63033740 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2905 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.832340:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9070 00000100:00100000:2.0:1780298642.832349:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609420032:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.832355:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298642.832360:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b41880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604906240:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.832364:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007b6539a7/0xaf35132e63033747 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d28fe expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.832394:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439c00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.832396:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b01500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604906368:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.832427:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00002000:00010000:2.0:1780298642.832903:0:8356:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks: 0 -> 1 00000100:00100000:2.0:1780298642.832909:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b41880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604906240:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.832920:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004c18c87d/0xaf35132e63033740 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2905 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.832939:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c18c87d/0xaf35132e63033740 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2905 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.832949:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34841ce00, rc 0) 00000100:00100000:2.0:1780298642.832976:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609420160:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 657us (769us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.832982:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9069 00000100:00100000:2.0:1780298642.833089:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b01500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604906368:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.833104:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007b6539a7/0xaf35132e63033747 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28fe expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.833116:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007b6539a7/0xaf35132e63033747 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28fe expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.833145:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae800, rc 0) 00000100:00100000:2.0:1780298642.833167:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609420032:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 819us (893us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.833173:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9070 00000100:00100000:0.0:1780298642.833823:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.833853:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609420416 00000100:00100000:2.0:1780298642.833857:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609420416 00000100:00100000:2.0:1780298642.833880:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9071 00000100:00100000:2.0:1780298642.833886:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255090700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609420416:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.833893:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.833902:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e6303374e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2913 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.833902:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.833925:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609420288 00000100:00100000:3.0:1780298642.833929:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609420288 00000100:00100000:2.0:1780298642.833933:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b01500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604906496:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.833973:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9072 00000100:00100000:2.0:1780298642.833979:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.833987:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b40a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609420288:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.833996:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.834006:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e63033755 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d290c expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.834036:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604906624:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.834068:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417d80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.834512:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b01500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604906496:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.834527:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e6303374e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2913 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.834539:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e6303374e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2913 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.834549:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf600, rc 0) 00000100:00100000:2.0:1780298642.834574:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255090700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609420416:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 689us (752us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.834578:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9071 00000100:00100000:0.0:1780298642.834752:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604906624:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.834768:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e63033755 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d290c expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.834789:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e63033755 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d290c expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.834799:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3200, rc 0) 00000100:00100000:0.0:1780298642.834826:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b40a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609420288:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 845us (925us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.834833:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9072 00000100:00100000:1.0:1780298642.835528:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.835574:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609420544 00000100:00100000:3.0:1780298642.835586:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609420544 00000100:00100000:3.0:1780298642.835617:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9073 00000100:00100000:3.0:1780298642.835625:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b41f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:29360:x1866777609420544:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.835634:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.835641:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000019d620bb/0xaf35132e6303375c lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d291a expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.835662:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e630336f3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28bf expref: 10 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.835671:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(125 177) 00010000:00010000:3.0:1780298642.835675:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(126 177) 00010000:00010000:3.0:1780298642.835680:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000019d620bb/0xaf35132e6303375c lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3416064->3424255] (req 3416064->3424255) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d291a expref: 10 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.835700:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e630336f3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28bf expref: 10 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.835713:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e630336f3 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28bf expref: 10 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.835722:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604906752:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298642.835760:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.836240:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604906752:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298642.836250:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000019d620bb/0xaf35132e6303375c lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3416064->3424255] (req 3416064->3424255) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d291a expref: 10 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.836258:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf36d5a8200, rc 0) 00000100:00100000:3.0:1780298642.836283:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b41f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:29360:x1866777609420544:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 659us (754us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298642.836306:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9073 00000100:00100000:0.0:1780298642.836477:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.836513:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609420672 00000100:00100000:2.0:1780298642.836517:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609420672 00010000:00010000:2.0:1780298642.836534:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e630336f3 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28bf expref: 11 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.836554:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15038 00000100:00100000:2.0:1780298642.836559:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255092680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609420672:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298642.836565:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298642.836573:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e630336f3 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28bf expref: 11 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.836582:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e630336f3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28bf expref: 11 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.836592:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e630336f3 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28bf expref: 11 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.836599:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(126 177) 00010000:00010000:2.0:1780298642.836602:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(126 177) 00010000:00010000:2.0:1780298642.836606:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000019d620bb/0xaf35132e6303375c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3416064->36970495] (req 3416064->3424255) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d291a expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.836612:0:6463:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000019d620bb/0xaf35132e6303375c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3416064->36970495] (req 3416064->3424255) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d291a expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.836625:0:6463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000019d620bb/0xaf35132e6303375c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3416064->36970495] (req 3416064->3424255) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d291a expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.836639:0:6463:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b01500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6463:1866777604906880:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298642.836666:0:6463:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.837207:0:6463:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b01500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6463:1866777604906880:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298642.837218:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298642.837243:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255092680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609420672:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 683us (766us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.837248:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15038 00000100:00100000:1.0:1780298642.837371:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.837417:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609420800 00000100:00100000:3.0:1780298642.837421:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609420800 00002000:00100000:3.0:1780298642.837437:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (3416064->3420159) req@ffff9cf242b43480 x1866777609420800/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298642.837443:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b43480 with x1866777609420800 ext(3416064->3420159) 00002000:00010000:3.0:1780298642.837446:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf242b43480 00000100:00100000:3.0:1780298642.837461:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6045 00000100:00100000:3.0:1780298642.837468:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609420800:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298642.837547:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609420800:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 80us (176us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1780298642.837552:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6045 00002000:00100000:3.0:1780298642.837556:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (3416064->3420159) req@ffff9cf242b43480 x1866777609420800/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/4536 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298642.837562:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b43480 with x1866777609420800 ext(3416064->3420159) 00002000:00010000:3.0:1780298642.837565:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf242b43480 00000100:00100000:0.0:1780298642.838312:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.838339:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609420928 00000100:00100000:1.0:1780298642.838343:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609420928 00002000:00100000:1.0:1780298642.838358:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x2c0000401:0x11:0x0] (3420160->3424255) req@ffff9cf255093b80 x1866777609420928/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298642.838365:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255093b80 with x1866777609420928 ext(3420160->3424255) 00002000:00010000:1.0:1780298642.838368:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255093b80 00000100:00100000:1.0:1780298642.838382:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6046 00000100:00100000:1.0:1780298642.838387:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255093b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609420928:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298642.838458:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255093b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609420928:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 70us (145us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298642.838463:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6046 00002000:00100000:1.0:1780298642.838468:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x2c0000401:0x11:0x0] (3420160->3424255) req@ffff9cf255093b80 x1866777609420928/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/4536 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298642.838474:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255093b80 with x1866777609420928 ext(3420160->3424255) 00002000:00010000:1.0:1780298642.838476:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255093b80 00000100:00100000:1.0:1780298642.839232:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.839268:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609421056 00000100:00100000:2.0:1780298642.839273:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609421056 00000100:00100000:2.0:1780298642.839316:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9074 00000100:00100000:2.0:1780298642.839325:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b43100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609421056:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.839334:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.839344:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005d10d3aa/0xaf35132e63033763 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2921 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.839347:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.839380:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b01500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604907008:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.839381:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609421184 00000100:00100000:3.0:1780298642.839386:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609421184 00000100:00100000:3.0:1780298642.839410:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9075 00000100:00100000:3.0:1780298642.839416:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b6bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609421184:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.839422:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.839429:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063c74321/0xaf35132e6303376a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2928 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.839432:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.839458:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604907136:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.839487:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.840093:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b01500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604907008:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.840096:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604907136:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.840142:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005d10d3aa/0xaf35132e63033763 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2921 expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.840147:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000063c74321/0xaf35132e6303376a lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2928 expref: 13 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.840168:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063c74321/0xaf35132e6303376a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2928 expref: 13 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.840168:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005d10d3aa/0xaf35132e63033763 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2921 expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.840188:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf36d5a9000, rc 0) 00010000:00010000:2.0:1780298642.840193:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baec00, rc 0) 00000100:00100000:3.0:1780298642.840213:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b6bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609421184:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 797us (868us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.840218:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9075 00000100:00100000:2.0:1780298642.840224:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609421056:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 898us (989us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.840233:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9074 00000100:00100000:1.0:1780298642.841197:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.841220:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.841235:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609421312 00000100:00100000:2.0:1780298642.841240:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609421312 00000100:00100000:2.0:1780298642.841264:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609421440 00000100:00100000:2.0:1780298642.841266:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609421440 00000100:00100000:1.0:1780298642.841276:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9076 00000100:00100000:1.0:1780298642.841284:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b43800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609421312:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298642.841302:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9077 00010000:00010000:1.0:1780298642.841302:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.841311:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e8a7e8eb/0xaf35132e63033771 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d292f expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.841317:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b41180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609421440:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.841328:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.841337:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005dffef48/0xaf35132e63033778 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2936 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.841340:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b42300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604907264:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.841374:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604907392:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.841377:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.841403:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.841880:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b42300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604907264:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.841892:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e8a7e8eb/0xaf35132e63033771 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d292f expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.841902:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e8a7e8eb/0xaf35132e63033771 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d292f expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.841912:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341499e00, rc 0) 00000100:00100000:1.0:1780298642.841936:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609421312:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 655us (740us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.841941:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9076 00000100:00100000:2.0:1780298642.841999:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604907392:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.842016:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005dffef48/0xaf35132e63033778 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2936 expref: 13 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.842031:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005dffef48/0xaf35132e63033778 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2936 expref: 13 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.842041:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf36d5a8000, rc 0) 00000100:00100000:2.0:1780298642.842066:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b41180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609421440:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 755us (847us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.842073:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9077 00000100:00100000:0.0:1780298642.842871:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.842895:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.842906:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609421568 00000100:00100000:2.0:1780298642.842910:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609421568 00000100:00100000:3.0:1780298642.842918:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609421696 00000100:00100000:3.0:1780298642.842924:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609421696 00000100:00100000:2.0:1780298642.842938:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9078 00000100:00100000:2.0:1780298642.842948:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b68700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609421568:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.842957:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298642.842961:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9079 00010000:00010000:2.0:1780298642.842966:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002ee949fd/0xaf35132e6303377f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d293d expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.842971:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609421696:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.842981:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.842988:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ac4408d/0xaf35132e63033786 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2944 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.843002:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b01500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604907520:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.843011:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b42300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604907648:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.843047:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.843047:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.843685:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b01500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604907520:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.843707:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002ee949fd/0xaf35132e6303377f lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d293d expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.843717:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002ee949fd/0xaf35132e6303377f lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d293d expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.843725:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bafc00, rc 0) 00000100:00100000:2.0:1780298642.843754:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b68700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609421568:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 809us (883us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.843761:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9078 00000100:00100000:1.0:1780298642.843769:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b42300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604907648:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.843784:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000ac4408d/0xaf35132e63033786 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2944 expref: 13 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.843795:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ac4408d/0xaf35132e63033786 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2944 expref: 13 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.843802:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341499200, rc 0) 00000100:00100000:1.0:1780298642.843818:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609421696:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 852us (924us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.843822:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9079 00000100:00100000:0.0:1780298642.844612:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.844650:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609421824 00000100:00100000:2.0:1780298642.844655:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609421824 00000100:00100000:2.0:1780298642.844681:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9080 00000100:00100000:0.0:1780298642.844688:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.844692:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254919880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609421824:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.844700:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298642.844705:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609421952 00010000:00010000:2.0:1780298642.844709:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e6303378d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d294b expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.844709:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609421952 00000100:00100000:2.0:1780298642.844734:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b01500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604907776:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.844736:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9081 00000100:00100000:1.0:1780298642.844743:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609421952:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.844750:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.844758:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008be5bdd7/0xaf35132e63033794 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2952 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.844775:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.844781:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b42300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604907904:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.844804:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.845263:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b01500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604907776:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.845278:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e6303378d lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d294b expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.845304:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e6303378d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d294b expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.845314:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae000, rc 0) 00000100:00100000:2.0:1780298642.845341:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254919880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609421824:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 653us (730us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.845346:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9080 00000100:00100000:1.0:1780298642.845415:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b42300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604907904:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.845425:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008be5bdd7/0xaf35132e63033794 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2952 expref: 13 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.845434:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008be5bdd7/0xaf35132e63033794 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2952 expref: 13 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.845440:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341499c00, rc 0) 00000100:00100000:1.0:1780298642.845455:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609421952:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 714us (771us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.845459:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9081 00000100:00100000:0.0:1780298642.846364:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.846389:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609422080 00000100:00100000:3.0:1780298642.846394:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609422080 00000100:00100000:0.0:1780298642.846398:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.846418:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609422208 00000100:00100000:2.0:1780298642.846430:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9082 00000100:00100000:2.0:1780298642.846440:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609422080:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298642.846447:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609422208 00010000:00010000:2.0:1780298642.846447:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.846457:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e6303379b lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2959 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.846470:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9083 00000100:00100000:2.0:1780298642.846479:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b01500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604908032:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.846481:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254919180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609422208:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.846490:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.846497:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063c74321/0xaf35132e630337a2 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2960 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.846518:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.846523:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604908160:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.846547:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.847053:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b01500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604908032:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.847073:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e6303379b lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2959 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.847087:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e6303379b lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2959 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.847097:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf600, rc 0) 00000100:00100000:3.0:1780298642.847140:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604908160:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.847147:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609422080:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 707us (780us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.847154:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9082 00010000:00010000:3.0:1780298642.847158:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000063c74321/0xaf35132e630337a2 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2960 expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.847169:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063c74321/0xaf35132e630337a2 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2960 expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.847177:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf36d5a9000, rc 0) 00000100:00100000:3.0:1780298642.847196:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254919180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609422208:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 720us (798us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.847201:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9083 00000100:00100000:0.0:1780298642.847991:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.848015:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609422336 00000100:00100000:3.0:1780298642.848019:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609422336 00000100:00100000:3.0:1780298642.848045:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9084 00000100:00100000:3.0:1780298642.848053:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:29360:x1866777609422336:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.848062:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.848070:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d5524779/0xaf35132e630337a9 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2967 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.848085:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(126 177) 00010000:00010000:3.0:1780298642.848089:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(126 177) 00010000:00010000:3.0:1780298642.848099:0:20252:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d5524779/0xaf35132e630337a9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3067904->3416063] (req 3067904->3071999) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2967 expref: 11 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.848113:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d5524779/0xaf35132e630337a9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3067904->3416063] (req 3067904->3071999) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2967 expref: 11 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.848121:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf36d5a9c00, rc 0) 00000100:00100000:3.0:1780298642.848174:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:29360:x1866777609422336:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 121us (182us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298642.848182:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9084 00000100:00100000:0.0:1780298642.848796:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.848834:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609422464 00000100:00100000:2.0:1780298642.848839:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609422464 00002000:00100000:2.0:1780298642.848864:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (3067904->3071999) req@ffff9cf25491ad80 x1866777609422464/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.848881:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25491ad80 with x1866777609422464 ext(3067904->3071999) 00002000:00010000:2.0:1780298642.848886:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf25491ad80 00000100:00100000:2.0:1780298642.848907:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6047 00000100:00100000:2.0:1780298642.848916:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609422464:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298642.849022:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609422464:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 107us (225us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298642.849029:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6047 00002000:00100000:2.0:1780298642.849035:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (3067904->3071999) req@ffff9cf25491ad80 x1866777609422464/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/4536 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.849044:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25491ad80 with x1866777609422464 ext(3067904->3071999) 00002000:00010000:2.0:1780298642.849047:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf25491ad80 00000100:00100000:0.0:1780298642.850229:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.850269:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609422592 00000100:00100000:2.0:1780298642.850273:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609422592 00002000:00100000:2.0:1780298642.850303:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (3072000->3076095) req@ffff9cf254918000 x1866777609422592/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.850310:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf254918000 with x1866777609422592 ext(3072000->3076095) 00002000:00010000:2.0:1780298642.850313:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf254918000 00000100:00100000:2.0:1780298642.850328:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6048 00000100:00100000:2.0:1780298642.850334:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254918000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609422592:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298642.850389:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254918000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609422592:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 55us (162us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298642.850394:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6048 00002000:00100000:2.0:1780298642.850398:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (3072000->3076095) req@ffff9cf254918000 x1866777609422592/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/4536 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.850404:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf254918000 with x1866777609422592 ext(3072000->3076095) 00002000:00010000:2.0:1780298642.850407:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf254918000 00000100:00100000:0.0:1780298642.851272:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.851308:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609422720 00000100:00100000:3.0:1780298642.851312:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609422720 00002000:00100000:3.0:1780298642.851328:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (3067904->3424255) req@ffff9cf37f734a80 x1866777609422720/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 504/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298642.851335:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f734a80 with x1866777609422720 ext(3067904->3424255) 00002000:00010000:3.0:1780298642.851340:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f734a80 00000100:00100000:3.0:1780298642.851355:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6049 00000100:00100000:3.0:1780298642.851360:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f734a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609422720:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:1.0:1780298642.852404:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f734a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2397:x1866777609422720:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 1041us (1131us total) trans 4294969646 rc 0/0 00000100:00100000:1.0:1780298642.852413:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6049 00002000:00100000:1.0:1780298642.852420:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (3067904->3424255) req@ffff9cf37f734a80 x1866777609422720/t4294969646(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 504/448 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298642.852430:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f734a80 with x1866777609422720 ext(3067904->3424255) 00002000:00010000:1.0:1780298642.852437:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f734a80 00000100:00100000:0.0:1780298642.853045:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.853087:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609422848 00000100:00100000:1.0:1780298642.853091:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609422848 00000100:00100000:1.0:1780298642.853118:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9085 00000100:00100000:1.0:1780298642.853144:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f734000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2398:x1866777609422848:12345-192.168.203.25@tcp:16:fsx.0 00000001:00080000:2.0:1780298642.853864:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969646 is committed 00000001:00080000:2.0:1780298642.853872:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969645 is committed 00000100:00100000:1.0:1780298642.853938:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f734000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609422848:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 813us (892us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.853943:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9085 00000100:00100000:0.0:1780298642.854854:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.854905:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609422976 00000100:00100000:3.0:1780298642.854909:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609422976 00000100:00100000:0.0:1780298642.854917:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.854954:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609423104 00000100:00100000:2.0:1780298642.854959:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609423104 00000100:00100000:3.0:1780298642.854964:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9086 00000100:00100000:3.0:1780298642.854973:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f734700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2397:x1866777609422976:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.854980:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.854990:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000034be4f60/0xaf35132e630337b0 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d296e expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.854996:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9087 00000100:00100000:0.0:1780298642.855006:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f734e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609423104:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.855014:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.855023:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e630337b7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2975 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.855043:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604908288:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.855058:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f737800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604908416:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.855103:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.855103:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417d80 going to sleep for 11 seconds 00002000:00010000:3.0:1780298642.855932:0:8356:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks: 352 -> 384 00000100:00100000:3.0:1780298642.855942:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604908288:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.855945:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f737800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604908416:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.855964:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000034be4f60/0xaf35132e630337b0 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d296e expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.855981:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000034be4f60/0xaf35132e630337b0 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d296e expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.855981:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e630337b7 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2975 expref: 13 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.855999:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf36d5a8400, rc 0) 00010000:00010000:2.0:1780298642.856010:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e630337b7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2975 expref: 13 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.856022:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3800, rc 0) 00000100:00100000:3.0:1780298642.856025:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f734700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2397:x1866777609422976:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1055us (1170us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.856031:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9086 00000100:00100000:2.0:1780298642.856046:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f734e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609423104:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1041us (1128us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.856052:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9087 00000100:00100000:1.0:1780298642.856874:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.856913:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609423232 00000100:00100000:1.0:1780298642.856918:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.856919:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609423232 00000100:00100000:0.0:1780298642.856943:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609423360 00000100:00100000:0.0:1780298642.856945:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609423360 00000100:00100000:1.0:1780298642.856949:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9088 00000100:00100000:1.0:1780298642.856958:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2398:x1866777609423232:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298642.856964:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9089 00010000:00010000:1.0:1780298642.856968:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298642.856977:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b40000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609423360:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.856977:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000ac4408d/0xaf35132e630337be lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d297c expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.856991:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.857001:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002ee949fd/0xaf35132e630337c5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2983 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.857007:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b41880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604908544:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.857045:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b01500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604908672:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.857052:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.857075:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4000 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.857710:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b41880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604908544:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.857725:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b01500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604908672:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.857730:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000ac4408d/0xaf35132e630337be lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d297c expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.857743:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002ee949fd/0xaf35132e630337c5 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2983 expref: 13 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.857758:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000ac4408d/0xaf35132e630337be lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d297c expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.857761:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002ee949fd/0xaf35132e630337c5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2983 expref: 13 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.857777:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341499200, rc 0) 00010000:00010000:2.0:1780298642.857779:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bafc00, rc 0) 00000100:00100000:1.0:1780298642.857805:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2398:x1866777609423232:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 848us (931us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.857806:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b40000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609423360:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 833us (888us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.857817:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9088 00000100:00100000:2.0:1780298642.857825:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9089 00000100:00100000:0.0:1780298642.858650:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.858690:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609423616 00000100:00100000:2.0:1780298642.858695:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609423616 00000100:00100000:2.0:1780298642.858726:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9090 00000100:00100000:2.0:1780298642.858738:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f735f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609423616:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298642.858739:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:2.0:1780298642.858748:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.858758:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e630337cc lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2991 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.858767:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609423488 00000100:00100000:3.0:1780298642.858773:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609423488 00000100:00100000:2.0:1780298642.858786:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b01500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604908800:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.858807:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9091 00000100:00100000:3.0:1780298642.858818:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f735c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2398:x1866777609423488:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.858829:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298642.858836:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4000 going to sleep for 11 seconds 00010000:00010000:3.0:1780298642.858838:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000063c74321/0xaf35132e630337d3 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d298a expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.858869:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604908928:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.858963:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.859470:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604908928:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.859478:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b01500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604908800:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.859486:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000063c74321/0xaf35132e630337d3 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d298a expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.859497:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000063c74321/0xaf35132e630337d3 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d298a expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.859501:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e630337cc lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2991 expref: 13 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.859506:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf36d5a9000, rc 0) 00010000:00010000:2.0:1780298642.859522:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e630337cc lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2991 expref: 13 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.859531:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f735c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2398:x1866777609423488:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 716us (794us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.859536:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9091 00010000:00010000:2.0:1780298642.859536:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf600, rc 0) 00000100:00100000:2.0:1780298642.859566:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f735f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609423616:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 831us (914us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.859574:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9090 00000100:00100000:0.0:1780298642.860373:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.860411:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609423744 00000100:00100000:2.0:1780298642.860417:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609423744 00000100:00100000:2.0:1780298642.860449:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9092 00000100:00100000:2.0:1780298642.860458:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f736d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2397:x1866777609423744:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.860466:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.860477:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e630337da lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2998 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.860487:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.860512:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609423872 00000100:00100000:2.0:1780298642.860515:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b01500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604909056:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.860517:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609423872 00000100:00100000:1.0:1780298642.860545:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9093 00000100:00100000:1.0:1780298642.860551:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f735880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609423872:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.860556:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.860563:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008be5bdd7/0xaf35132e630337e1 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d299f expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.860575:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4000 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.860595:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b41880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604909184:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.860622:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.861246:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b41880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604909184:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.861262:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008be5bdd7/0xaf35132e630337e1 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d299f expref: 13 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.861272:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008be5bdd7/0xaf35132e630337e1 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d299f expref: 13 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.861280:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b01500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604909056:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.861306:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341499c00, rc 0) 00010000:00010000:2.0:1780298642.861310:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e630337da lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2998 expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.861325:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e630337da lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2998 expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.861333:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae000, rc 0) 00000100:00100000:1.0:1780298642.861338:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f735880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609423872:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 786us (852us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.861344:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9093 00000100:00100000:2.0:1780298642.861354:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f736d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2397:x1866777609423744:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 897us (981us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.861359:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9092 00000100:00100000:0.0:1780298642.862343:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.862363:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609424000 00000100:00100000:3.0:1780298642.862367:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609424000 00000100:00100000:3.0:1780298642.862389:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9094 00000100:00100000:3.0:1780298642.862394:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f737b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:29360:x1866777609424000:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.862401:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.862407:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004cd65b27/0xaf35132e630337e8 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d29a6 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.862423:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(126 177) 00010000:00010000:3.0:1780298642.862425:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(126 177) 00010000:00010000:3.0:1780298642.862433:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004cd65b27/0xaf35132e630337e8 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [819200->3067903] (req 819200->831487) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29a6 expref: 9 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.862442:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004cd65b27/0xaf35132e630337e8 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [819200->3067903] (req 819200->831487) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29a6 expref: 9 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.862447:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf36d5a8600, rc 0) 00000100:00100000:3.0:1780298642.862467:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f737b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609424000:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 74us (127us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298642.862471:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9094 00000100:00100000:0.0:1780298642.863253:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.863277:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609424128 00000100:00100000:3.0:1780298642.863281:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609424128 00002000:00100000:3.0:1780298642.863313:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (819200->831487) req@ffff9cf37f736680 x1866777609424128/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298642.863319:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f736680 with x1866777609424128 ext(819200->831487) 00002000:00010000:3.0:1780298642.863323:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f736680 00000100:00100000:3.0:1780298642.863338:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6050 00000100:00100000:3.0:1780298642.863343:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f736680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609424128:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298642.863461:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f736680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609424128:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 117us (207us total) trans 0 rc 12288/12288 00000100:00100000:3.0:1780298642.863468:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6050 00002000:00100000:3.0:1780298642.863475:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (819200->831487) req@ffff9cf37f736680 x1866777609424128/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/12728 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 12288/12288 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298642.863484:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f736680 with x1866777609424128 ext(819200->831487) 00002000:00010000:3.0:1780298642.863487:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f736680 00000100:00100000:0.0:1780298642.864418:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.864441:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.864464:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609424256 00000100:00100000:2.0:1780298642.864470:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609424256 00000100:00100000:1.0:1780298642.864476:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609424384 00000100:00100000:1.0:1780298642.864480:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609424384 00000100:00100000:2.0:1780298642.864504:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9095 00000100:00100000:3.0:1780298642.864518:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9096 00000100:00100000:2.0:1780298642.864520:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f736300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609424256:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.864528:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298642.864533:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f736a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609424384:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.864537:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005d10d3aa/0xaf35132e630337ef lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d29ad expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.864544:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.864555:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e2336867/0xaf35132e630337f6 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d29b4 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.864567:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b01500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604909312:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.864593:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604909440:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.864619:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.864624:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.865383:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604909440:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.865383:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b01500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604909312:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.865411:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005d10d3aa/0xaf35132e630337ef lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29ad expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.865414:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e2336867/0xaf35132e630337f6 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29b4 expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.865438:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005d10d3aa/0xaf35132e630337ef lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29ad expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.865442:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e2336867/0xaf35132e630337f6 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29b4 expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.865468:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf36d5a9200, rc 0) 00010000:00010000:2.0:1780298642.865469:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baec00, rc 0) 00000100:00100000:3.0:1780298642.865504:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f736a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609424384:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 975us (1060us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.865505:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f736300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609424256:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 993us (1085us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.865521:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9096 00000100:00100000:2.0:1780298642.865528:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9095 00000100:00100000:0.0:1780298642.866407:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.866433:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609424512 00000100:00100000:0.0:1780298642.866437:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.866438:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609424512 00000100:00100000:2.0:1780298642.866462:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609424640 00000100:00100000:2.0:1780298642.866464:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609424640 00000100:00100000:3.0:1780298642.866472:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9097 00000100:00100000:3.0:1780298642.866483:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f737480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609424512:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298642.866488:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9098 00010000:00010000:3.0:1780298642.866495:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298642.866500:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f737800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609424640:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.866505:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fb9fa441/0xaf35132e630337fd lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d29bb expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.866510:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.866522:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e9b43ec/0xaf35132e63033804 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d29c2 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.866538:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604909568:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.866558:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604909696:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.866588:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.866590:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417d80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.867235:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604909568:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.867261:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000fb9fa441/0xaf35132e630337fd lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29bb expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.867272:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604909696:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.867278:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fb9fa441/0xaf35132e630337fd lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29bb expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.867300:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf36d5a9800, rc 0) 00010000:00010000:1.0:1780298642.867302:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004e9b43ec/0xaf35132e63033804 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29c2 expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.867314:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e9b43ec/0xaf35132e63033804 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29c2 expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.867323:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2a00, rc 0) 00000100:00100000:3.0:1780298642.867333:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f737480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609424512:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 852us (924us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.867339:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9097 00000100:00100000:1.0:1780298642.867350:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f737800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609424640:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 848us (907us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.867355:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9098 00000100:00100000:1.0:1780298642.868228:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.868281:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609424768 00000100:00100000:3.0:1780298642.868286:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609424768 00000100:00100000:3.0:1780298642.868327:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9099 00000100:00100000:3.0:1780298642.868336:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b41880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609424768:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.868344:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.868352:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f8257a7e/0xaf35132e6303380b lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d29c9 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.868364:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.868387:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604909824:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.868395:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609424896 00000100:00100000:2.0:1780298642.868399:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609424896 00000100:00100000:2.0:1780298642.868419:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9100 00000100:00100000:2.0:1780298642.868428:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609424896:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.868434:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.868442:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007b6539a7/0xaf35132e63033812 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d29d0 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.868463:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b01500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604909952:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.868472:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.868487:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.869083:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604909824:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.869097:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f8257a7e/0xaf35132e6303380b lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29c9 expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.869107:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f8257a7e/0xaf35132e6303380b lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29c9 expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.869114:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf36d5a8a00, rc 0) 00000100:00100000:3.0:1780298642.869168:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b41880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609424768:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 833us (938us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.869194:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b01500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604909952:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.869212:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9099 00010000:00010000:2.0:1780298642.869219:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007b6539a7/0xaf35132e63033812 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29d0 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.869235:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007b6539a7/0xaf35132e63033812 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29d0 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.869250:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae800, rc 0) 00000100:00100000:2.0:1780298642.869281:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609424896:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 856us (920us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.869301:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9100 00000100:00100000:0.0:1780298642.870059:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.870089:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.870106:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609425024 00000100:00100000:3.0:1780298642.870112:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609425152 00000100:00100000:1.0:1780298642.870112:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609425024 00000100:00100000:3.0:1780298642.870117:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609425152 00000100:00100000:3.0:1780298642.870173:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9101 00000100:00100000:3.0:1780298642.870182:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e582850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609425152:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.870188:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298642.870188:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9102 00010000:00010000:3.0:1780298642.870197:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000034be4f60/0xaf35132e63033819 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d29de expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.870199:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609425024:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.870210:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.870218:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a269a887/0xaf35132e63033820 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d29d7 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.870240:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604910080:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.870271:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b01500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604910208:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.870274:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417540 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.870314:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4000 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.871022:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604910080:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.871034:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b01500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604910208:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.871044:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a269a887/0xaf35132e63033820 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29d7 expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.871051:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000034be4f60/0xaf35132e63033819 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29de expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.871069:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a269a887/0xaf35132e63033820 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29d7 expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.871073:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000034be4f60/0xaf35132e63033819 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29de expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.871087:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3e00, rc 0) 00010000:00010000:2.0:1780298642.871099:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf36d5a8400, rc 0) 00000100:00100000:0.0:1780298642.871121:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609425024:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 925us (1062us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.871154:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e582850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609425152:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 946us (1036us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.871154:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9102 00000100:00100000:2.0:1780298642.871166:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9101 00000100:00100000:1.0:1780298642.872628:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.872665:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609425408 00000100:00100000:2.0:1780298642.872670:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609425408 00000100:00100000:2.0:1780298642.872708:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61957 00000100:00100000:2.0:1780298642.872718:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b41500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+119:29360:x1866777609425408:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:2.0:1780298642.872751:0:6477:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298642.872766:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841c800/0xaf35132e63033708 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 65 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28cd expref: 119 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.872777:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.872780:0:6477:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298642.872904:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafc00/0xaf35132e63033827 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 64 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.872914:0:6477:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baee00/0xaf35132e6303371d lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 64 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28db expref: 102 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.872928:0:6477:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafc00/0xaf35132e63033827 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 64 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.872944:0:6477:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baee00/0xaf35132e6303371d lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 64 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28db expref: 102 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.872954:0:6477:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baee00/0xaf35132e6303371d lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 64 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28db expref: 102 pid: 9976 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.872963:0:6477:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b00000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6477:1866777604910336:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298642.873009:0:6477:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4900 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.873590:0:6477:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b00000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6477:1866777604910336:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298642.873601:0:6477:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafc00/0xaf35132e63033827 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 64 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.873797:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.873833:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609425536 00000100:00100000:0.0:1780298642.873838:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609425536 00010000:00010000:0.0:1780298642.873859:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baee00/0xaf35132e6303371d lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 64 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28db expref: 103 pid: 9976 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.873887:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15039 00000100:00100000:0.0:1780298642.873897:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b40380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+103:2399:x1866777609425536:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298642.873906:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298642.873911:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baee00/0xaf35132e6303371d lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 65 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28db expref: 103 pid: 9976 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.873922:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baee00/0xaf35132e6303371d lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 65 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28db expref: 103 pid: 9976 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.873936:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baee00/0xaf35132e6303371d lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 65 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28db expref: 103 pid: 9976 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.873949:0:8374:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298642.873953:0:8374:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafc00/0xaf35132e63033827 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 64 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.873964:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafc00/0xaf35132e63033827 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 64 type: IBT flags: 0x40290000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.873972:0:8374:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafc00/0xaf35132e63033827 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 64 type: IBT flags: 0x40290000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.873992:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298642.874010:0:6477:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafc00/0xaf35132e63033827 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 63 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.874017:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafc00/0xaf35132e63033827 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 63 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.874023:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b40380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+102:2399:x1866777609425536:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 126us (224us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.874032:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15039 00000001:00080000:2.0:1780298642.874064:0:6477:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991030 00000004:00010000:2.0:1780298642.874082:0:6477:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000c8fd47a0 reply state 0000000021c78cc3 transno 4294991031 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafc00/0xaf35132e63033827 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 63 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.874092:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafc00/0xaf35132e63033827 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 63 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.874100:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.874102:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1780298642.874105:0:6477:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1780298642.874180:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b41500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+121:29360:x1866777609425408:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1464us (1552us total) trans 4294991031 rc 0/0 00000100:00100000:2.0:1780298642.874186:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61957 00000020:00080000:2.0:1780298642.874189:0:6477:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1780298642.874972:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.875011:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609425664 00000100:00100000:0.0:1780298642.875016:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609425664 00000100:00100000:0.0:1780298642.875046:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9103 00000100:00100000:0.0:1780298642.875054:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b40a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609425664:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.875062:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.875073:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e6303382e lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d29e5 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.875094:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d5524779/0xaf35132e630337a9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3067904->3416063] (req 3067904->3071999) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2967 expref: 7 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.875103:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000019d620bb/0xaf35132e6303375c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3416064->36970495] (req 3416064->3424255) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d291a expref: 7 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.875112:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000004cd65b27/0xaf35132e630337e8 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [819200->3067903] (req 819200->831487) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29a6 expref: 10 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.875120:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 3 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(126 177) 00010000:00010000:0.0:1780298642.875147:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(129 177) 00010000:00010000:0.0:1780298642.875154:0:20252:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e6303382e lrc: 6/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29e5 expref: 10 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.875175:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000004cd65b27/0xaf35132e630337e8 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [819200->3067903] (req 819200->831487) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29a6 expref: 10 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.875187:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000004cd65b27/0xaf35132e630337e8 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [819200->3067903] (req 819200->831487) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29a6 expref: 10 pid: 12101 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.875206:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604910464:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298642.875256:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000019d620bb/0xaf35132e6303375c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3416064->36970495] (req 3416064->3424255) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d291a expref: 7 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.875267:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000019d620bb/0xaf35132e6303375c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3416064->36970495] (req 3416064->3424255) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d291a expref: 7 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.875278:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604910592:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298642.875330:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d5524779/0xaf35132e630337a9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3067904->3416063] (req 3067904->3071999) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2967 expref: 7 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.875341:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d5524779/0xaf35132e630337a9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3067904->3416063] (req 3067904->3071999) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2967 expref: 7 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.875350:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481ffb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604910720:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298642.875385:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417540 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.875824:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604910464:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298642.875943:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604910592:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298642.876079:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481ffb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604910720:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298642.876101:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e6303382e lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29e5 expref: 10 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.876119:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2000, rc 0) 00000100:00100000:0.0:1780298642.876176:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b40a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609425664:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1122us (1202us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.876184:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9103 00000100:00100000:1.0:1780298642.876195:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.876233:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.876267:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609425920 00000100:00100000:0.0:1780298642.876272:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609425920 00000100:00100000:3.0:1780298642.876303:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609425792 00010000:00010000:0.0:1780298642.876303:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d5524779/0xaf35132e630337a9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3067904->3416063] (req 3067904->3071999) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2967 expref: 8 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.876308:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609425792 00010000:00010000:3.0:1780298642.876326:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000004cd65b27/0xaf35132e630337e8 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [819200->3067903] (req 819200->831487) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29a6 expref: 10 pid: 12101 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.876336:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.876341:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15040 00000100:00100000:0.0:1780298642.876352:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b41f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609425920:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298642.876362:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1780298642.876363:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609426048 00000100:00100000:3.0:1780298642.876367:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15041 00000100:00100000:1.0:1780298642.876370:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609426048 00000100:00100000:3.0:1780298642.876378:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b00850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609425792:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298642.876378:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d5524779/0xaf35132e630337a9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [3067904->3416063] (req 3067904->3071999) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2967 expref: 9 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.876388:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298642.876393:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000019d620bb/0xaf35132e6303375c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [3416064->36970495] (req 3416064->3424255) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d291a expref: 9 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.876393:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d5524779/0xaf35132e630337a9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 8 type: EXT [3067904->3416063] (req 3067904->3071999) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2967 expref: 9 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.876420:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000004cd65b27/0xaf35132e630337e8 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [819200->3067903] (req 819200->831487) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29a6 expref: 10 pid: 12101 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.876428:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d5524779/0xaf35132e630337a9 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 8 type: EXT [3067904->3416063] (req 3067904->3071999) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2967 expref: 9 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.876433:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000004cd65b27/0xaf35132e630337e8 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 8 type: EXT [819200->3067903] (req 819200->831487) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29a6 expref: 10 pid: 12101 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.876442:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15042 00010000:00010000:0.0:1780298642.876450:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298642.876454:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b00000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609426048:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298642.876458:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004cd65b27/0xaf35132e630337e8 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [819200->3067903] (req 819200->831487) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29a6 expref: 10 pid: 12101 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.876464:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298642.876468:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298642.876475:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000019d620bb/0xaf35132e6303375c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3416064->36970495] (req 3416064->3424255) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d291a expref: 8 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.876483:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b41f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609425920:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 133us (290us total) trans 0 rc 0/0 00010000:00010000:1.0:1780298642.876487:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000019d620bb/0xaf35132e6303375c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3416064->36970495] (req 3416064->3424255) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d291a expref: 8 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.876507:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b00850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609425792:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 131us (272us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.876508:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15040 00010000:00010000:1.0:1780298642.876510:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000019d620bb/0xaf35132e6303375c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3416064->36970495] (req 3416064->3424255) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d291a expref: 8 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.876520:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(129 177) 00000100:00100000:2.0:1780298642.876525:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15041 00010000:00010000:1.0:1780298642.876526:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(129 177) 00010000:00010000:1.0:1780298642.876532:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e6303382e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29e5 expref: 9 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.876538:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e6303382e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29e5 expref: 8 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.876555:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e6303382e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29e5 expref: 8 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.876563:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777604910848:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298642.876595:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.877207:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777604910848:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298642.877215:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298642.877233:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b00000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609426048:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 784us (904us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.877239:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15042 00000100:00100000:0.0:1780298642.877349:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.877384:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609426176 00000100:00100000:2.0:1780298642.877389:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609426176 00000100:00100000:2.0:1780298642.877417:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9104 00000100:00100000:2.0:1780298642.877423:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481ffb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:29360:x1866777609426176:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.877435:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.877443:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63033835 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d29ec expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.877456:0:19992:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e6303370f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [733184->34287615] (req 733184->745471) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28d4 expref: 8 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.877461:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(91 130) 00010000:00010000:2.0:1780298642.877464:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(92 130) 00010000:00010000:2.0:1780298642.877467:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63033835 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29ec expref: 13 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.877479:0:19992:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e6303370f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [733184->34287615] (req 733184->745471) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28d4 expref: 8 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.877489:0:19992:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e6303370f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [733184->34287615] (req 733184->745471) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28d4 expref: 8 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.877495:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b01500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604910976:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298642.877523:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.878000:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b01500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604910976:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298642.878015:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63033835 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29ec expref: 13 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.878023:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae000, rc 0) 00000100:00100000:2.0:1780298642.878047:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481ffb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:29360:x1866777609426176:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 625us (699us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.878051:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9104 00000100:00100000:1.0:1780298642.878266:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.878322:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609426304 00000100:00100000:2.0:1780298642.878327:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609426304 00002000:00100000:2.0:1780298642.878351:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (2519040->2523135) req@ffff9cf242b43100 x1866777609426304/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 4584/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.878357:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b43100 with x1866777609426304 ext(2519040->2523135) 00010000:00010000:2.0:1780298642.878362:0:19951:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e6303370f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [733184->34287615] (req 733184->745471) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28d4 expref: 9 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.878370:0:19951:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e6303370f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [733184->34287615] (req 733184->745471) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28d4 expref: 9 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1780298642.878375:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9cf242b43100 00000100:00100000:2.0:1780298642.878391:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 189 00000100:00100000:2.0:1780298642.878396:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b43100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609426304:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:2.0:1780298642.878769:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609426304:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 372us (502us total) trans 4294969206 rc 0/0 00000100:00100000:2.0:1780298642.878773:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 189 00002000:00100000:2.0:1780298642.878778:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (2519040->2523135) req@ffff9cf242b43100 x1866777609426304/t4294969206(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 4584/448 e 0 to 0 dl 1780298653 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.878783:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b43100 with x1866777609426304 ext(2519040->2523135) 00010000:00010000:2.0:1780298642.878787:0:19951:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e6303370f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [733184->34287615] (req 733184->745471) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28d4 expref: 11 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.878799:0:19951:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e6303370f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [733184->34287615] (req 733184->745471) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28d4 expref: 11 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1780298642.878806:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9cf242b43100 00000100:00100000:0.0:1780298642.879472:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.879512:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609426432 00000100:00100000:2.0:1780298642.879517:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609426432 00010000:00010000:2.0:1780298642.879534:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e6303370f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [733184->34287615] (req 733184->745471) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28d4 expref: 11 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.879560:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15043 00000100:00100000:2.0:1780298642.879568:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fc380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609426432:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298642.879577:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:2.0:1780298642.879584:0:6463:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 1 -> 8 00010000:00010000:2.0:1780298642.879590:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e6303370f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [733184->34287615] (req 733184->745471) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28d4 expref: 11 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.879601:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e6303370f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [733184->34287615] (req 733184->745471) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28d4 expref: 11 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.879615:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e6303370f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [733184->34287615] (req 733184->745471) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d28d4 expref: 11 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.879627:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(92 130) 00010000:00010000:2.0:1780298642.879631:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(92 130) 00010000:00010000:2.0:1780298642.879636:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63033835 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29ec expref: 12 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.879644:0:6463:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63033835 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29ec expref: 12 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.879672:0:6463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63033835 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29ec expref: 12 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.879683:0:6463:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b01500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777604911104:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298642.879724:0:6463:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.880322:0:6463:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b01500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777604911104:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298642.880334:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298642.880359:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fc380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609426432:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 793us (887us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.880368:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15043 00000100:00100000:1.0:1780298642.880506:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.880524:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.880541:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609426560 00000100:00100000:2.0:1780298642.880546:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609426560 00000100:00100000:0.0:1780298642.880548:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609426688 00000100:00100000:0.0:1780298642.880553:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609426688 00002000:00010000:2.0:1780298642.880561:0:19949:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (5838314->18446744073709551615) 00002000:00100000:2.0:1780298642.880565:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b43800 with x1866777609426560 ext(5838314->18446744073709551615) 00002000:00010000:0.0:1780298642.880566:0:19950:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4194304->18446744073709551615) 00002000:00100000:0.0:1780298642.880569:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b41180 with x1866777609426688 ext(4194304->18446744073709551615) 00002000:00010000:2.0:1780298642.880570:0:19949:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf242b43800. 00002000:00010000:0.0:1780298642.880574:0:19950:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf242b41180. 00000100:00100000:2.0:1780298642.880590:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6051 00000100:00100000:2.0:1780298642.880595:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b43800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609426560:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:0.0:1780298642.880610:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6052 00000100:00100000:0.0:1780298642.880621:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b41180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609426688:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:0.0:1780298642.880716:0:19950:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 2855056 -> 4194304 00002000:00010000:0.0:1780298642.880719:0:19950:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:0.0:1780298642.880722:0:19950:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:0.0:1780298642.880725:0:19950:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:0.0:1780298642.880727:0:19950:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 8 -> 8 00002000:00010000:2.0:1780298642.880743:0:19949:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 5838314 00002000:00010000:2.0:1780298642.880748:0:19949:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:2.0:1780298642.880751:0:19949:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:2.0:1780298642.880756:0:19949:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00000100:00100000:0.0:1780298642.880756:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b41180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609426688:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 137us (228us total) trans 4294969207 rc 0/0 00002000:00010000:2.0:1780298642.880761:0:19949:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 384 -> 384 00000100:00100000:0.0:1780298642.880770:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6052 00002000:00010000:0.0:1780298642.880774:0:19950:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4194304->18446744073709551615) 00002000:00100000:0.0:1780298642.880777:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b41180 with x1866777609426688 ext(4194304->18446744073709551615) 00002000:00010000:0.0:1780298642.880783:0:19950:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf242b41180. 00000100:00100000:2.0:1780298642.880794:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609426560:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 196us (285us total) trans 4294969647 rc 0/0 00000100:00100000:2.0:1780298642.880802:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6051 00002000:00010000:2.0:1780298642.880806:0:19949:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (5838314->18446744073709551615) 00002000:00100000:2.0:1780298642.880809:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b43800 with x1866777609426560 ext(5838314->18446744073709551615) 00002000:00010000:2.0:1780298642.880814:0:19949:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf242b43800. 00000100:00100000:0.0:1780298642.881534:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.881561:0:7411:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609426816 00000100:00100000:2.0:1780298642.881565:0:7411:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609426816 00000100:00100000:2.0:1780298642.881590:0:7411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61958 00000100:00100000:2.0:1780298642.881595:0:7411:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+102:29360:x1866777609426816:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.881603:0:7411:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.881612:0:7411:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf600/0xaf35132e6303383c lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 64 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d29f3 expref: -99 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.881664:0:7411:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf200/0xaf35132e63033843 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 65 type: IBT flags: 0x40000000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.881673:0:7411:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf200/0xaf35132e63033843 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 65 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.881680:0:7411:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf200/0xaf35132e63033843 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 65 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298642.881695:0:7411:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 10032618 00000004:00200000:2.0:1780298642.881697:0:7411:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298642.881709:0:7411:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf200/0xaf35132e63033843 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 65 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.881722:0:7411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf600/0xaf35132e6303383c lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 65 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29f3 expref: 104 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.881735:0:7411:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf200/0xaf35132e63033843 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 64 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29f3 expref: 103 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.881740:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.881743:0:7411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298642.881744:0:7411:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf200, rc 0) 00000100:00100000:2.0:1780298642.881766:0:7411:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+103:29360:x1866777609426816:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 171us (232us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.881770:0:7411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61958 00000100:00100000:3.0:1780298642.882534:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.882578:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609426944 00000100:00100000:1.0:1780298642.882583:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609426944 00000100:00100000:3.0:1780298642.882586:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.882623:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609427072 00000100:00100000:1.0:1780298642.882627:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609427072 00000100:00100000:0.0:1780298642.882632:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9105 00000100:00100000:0.0:1780298642.882642:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609426944:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.882653:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298642.882656:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9106 00010000:00010000:0.0:1780298642.882662:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001d4afe0b/0xaf35132e6303384a lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d29fa expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.882668:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609427072:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.882679:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.882689:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008be5bdd7/0xaf35132e63033851 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2a01 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.882696:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604911232:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.882725:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b42300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604911360:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.882739:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417540 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.882756:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.883419:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604911232:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.883436:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001d4afe0b/0xaf35132e6303384a lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29fa expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.883447:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001d4afe0b/0xaf35132e6303384a lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29fa expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.883454:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3000, rc 0) 00000100:00100000:0.0:1780298642.883480:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609426944:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 841us (989us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.883487:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9105 00000100:00100000:1.0:1780298642.883498:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b42300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604911360:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.883516:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008be5bdd7/0xaf35132e63033851 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a01 expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.883532:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008be5bdd7/0xaf35132e63033851 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a01 expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.883542:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341499c00, rc 0) 00000100:00100000:1.0:1780298642.883573:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609427072:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 909us (989us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.883580:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9106 00000100:00100000:3.0:1780298642.884440:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.884475:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609427200 00000100:00100000:0.0:1780298642.884478:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609427200 00000100:00100000:3.0:1780298642.884485:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.884509:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609427328 00000100:00100000:0.0:1780298642.884511:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609427328 00000100:00100000:1.0:1780298642.884521:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9107 00000100:00100000:1.0:1780298642.884530:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609427200:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.884537:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298642.884538:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9108 00000100:00100000:0.0:1780298642.884547:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609427328:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.884548:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bb478d6d/0xaf35132e63033858 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2a08 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.884555:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.884564:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5a492f0/0xaf35132e6303385f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2a0f expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.884581:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b42300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604911488:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.884589:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604911616:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.884630:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.884630:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417540 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.885352:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b42300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604911488:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.885365:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bb478d6d/0xaf35132e63033858 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a08 expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.885375:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bb478d6d/0xaf35132e63033858 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a08 expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.885383:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341498c00, rc 0) 00000100:00100000:0.0:1780298642.885390:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604911616:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.885407:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609427200:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 879us (966us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.885412:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9107 00010000:00010000:0.0:1780298642.885417:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a5a492f0/0xaf35132e6303385f lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a0f expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.885432:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5a492f0/0xaf35132e6303385f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a0f expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.885443:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3600, rc 0) 00000100:00100000:0.0:1780298642.885467:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609427328:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 922us (982us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.885475:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9108 00000100:00100000:2.0:1780298642.886440:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.886472:0:6476:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609427584 00000100:00100000:1.0:1780298642.886476:0:6476:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609427584 00000100:00100000:1.0:1780298642.886502:0:6476:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61959 00000100:00100000:1.0:1780298642.886508:0:6476:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b01500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+121:29360:x1866777609427584:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:1.0:1780298642.886662:0:6476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499e00/0xaf35132e63033866 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 65 type: IBT flags: 0x40000000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.886672:0:6476:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf200/0xaf35132e63033843 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 65 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29f3 expref: 102 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.886687:0:6476:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499e00/0xaf35132e63033866 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 65 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.886709:0:6476:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf200/0xaf35132e63033843 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 65 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29f3 expref: 102 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.886722:0:6476:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf200/0xaf35132e63033843 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 65 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29f3 expref: 102 pid: 7411 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.886732:0:6476:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b42300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6476:1866777604911744:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298642.886762:0:6476:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439c00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.887316:0:6476:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b42300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6476:1866777604911744:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298642.887324:0:6476:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499e00/0xaf35132e63033866 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 65 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.887507:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.887537:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609427712 00000100:00100000:0.0:1780298642.887541:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609427712 00010000:00010000:0.0:1780298642.887556:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf200/0xaf35132e63033843 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 65 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29f3 expref: 103 pid: 7411 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.887583:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15044 00000100:00100000:0.0:1780298642.887590:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b03b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+103:2400:x1866777609427712:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298642.887598:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298642.887602:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf200/0xaf35132e63033843 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 66 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29f3 expref: 103 pid: 7411 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.887613:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf200/0xaf35132e63033843 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 66 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29f3 expref: 103 pid: 7411 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.887623:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf200/0xaf35132e63033843 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 66 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29f3 expref: 103 pid: 7411 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.887633:0:8374:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298642.887637:0:8374:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499e00/0xaf35132e63033866 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 65 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.887648:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499e00/0xaf35132e63033866 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 65 type: IBT flags: 0x40290000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.887655:0:8374:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499e00/0xaf35132e63033866 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 65 type: IBT flags: 0x40290000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.887673:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298642.887688:0:6476:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499e00/0xaf35132e63033866 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 64 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.887695:0:6476:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499e00/0xaf35132e63033866 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 64 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.887704:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b03b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+102:2400:x1866777609427712:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 109us (190us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.887711:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15044 00000001:00080000:1.0:1780298642.887747:0:6476:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991032 00000004:00010000:1.0:1780298642.887767:0:6476:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 000000000ead597a reply state 000000007387a1fc transno 4294991033 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499e00/0xaf35132e63033866 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 64 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.887776:0:6476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499e00/0xaf35132e63033866 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 64 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.887780:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298642.887782:0:6476:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1780298642.887784:0:6476:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1780298642.887828:0:6476:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b01500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+124:29360:x1866777609427584:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1320us (1389us total) trans 4294991033 rc 0/0 00000100:00100000:1.0:1780298642.887833:0:6476:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61959 00000020:00080000:1.0:1780298642.887837:0:6476:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1780298642.888519:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.888552:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609427840 00000100:00100000:1.0:1780298642.888556:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609427840 00000100:00100000:2.0:1780298642.888567:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00002000:00010000:1.0:1780298642.888574:0:8362:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (2499556->18446744073709551615) 00002000:00100000:1.0:1780298642.888578:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf241b02d80 with x1866777609427840 ext(2499556->18446744073709551615) 00002000:00010000:1.0:1780298642.888585:0:8362:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf241b02d80. 00000100:00100000:1.0:1780298642.888599:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609427968 00000100:00100000:1.0:1780298642.888603:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609427968 00002000:00010000:1.0:1780298642.888620:0:8362:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00000100:00100000:0.0:1780298642.888620:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6053 00002000:00100000:1.0:1780298642.888627:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf241b00380 with x1866777609427968 ext(0->18446744073709551615) 00002000:00010000:1.0:1780298642.888632:0:8362:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf241b00380. 00000100:00100000:0.0:1780298642.888636:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609427840:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:1.0:1780298642.888655:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6054 00000100:00100000:1.0:1780298642.888664:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b00380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609427968:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:0.0:1780298642.890205:0:8364:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 5838314 -> 2499556 00002000:00010000:0.0:1780298642.890209:0:8364:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:0.0:1780298642.890211:0:8364:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:0.0:1780298642.890212:0:8364:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:0.0:1780298642.890215:0:8364:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 384 -> 344 00000100:00100000:0.0:1780298642.890240:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609427840:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 1610us (1722us total) trans 4294969648 rc 0/0 00000100:00100000:0.0:1780298642.890247:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6053 00002000:00010000:0.0:1780298642.890251:0:8364:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (2499556->18446744073709551615) 00002000:00100000:0.0:1780298642.890255:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf241b02d80 with x1866777609427840 ext(2499556->18446744073709551615) 00002000:00010000:0.0:1780298642.890260:0:8364:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf241b02d80. 00002000:00010000:1.0:1780298642.890754:0:8362:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4194304 -> 0 00002000:00010000:1.0:1780298642.890758:0:8362:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298642.890760:0:8362:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298642.890762:0:8362:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298642.890764:0:8362:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 8 -> 0 00000100:00100000:1.0:1780298642.890788:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b00380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2400:x1866777609427968:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 2127us (2221us total) trans 4294969208 rc 0/0 00000100:00100000:1.0:1780298642.890793:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6054 00002000:00010000:1.0:1780298642.890796:0:8362:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00002000:00100000:1.0:1780298642.890798:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf241b00380 with x1866777609427968 ext(0->18446744073709551615) 00002000:00010000:1.0:1780298642.890802:0:8362:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf241b00380. 00000100:00100000:2.0:1780298642.891507:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.891545:0:9976:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609428096 00000100:00100000:0.0:1780298642.891550:0:9976:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609428096 00000100:00100000:0.0:1780298642.891582:0:9976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61960 00000100:00100000:0.0:1780298642.891591:0:9976:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b02300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+124:29360:x1866777609428096:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.891602:0:9976:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.891615:0:9976:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2e00/0xaf35132e6303386d lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 65 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2a16 expref: -99 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.891673:0:9976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3a00/0xaf35132e63033874 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 66 type: IBT flags: 0x40000000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.891681:0:9976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3a00/0xaf35132e63033874 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 66 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.891687:0:9976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3a00/0xaf35132e63033874 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 66 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1780298642.891703:0:9976:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 2499556 00000004:00200000:0.0:1780298642.891707:0:9976:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1780298642.891720:0:9976:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3a00/0xaf35132e63033874 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 66 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.891744:0:9976:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2e00/0xaf35132e6303386d lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 66 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a16 expref: 126 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.891756:0:9976:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3a00/0xaf35132e63033874 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 65 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a16 expref: 125 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.891761:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298642.891763:0:9976:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1780298642.891765:0:9976:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3a00, rc 0) 00000100:00100000:0.0:1780298642.891786:0:9976:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b02300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+125:29360:x1866777609428096:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 199us (279us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.891790:0:9976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61960 00000100:00100000:3.0:1780298642.896170:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.896210:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609428224 00000100:00100000:2.0:1780298642.896216:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609428224 00002000:00100000:2.0:1780298642.896245:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x2c0000401:0x11:0x0] (0->2502655) req@ffff9cf24519c700 x1866777609428224/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.896252:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24519c700 with x1866777609428224 ext(0->2502655) 00002000:00010000:2.0:1780298642.896256:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24519c700 00000100:00100000:2.0:1780298642.896273:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6055 00000100:00100000:2.0:1780298642.896279:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609428224:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298642.901330:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609428224:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 5040us (5152us total) trans 0 rc 2502656/2502656 00000100:00100000:1.0:1780298642.901340:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6055 00002000:00100000:1.0:1780298642.901352:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x2c0000401:0x11:0x0] (0->2502655) req@ffff9cf24519c700 x1866777609428224/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/440 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 2502656/2502656 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298642.901359:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24519c700 with x1866777609428224 ext(0->2502655) 00002000:00010000:1.0:1780298642.901364:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24519c700 00000100:00100000:3.0:1780298642.903144:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.903193:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609428352 00000100:00100000:2.0:1780298642.903203:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609428352 00000100:00100000:2.0:1780298642.903255:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61961 00000100:00100000:2.0:1780298642.903267:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+102:29360:x1866777609428352:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.903299:0:6477:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.903320:0:6477:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae400/0xaf35132e6303387b lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 66 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2a1d expref: -99 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.903409:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf400/0xaf35132e63033882 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 67 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.903432:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf400/0xaf35132e63033882 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 67 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.903444:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf400/0xaf35132e63033882 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 67 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298642.903470:0:6477:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 2499556 00000004:00200000:2.0:1780298642.903474:0:6477:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298642.903490:0:6477:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf400/0xaf35132e63033882 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 67 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.903513:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae400/0xaf35132e6303387b lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 67 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a1d expref: 104 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.903533:0:6477:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf400/0xaf35132e63033882 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 66 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a1d expref: 103 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.903542:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.903545:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298642.903548:0:6477:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf400, rc 0) 00000100:00100000:2.0:1780298642.903581:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+103:29360:x1866777609428352:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 317us (461us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.903591:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61961 00000100:00100000:0.0:1780298642.904444:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.904477:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609428480 00000100:00100000:3.0:1780298642.904481:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609428480 00000100:00100000:3.0:1780298642.904506:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9109 00000100:00100000:3.0:1780298642.904511:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609428480:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.904572:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.904586:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fb9fa441/0xaf35132e63033889 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2a24 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.904615:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.904652:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609428608 00000100:00100000:0.0:1780298642.904658:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609428608 00000100:00100000:3.0:1780298642.904659:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604911872:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.904704:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9110 00000100:00100000:0.0:1780298642.904720:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609428608:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298642.904724:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00010000:00010000:0.0:1780298642.904730:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.904743:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000070f5434e/0xaf35132e63033890 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2a2b expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.904775:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604912000:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.904814:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.905439:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604911872:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.905460:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000fb9fa441/0xaf35132e63033889 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a24 expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.905467:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604912000:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.905477:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fb9fa441/0xaf35132e63033889 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a24 expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.905485:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf36d5a9800, rc 0) 00010000:00010000:0.0:1780298642.905489:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000070f5434e/0xaf35132e63033890 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a2b expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.905507:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000070f5434e/0xaf35132e63033890 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a2b expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.905515:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243376e00, rc 0) 00000100:00100000:3.0:1780298642.905517:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609428480:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1004us (1072us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.905526:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9109 00000100:00100000:0.0:1780298642.905537:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609428608:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 822us (924us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.905542:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9110 00000100:00100000:1.0:1780298642.906370:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.906389:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.906391:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609428736 00000100:00100000:3.0:1780298642.906395:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609428736 00000100:00100000:3.0:1780298642.906425:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9111 00000100:00100000:2.0:1780298642.906426:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609428864 00000100:00100000:2.0:1780298642.906431:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609428864 00000100:00100000:3.0:1780298642.906432:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b40000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609428736:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.906439:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.906447:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004cd65b27/0xaf35132e63033897 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2a32 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.906460:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9112 00000100:00100000:2.0:1780298642.906468:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b41880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609428864:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298642.906471:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604912128:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.906478:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.906486:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d4d8e3d7/0xaf35132e6303389e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2a39 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.906511:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.906512:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b03800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604912256:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.906540:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.907165:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b03800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604912256:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.907176:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604912128:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.907185:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d4d8e3d7/0xaf35132e6303389e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a39 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.907195:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004cd65b27/0xaf35132e63033897 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a32 expref: 7 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.907203:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d4d8e3d7/0xaf35132e6303389e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a39 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.907215:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004cd65b27/0xaf35132e63033897 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a32 expref: 7 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.907217:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf000, rc 0) 00010000:00010000:3.0:1780298642.907228:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf36d5a8600, rc 0) 00000100:00100000:2.0:1780298642.907252:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b41880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609428864:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 784us (861us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298642.907253:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b40000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609428736:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 821us (882us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.907266:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9112 00000100:00100000:3.0:1780298642.907277:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9111 00000100:00100000:1.0:1780298642.908140:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.908195:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609428992 00000100:00100000:3.0:1780298642.908200:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609428992 00000100:00100000:1.0:1780298642.908212:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.908238:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609429120 00000100:00100000:0.0:1780298642.908254:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9113 00000100:00100000:0.0:1780298642.908264:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609428992:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.908273:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.908281:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f1945c34/0xaf35132e630338a5 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2a40 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.908305:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609429120 00000100:00100000:0.0:1780298642.908322:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604912384:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.908344:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9114 00000100:00100000:3.0:1780298642.908355:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609429120:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.908366:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298642.908373:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00010000:00010000:2.0:1780298642.908402:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e630338ac lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2a47 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.908434:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b03800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604912512:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.908458:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.908943:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604912384:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.908959:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f1945c34/0xaf35132e630338a5 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a40 expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.908973:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f1945c34/0xaf35132e630338a5 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a40 expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.908983:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243377600, rc 0) 00000100:00100000:0.0:1780298642.909016:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609428992:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 754us (889us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.909031:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9113 00000100:00100000:2.0:1780298642.909105:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b03800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604912512:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.909121:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e630338ac lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a47 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.909164:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e630338ac lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a47 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.909175:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae400, rc 0) 00000100:00100000:2.0:1780298642.909209:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609429120:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 857us (996us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.909217:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9114 00000100:00100000:3.0:1780298642.909975:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.910013:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609429248 00000100:00100000:0.0:1780298642.910017:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609429248 00000100:00100000:0.0:1780298642.910051:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9115 00000100:00100000:0.0:1780298642.910060:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609429248:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.910073:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.910082:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000008dd0310/0xaf35132e630338b3 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2a4e expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.910099:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.910109:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604912640:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.910118:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609429376 00000100:00100000:3.0:1780298642.910121:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609429376 00000100:00100000:0.0:1780298642.910179:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.910255:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9116 00000100:00100000:2.0:1780298642.910265:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b41f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609429376:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.910273:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.910280:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b35ce980/0xaf35132e630338ba lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2a55 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.910320:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b03800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604912768:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.910353:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.910817:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604912640:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.910828:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000008dd0310/0xaf35132e630338b3 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a4e expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.910837:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000008dd0310/0xaf35132e630338b3 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a4e expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.910844:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243376400, rc 0) 00000100:00100000:0.0:1780298642.910866:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609429248:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 809us (892us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.910871:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9115 00000100:00100000:3.0:1780298642.911003:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b03800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604912768:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.911020:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b35ce980/0xaf35132e630338ba lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a55 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.911076:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b35ce980/0xaf35132e630338ba lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a55 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.911086:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bafa00, rc 0) 00000100:00100000:2.0:1780298642.911170:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b41f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609429376:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 906us (1069us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.911182:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9116 00000100:00100000:2.0:1780298642.911973:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.912001:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609429504 00000100:00100000:0.0:1780298642.912006:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609429504 00000100:00100000:0.0:1780298642.912030:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9117 00000100:00100000:0.0:1780298642.912037:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b01f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:29360:x1866777609429504:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.912044:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.912051:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000070f5434e/0xaf35132e630338c1 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2a5c expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.912064:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e6303382e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29e5 expref: 10 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.912071:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(129 177) 00010000:00010000:0.0:1780298642.912074:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(130 177) 00010000:00010000:0.0:1780298642.912076:0:20252:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000070f5434e/0xaf35132e630338c1 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2080768->2093055] (req 2080768->2093055) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a5c expref: 7 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.912090:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e6303382e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29e5 expref: 10 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.912100:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e6303382e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29e5 expref: 10 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.912107:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604912896:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298642.912150:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.912817:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604912896:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298642.912833:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000070f5434e/0xaf35132e630338c1 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2080768->2093055] (req 2080768->2093055) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a5c expref: 7 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.912844:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243376e00, rc 0) 00000100:00100000:0.0:1780298642.912874:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b01f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609429504:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 838us (901us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.912880:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9117 00000100:00100000:1.0:1780298642.915174:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.915202:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609429632 00000100:00100000:2.0:1780298642.915208:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609429632 00010000:00010000:2.0:1780298642.915235:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e6303382e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29e5 expref: 11 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.915265:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15045 00000100:00100000:2.0:1780298642.915274:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343967b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609429632:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298642.915285:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298642.915312:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e6303382e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29e5 expref: 11 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.915324:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e6303382e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29e5 expref: 11 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.915342:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e6303382e lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29e5 expref: 11 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.915356:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(130 177) 00010000:00010000:2.0:1780298642.915359:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(130 177) 00010000:00010000:2.0:1780298642.915368:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000070f5434e/0xaf35132e630338c1 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2080768->2093055) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a5c expref: 6 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.915376:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000070f5434e/0xaf35132e630338c1 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2080768->2093055) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a5c expref: 6 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.915404:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000070f5434e/0xaf35132e630338c1 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2080768->2093055) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a5c expref: 6 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.915417:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b00a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777604913024:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298642.915468:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.916233:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b00a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777604913024:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298642.916245:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298642.916269:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343967b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609429632:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 998us (1103us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.916276:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15045 00000100:00100000:1.0:1780298642.919789:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.919831:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609429760 00000100:00100000:2.0:1780298642.919839:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609429760 00002000:00100000:2.0:1780298642.919875:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (0->2502655) req@ffff9cf343964000 x1866777609429760/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.919887:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf343964000 with x1866777609429760 ext(0->2502655) 00002000:00010000:2.0:1780298642.919892:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf343964000 00000100:00100000:2.0:1780298642.919916:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6056 00000100:00100000:2.0:1780298642.919927:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343964000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609429760:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298642.925180:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343964000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609429760:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 5243us (5381us total) trans 0 rc 2502656/2502656 00000100:00100000:2.0:1780298642.925195:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6056 00002000:00100000:2.0:1780298642.925210:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (0->2502655) req@ffff9cf343964000 x1866777609429760/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:348/0 lens 488/440 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 2502656/2502656 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.925221:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf343964000 with x1866777609429760 ext(0->2502655) 00002000:00010000:2.0:1780298642.925227:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf343964000 00000100:00100000:1.0:1780298642.927034:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.927085:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609429888 00000100:00100000:2.0:1780298642.927096:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609429888 00000100:00100000:2.0:1780298642.927164:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9118 00000100:00100000:2.0:1780298642.927176:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343966a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609429888:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.927191:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.927210:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e630338c8 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2a63 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.927270:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b03100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604913152:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.927369:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4900 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.928184:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b03100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604913152:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.928210:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e630338c8 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a63 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.928231:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e630338c8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a63 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.928241:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf200, rc 0) 00000100:00100000:2.0:1780298642.928272:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343966a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609429888:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1099us (1255us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.928279:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9118 00000100:00100000:0.0:1780298642.929107:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.929147:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609430016 00000100:00100000:1.0:1780298642.929152:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609430016 00000100:00100000:1.0:1780298642.929184:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9119 00000100:00100000:1.0:1780298642.929190:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609430016:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.929198:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.929205:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ac4408d/0xaf35132e630338cf lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2a6a expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.929228:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343966300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604913280:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.929263:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.929942:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343966300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604913280:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298642.929958:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000ac4408d/0xaf35132e630338cf lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a6a expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.929973:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ac4408d/0xaf35132e630338cf lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a6a expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.929983:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341499200, rc 0) 00000100:00100000:0.0:1780298642.930015:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609430016:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 824us (908us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.930024:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9119 00000100:00100000:0.0:1780298642.930872:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.930911:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609430144 00000100:00100000:2.0:1780298642.930916:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609430144 00000100:00100000:2.0:1780298642.930945:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9120 00000100:00100000:2.0:1780298642.930953:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609430144:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.930961:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.930972:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e630338d6 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2a71 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.930992:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(130 177) 00010000:00010000:2.0:1780298642.930996:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(130 177) 00010000:00010000:2.0:1780298642.931003:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e630338d6 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a71 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.931018:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e630338d6 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a71 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.931027:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf600, rc 0) 00000100:00100000:2.0:1780298642.931053:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609430144:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 102us (182us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.931058:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9120 00000100:00100000:1.0:1780298642.932910:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.932952:0:7411:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609430400 00000100:00100000:2.0:1780298642.932958:0:7411:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609430400 00000100:00100000:2.0:1780298642.932998:0:7411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61962 00000100:00100000:2.0:1780298642.933007:0:7411:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343964e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+125:29360:x1866777609430400:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:2.0:1780298642.933052:0:7411:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298642.933073:0:7411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3a00/0xaf35132e63033874 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 67 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a16 expref: 125 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.933091:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.933096:0:7411:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298642.933298:0:7411:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf800/0xaf35132e630338dd lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 66 type: IBT flags: 0x40000000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.933319:0:7411:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf400/0xaf35132e63033882 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 66 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a1d expref: 102 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.933337:0:7411:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf800/0xaf35132e630338dd lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 66 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.933364:0:7411:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf400/0xaf35132e63033882 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 66 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a1d expref: 102 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.933378:0:7411:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf400/0xaf35132e63033882 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 66 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a1d expref: 102 pid: 6477 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.933392:0:7411:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b03100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7411:1866777604913408:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298642.933451:0:7411:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.934069:0:7411:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b03100 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7411:1866777604913408:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298642.934086:0:7411:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf800/0xaf35132e630338dd lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 66 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.934281:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.934370:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609430528 00000100:00100000:3.0:1780298642.934375:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609430528 00010000:00010000:3.0:1780298642.934396:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf400/0xaf35132e63033882 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 66 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a1d expref: 103 pid: 6477 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.934424:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15046 00000100:00100000:3.0:1780298642.934432:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343966300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+103:2399:x1866777609430528:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298642.934442:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298642.934447:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf400/0xaf35132e63033882 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 67 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a1d expref: 103 pid: 6477 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.934458:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf400/0xaf35132e63033882 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 67 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a1d expref: 103 pid: 6477 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.934473:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf400/0xaf35132e63033882 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 67 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a1d expref: 103 pid: 6477 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.934484:0:7400:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298642.934488:0:7400:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf800/0xaf35132e630338dd lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 66 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.934503:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf800/0xaf35132e630338dd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 66 type: IBT flags: 0x40290000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.934510:0:7400:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf800/0xaf35132e630338dd lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 66 type: IBT flags: 0x40290000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.934530:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298642.934551:0:7411:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf800/0xaf35132e630338dd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 65 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.934561:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343966300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+102:2399:x1866777609430528:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 129us (280us total) trans 0 rc 0/0 00010000:00010000:2.0:1780298642.934561:0:7411:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf800/0xaf35132e630338dd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 65 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.934575:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15046 00000001:00080000:2.0:1780298642.934613:0:7411:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991034 00000004:00010000:2.0:1780298642.934639:0:7411:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 000000005cf31257 reply state 00000000eed9d570 transno 4294991035 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf800/0xaf35132e630338dd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 65 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.934656:0:7411:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf800/0xaf35132e630338dd lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 65 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.934664:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.934666:0:7411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1780298642.934669:0:7411:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1780298642.934732:0:7411:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343964e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+127:29360:x1866777609430400:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1725us (1825us total) trans 4294991035 rc 0/0 00000100:00100000:2.0:1780298642.934740:0:7411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61962 00000020:00080000:2.0:1780298642.934744:0:7411:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1780298642.935510:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.935544:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609430656 00000100:00100000:0.0:1780298642.935549:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609430656 00000100:00100000:0.0:1780298642.935581:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9121 00000100:00100000:0.0:1780298642.935589:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:29360:x1866777609430656:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298642.935599:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298642.935611:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003a994673/0xaf35132e630338e4 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2a78 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.935632:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000070f5434e/0xaf35132e630338c1 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2080768->2093055) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a5c expref: 6 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.935641:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e630338d6 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a71 expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.935648:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(130 177) 00010000:00010000:0.0:1780298642.935652:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(130 177) 00010000:00010000:0.0:1780298642.935657:0:20252:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003a994673/0xaf35132e630338e4 lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a78 expref: 12 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.935677:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e630338d6 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a71 expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.935688:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e630338d6 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a71 expref: 12 pid: 8356 timeout: 932 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.935698:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604913536:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298642.935756:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000070f5434e/0xaf35132e630338c1 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2080768->2093055) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a5c expref: 6 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.935766:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000070f5434e/0xaf35132e630338c1 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2080768->2093055) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a5c expref: 6 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.935782:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fe680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604913664:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298642.935807:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298642.936528:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604913536:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298642.936539:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fe680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604913664:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298642.936557:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003a994673/0xaf35132e630338e4 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a78 expref: 12 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.936571:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243376600, rc 0) 00000100:00100000:0.0:1780298642.936604:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:29360:x1866777609430656:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1017us (1094us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298642.936611:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9121 00000100:00100000:2.0:1780298642.936926:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.936965:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609430784 00000100:00100000:1.0:1780298642.936969:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609430784 00010000:00010000:1.0:1780298642.936985:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e630338d6 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a71 expref: 12 pid: 8356 timeout: 932 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.937010:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15047 00000100:00100000:1.0:1780298642.937015:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b03100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609430784:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298642.937021:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298642.937034:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e630338d6 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a71 expref: 12 pid: 8356 timeout: 932 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.937043:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e630338d6 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a71 expref: 12 pid: 8356 timeout: 932 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.937054:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e630338d6 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a71 expref: 12 pid: 8356 timeout: 932 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.937063:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298642.937084:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b03100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609430784:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 70us (160us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.937088:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15047 00000100:00100000:1.0:1780298642.938602:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.938647:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609430912 00000100:00100000:3.0:1780298642.938656:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609430912 00010000:00010000:3.0:1780298642.938686:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000070f5434e/0xaf35132e630338c1 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2080768->2093055) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a5c expref: 7 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.938726:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15048 00000100:00100000:3.0:1780298642.938738:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b00050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609430912:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298642.938752:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298642.938768:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000070f5434e/0xaf35132e630338c1 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2080768->2093055) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a5c expref: 7 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.938778:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000070f5434e/0xaf35132e630338c1 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2080768->2093055) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a5c expref: 7 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.938794:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000070f5434e/0xaf35132e630338c1 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2080768->2093055) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a5c expref: 7 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.938811:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(130 177) 00010000:00010000:3.0:1780298642.938814:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(130 177) 00010000:00010000:3.0:1780298642.938819:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000003a994673/0xaf35132e630338e4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a78 expref: 10 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.938825:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003a994673/0xaf35132e630338e4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a78 expref: 10 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.938855:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000003a994673/0xaf35132e630338e4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a78 expref: 10 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.938866:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519d880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777604913792:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298642.938935:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf341359600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.939664:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519d880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777604913792:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298642.939677:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298642.939982:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b00050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609430912:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1226us (1359us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.939984:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.940010:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15048 00000100:00100000:0.0:1780298642.940048:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609431040 00000100:00100000:0.0:1780298642.940054:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609431040 00000100:00100000:1.0:1780298642.940064:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00002000:00010000:0.0:1780298642.940075:0:19949:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:0.0:1780298642.940079:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244ad9180 with x1866777609431040 ext(4194304->18446744073709551615) 00002000:00010000:0.0:1780298642.940086:0:19949:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf244ad9180. 00000100:00100000:2.0:1780298642.940119:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609431168 00000100:00100000:1.0:1780298642.940159:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6057 00000100:00100000:2.0:1780298642.940171:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609431168 00000100:00100000:1.0:1780298642.940177:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609431040:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:2.0:1780298642.940195:0:8363:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (3007139->18446744073709551615) 00002000:00100000:2.0:1780298642.940199:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244adb800 with x1866777609431168 ext(3007139->18446744073709551615) 00002000:00010000:2.0:1780298642.940208:0:8363:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf244adb800. 00000100:00100000:2.0:1780298642.940230:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6058 00000100:00100000:2.0:1780298642.940235:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2399:x1866777609431168:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:1.0:1780298642.940329:0:19943:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 2499556 -> 4194304 00002000:00010000:1.0:1780298642.940332:0:19943:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298642.940335:0:19943:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298642.940337:0:19943:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298642.940339:0:19943:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 344 -> 344 00000100:00100000:1.0:1780298642.940375:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609431040:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 208us (403us total) trans 4294969649 rc 0/0 00000100:00100000:1.0:1780298642.940381:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6057 00002000:00010000:1.0:1780298642.940384:0:19943:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:1.0:1780298642.940387:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244ad9180 with x1866777609431040 ext(4194304->18446744073709551615) 00002000:00010000:1.0:1780298642.940392:0:19943:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf244ad9180. 00002000:00010000:2.0:1780298642.940410:0:8363:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 0 -> 3007139 00002000:00010000:2.0:1780298642.940413:0:8363:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:2.0:1780298642.940417:0:8363:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:2.0:1780298642.940419:0:8363:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:2.0:1780298642.940421:0:8363:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:2.0:1780298642.940446:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2399:x1866777609431168:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 210us (381us total) trans 4294969209 rc 0/0 00000100:00100000:2.0:1780298642.940452:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6058 00002000:00010000:2.0:1780298642.940456:0:8363:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (3007139->18446744073709551615) 00002000:00100000:2.0:1780298642.940458:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244adb800 with x1866777609431168 ext(3007139->18446744073709551615) 00002000:00010000:2.0:1780298642.940463:0:8363:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf244adb800. 00000100:00100000:2.0:1780298642.947759:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.947803:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.947830:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609431296 00000100:00100000:1.0:1780298642.947842:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609431296 00000100:00100000:3.0:1780298642.947845:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609431424 00000100:00100000:3.0:1780298642.947860:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609431424 00002000:00100000:1.0:1780298642.947888:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (0->3006463) req@ffff9cf241b01c00 x1866777609431296/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 504/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298642.947901:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (2502656->4194303) req@ffff9cf241b00e00 x1866777609431424/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298642.947901:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf241b01c00 with x1866777609431296 ext(0->3006463) 00002000:00010000:1.0:1780298642.947910:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf241b01c00 00002000:00100000:3.0:1780298642.947920:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf241b00e00 with x1866777609431424 ext(2502656->4194303) 00002000:00010000:3.0:1780298642.947927:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf241b00e00 00000100:00100000:1.0:1780298642.947943:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6059 00000100:00100000:1.0:1780298642.947952:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b01c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609431296:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298642.947971:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6060 00000100:00100000:3.0:1780298642.947986:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b00e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609431424:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298642.957346:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b00e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609431424:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 9353us (9531us total) trans 0 rc 1691648/1691648 00000100:00100000:2.0:1780298642.957361:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6060 00002000:00100000:2.0:1780298642.957377:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (2502656->4194303) req@ffff9cf241b00e00 x1866777609431424/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/440 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 1691648/1691648 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.957389:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf241b00e00 with x1866777609431424 ext(2502656->4194303) 00002000:00010000:2.0:1780298642.957397:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf241b00e00 00000100:00100000:1.0:1780298642.959957:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b01c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609431296:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 11999us (12198us total) trans 0 rc 2506752/2506752 00000100:00100000:1.0:1780298642.959968:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6059 00002000:00100000:1.0:1780298642.959975:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (0->3006463) req@ffff9cf241b01c00 x1866777609431296/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 504/440 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 2506752/2506752 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298642.959983:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf241b01c00 with x1866777609431296 ext(0->3006463) 00002000:00010000:1.0:1780298642.959987:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf241b01c00 00000100:00100000:3.0:1780298642.962479:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.962536:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609431552 00000100:00100000:2.0:1780298642.962548:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609431552 00002000:00100000:2.0:1780298642.962590:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x12:0x0] (2502656->3010559) req@ffff9cf24519d880 x1866777609431552/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 504/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.962601:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24519d880 with x1866777609431552 ext(2502656->3010559) 00002000:00010000:2.0:1780298642.962605:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24519d880 00000100:00100000:2.0:1780298642.962637:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6061 00000100:00100000:2.0:1780298642.962647:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609431552:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298642.964689:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609431552:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 2042us (2222us total) trans 0 rc 503808/503808 00000100:00100000:2.0:1780298642.964696:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6061 00002000:00100000:2.0:1780298642.964703:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x12:0x0] (2502656->3010559) req@ffff9cf24519d880 x1866777609431552/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 504/440 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 503808/503808 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.964713:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24519d880 with x1866777609431552 ext(2502656->3010559) 00002000:00010000:2.0:1780298642.964716:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24519d880 00000100:00100000:0.0:1780298642.965931:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.965977:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609431680 00000100:00100000:2.0:1780298642.965981:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609431680 00002000:00100000:2.0:1780298642.966002:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (3002368->3007138) req@ffff9cf3481fe680 x1866777609431680/t0(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 5264/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.966008:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fe680 with x1866777609431680 ext(3002368->3007138) 00002000:00010000:2.0:1780298642.966011:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf3481fe680 00000100:00100000:2.0:1780298642.966029:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6062 00000100:00100000:2.0:1780298642.966034:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609431680:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:2.0:1780298642.966491:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609431680:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 455us (560us total) trans 4294969210 rc 0/0 00000100:00100000:2.0:1780298642.966500:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6062 00002000:00100000:2.0:1780298642.966507:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (3002368->3007138) req@ffff9cf3481fe680 x1866777609431680/t4294969210(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 5264/448 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298642.966518:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fe680 with x1866777609431680 ext(3002368->3007138) 00002000:00010000:2.0:1780298642.966522:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf3481fe680 00000100:00100000:0.0:1780298642.967172:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.967213:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609431808 00000100:00100000:2.0:1780298642.967218:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609431808 00000100:00100000:2.0:1780298642.967251:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9122 00000100:00100000:2.0:1780298642.967259:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609431808:12345-192.168.203.25@tcp:16:fsx.0 00000001:00080000:0.0:1780298642.967927:0:8351:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0000: transno 4294969210 is committed 00000001:00080000:0.0:1780298642.967943:0:8351:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0000: transno 4294969206 is committed 00000100:00100000:2.0:1780298642.968031:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609431808:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 772us (857us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.968043:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9122 00000100:00100000:3.0:1780298642.968785:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.968830:0:6476:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609431936 00000100:00100000:1.0:1780298642.968835:0:6476:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609431936 00000100:00100000:1.0:1780298642.968864:0:6476:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61963 00000100:00100000:1.0:1780298642.968870:0:6476:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+102:29360:x1866777609431936:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.968880:0:6476:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.968901:0:6476:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499200/0xaf35132e630338eb lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 66 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2a7f expref: -99 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.968984:0:6476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341498c00/0xaf35132e630338f2 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 67 type: IBT flags: 0x40000000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.969002:0:6476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341498c00/0xaf35132e630338f2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 67 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.969014:0:6476:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341498c00/0xaf35132e630338f2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 67 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1780298642.969041:0:6476:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 7201443 00000004:00200000:1.0:1780298642.969045:0:6476:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1780298642.969060:0:6476:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341498c00/0xaf35132e630338f2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 67 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.969081:0:6476:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499200/0xaf35132e630338eb lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 67 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a7f expref: 104 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.969093:0:6476:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341498c00/0xaf35132e630338f2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 66 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a7f expref: 103 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.969100:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298642.969101:0:6476:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1780298642.969104:0:6476:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341498c00, rc 0) 00000100:00100000:1.0:1780298642.969157:0:6476:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+103:29360:x1866777609431936:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 287us (378us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298642.969162:0:6476:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61963 00000100:00100000:0.0:1780298642.970081:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.970111:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.970152:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609432064 00000100:00100000:1.0:1780298642.970157:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609432064 00000100:00100000:3.0:1780298642.970183:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609432192 00000100:00100000:3.0:1780298642.970192:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609432192 00000100:00100000:1.0:1780298642.970199:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9123 00000100:00100000:1.0:1780298642.970215:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609432064:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.970225:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298642.970236:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008be5bdd7/0xaf35132e630338f9 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2a86 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.970248:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9124 00000100:00100000:3.0:1780298642.970257:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609432192:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298642.970271:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.970296:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d828f4f8/0xaf35132e63033900 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2a8d expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.970307:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604913920:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.970347:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604914048:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.970399:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298642.970402:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856600 going to sleep for 11 seconds 00002000:00010000:3.0:1780298642.971090:0:8358:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks: 0 -> 16 00000100:00100000:3.0:1780298642.971106:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604914048:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.971121:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604913920:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298642.971160:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d828f4f8/0xaf35132e63033900 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a8d expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.971181:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d828f4f8/0xaf35132e63033900 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a8d expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.971186:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008be5bdd7/0xaf35132e630338f9 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a86 expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298642.971199:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2e00, rc 0) 00010000:00010000:1.0:1780298642.971212:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008be5bdd7/0xaf35132e630338f9 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a86 expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.971224:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341499c00, rc 0) 00000100:00100000:1.0:1780298642.971262:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609432064:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1055us (1180us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.971269:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609432192:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1011us (1154us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.971276:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9123 00000100:00100000:2.0:1780298642.971295:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9124 00000100:00100000:0.0:1780298642.972272:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.972319:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609432320 00000100:00100000:2.0:1780298642.972323:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609432320 00000100:00100000:0.0:1780298642.972338:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.972360:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609432448 00000100:00100000:2.0:1780298642.972363:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609432448 00000100:00100000:2.0:1780298642.972382:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9125 00000100:00100000:2.0:1780298642.972390:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609432320:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.972396:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298642.972396:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9126 00010000:00010000:2.0:1780298642.972406:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000058e2d0/0xaf35132e63033907 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2a94 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.972413:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609432448:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.972431:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298642.972439:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b03480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604914176:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.972444:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078c7f460/0xaf35132e6303390e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2a9b expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298642.972478:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604914304:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298642.972490:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4900 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.972510:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.973093:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b03480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604914176:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.973101:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604914304:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298642.973109:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000058e2d0/0xaf35132e63033907 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a94 expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.973117:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000078c7f460/0xaf35132e6303390e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a9b expref: 7 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.973122:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000058e2d0/0xaf35132e63033907 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a94 expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.973162:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078c7f460/0xaf35132e6303390e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a9b expref: 7 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.973166:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baea00, rc 0) 00010000:00010000:1.0:1780298642.973178:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36c00, rc 0) 00000100:00100000:2.0:1780298642.973192:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609432320:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 804us (919us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298642.973198:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9125 00000100:00100000:1.0:1780298642.973204:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609432448:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 797us (865us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.973212:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9126 00000100:00100000:0.0:1780298642.974648:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.974681:0:9976:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609432704 00000100:00100000:2.0:1780298642.974685:0:9976:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609432704 00000100:00100000:2.0:1780298642.974730:0:9976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61964 00000100:00100000:2.0:1780298642.974740:0:9976:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+127:29360:x1866777609432704:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:2.0:1780298642.974888:0:9976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae800/0xaf35132e63033915 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 67 type: IBT flags: 0x40000000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.974901:0:9976:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341498c00/0xaf35132e630338f2 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 67 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a7f expref: 102 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.974917:0:9976:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae800/0xaf35132e63033915 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 67 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.974935:0:9976:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341498c00/0xaf35132e630338f2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 67 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a7f expref: 102 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.974950:0:9976:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341498c00/0xaf35132e630338f2 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 67 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a7f expref: 102 pid: 6476 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.974962:0:9976:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b03480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:9976:1866777604914432:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298642.975010:0:9976:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298642.975649:0:9976:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b03480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:9976:1866777604914432:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298642.975663:0:9976:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae800/0xaf35132e63033915 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 67 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298642.975744:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.975775:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609432832 00000100:00100000:2.0:1780298642.975780:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609432832 00010000:00010000:2.0:1780298642.975798:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341498c00/0xaf35132e630338f2 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 67 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a7f expref: 103 pid: 6476 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.975827:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15049 00000100:00100000:2.0:1780298642.975835:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+103:2399:x1866777609432832:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298642.975845:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298642.975851:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341498c00/0xaf35132e630338f2 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 68 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a7f expref: 103 pid: 6476 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.975862:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341498c00/0xaf35132e630338f2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 68 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a7f expref: 103 pid: 6476 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.975877:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341498c00/0xaf35132e630338f2 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 68 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a7f expref: 103 pid: 6476 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.975889:0:6463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.975893:0:6463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae800/0xaf35132e63033915 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 67 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.975901:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae800/0xaf35132e63033915 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 67 type: IBT flags: 0x40290000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.975906:0:6463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae800/0xaf35132e63033915 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 67 type: IBT flags: 0x40290000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.975921:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298642.975944:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+102:2399:x1866777609432832:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 110us (199us total) trans 0 rc 0/0 00010000:00010000:0.0:1780298642.975945:0:9976:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae800/0xaf35132e63033915 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 66 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298642.975959:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15049 00010000:00010000:0.0:1780298642.975960:0:9976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae800/0xaf35132e63033915 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 66 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1780298642.976006:0:9976:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991036 00000004:00010000:0.0:1780298642.976025:0:9976:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000bcb152bb reply state 000000001c192c27 transno 4294991037 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae800/0xaf35132e63033915 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 66 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.976034:0:9976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae800/0xaf35132e63033915 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 66 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.976039:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298642.976041:0:9976:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1780298642.976043:0:9976:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1780298642.976091:0:9976:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+130:29360:x1866777609432704:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1353us (1444us total) trans 4294991037 rc 0/0 00000100:00100000:0.0:1780298642.976096:0:9976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61964 00000020:00080000:0.0:1780298642.976099:0:9976:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1780298642.976823:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.976856:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.976858:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609433088 00000100:00100000:2.0:1780298642.976862:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609433088 00002000:00010000:2.0:1780298642.976882:0:8363:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4194304->18446744073709551615) 00002000:00100000:2.0:1780298642.976887:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36d4b5500 with x1866777609433088 ext(4194304->18446744073709551615) 00002000:00010000:2.0:1780298642.976896:0:8363:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf36d4b5500. 00000100:00100000:1.0:1780298642.976899:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609432960 00000100:00100000:1.0:1780298642.976904:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609432960 00000100:00100000:2.0:1780298642.976921:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6063 00002000:00010000:1.0:1780298642.976922:0:19951:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (6133578->18446744073709551615) 00002000:00100000:1.0:1780298642.976927:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36d4b6300 with x1866777609432960 ext(6133578->18446744073709551615) 00000100:00100000:2.0:1780298642.976932:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609433088:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:1.0:1780298642.976935:0:19951:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf36d4b6300. 00000100:00100000:1.0:1780298642.976965:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6064 00000100:00100000:1.0:1780298642.976975:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2399:x1866777609432960:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:2.0:1780298642.977048:0:8363:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 3007139 -> 4194304 00002000:00010000:2.0:1780298642.977052:0:8363:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:2.0:1780298642.977055:0:8363:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:2.0:1780298642.977058:0:8363:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:2.0:1780298642.977060:0:8363:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 16 -> 16 00002000:00010000:1.0:1780298642.977087:0:19951:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 6133578 00002000:00010000:1.0:1780298642.977090:0:19951:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298642.977092:0:19951:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298642.977093:0:19951:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298642.977095:0:19951:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 344 -> 344 00000100:00100000:2.0:1780298642.977099:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609433088:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 169us (275us total) trans 4294969211 rc 0/0 00000100:00100000:2.0:1780298642.977110:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6063 00002000:00010000:2.0:1780298642.977115:0:8363:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4194304->18446744073709551615) 00000100:00100000:1.0:1780298642.977118:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609432960:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 145us (263us total) trans 4294969650 rc 0/0 00002000:00100000:2.0:1780298642.977120:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36d4b5500 with x1866777609433088 ext(4194304->18446744073709551615) 00000100:00100000:1.0:1780298642.977148:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6064 00002000:00010000:1.0:1780298642.977153:0:19951:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (6133578->18446744073709551615) 00002000:00010000:2.0:1780298642.977156:0:8363:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf36d4b5500. 00002000:00100000:1.0:1780298642.977156:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36d4b6300 with x1866777609432960 ext(6133578->18446744073709551615) 00002000:00010000:1.0:1780298642.977167:0:19951:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf36d4b6300. 00000100:00100000:3.0:1780298642.984705:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298642.984735:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298642.984763:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609433216 00000100:00100000:1.0:1780298642.984772:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609433216 00000100:00100000:0.0:1780298642.984784:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609433344 00000100:00100000:0.0:1780298642.984792:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609433344 00002000:00100000:1.0:1780298642.984820:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (4194304->6135807) req@ffff9cf24519ca80 x1866777609433216/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.984828:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x12:0x0] (3010560->4194303) req@ffff9cf24519f480 x1866777609433344/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298642.984844:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24519ca80 with x1866777609433216 ext(4194304->6135807) 00002000:00010000:1.0:1780298642.984851:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24519ca80 00002000:00100000:0.0:1780298642.984858:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24519f480 with x1866777609433344 ext(3010560->4194303) 00002000:00010000:0.0:1780298642.984864:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24519f480 00000100:00100000:1.0:1780298642.984879:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6065 00000100:00100000:1.0:1780298642.984890:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609433216:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298642.984907:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6066 00000100:00100000:0.0:1780298642.984922:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609433344:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298642.990853:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609433344:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 5932us (6109us total) trans 0 rc 1183744/1183744 00000100:00100000:0.0:1780298642.990862:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6066 00002000:00100000:0.0:1780298642.990872:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x12:0x0] (3010560->4194303) req@ffff9cf24519f480 x1866777609433344/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/440 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 1183744/1183744 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.990880:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24519f480 with x1866777609433344 ext(3010560->4194303) 00002000:00010000:0.0:1780298642.990885:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24519f480 00000100:00100000:1.0:1780298642.992600:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609433216:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 7709us (7928us total) trans 0 rc 1941504/1941504 00000100:00100000:1.0:1780298642.992609:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6065 00002000:00100000:1.0:1780298642.992617:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (4194304->6135807) req@ffff9cf24519ca80 x1866777609433216/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 488/440 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 1941504/1941504 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298642.992629:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24519ca80 with x1866777609433216 ext(4194304->6135807) 00002000:00010000:1.0:1780298642.992633:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24519ca80 00000100:00100000:2.0:1780298642.994392:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.994445:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609433472 00000100:00100000:0.0:1780298642.994455:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609433472 00002000:00100000:0.0:1780298642.994499:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (6127616->6133577) req@ffff9cf241b03480 x1866777609433472/t0(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 6456/0 e 0 to 0 dl 1780298653 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.994510:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf241b03480 with x1866777609433472 ext(6127616->6133577) 00002000:00010000:0.0:1780298642.994515:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf241b03480 00000100:00100000:0.0:1780298642.994539:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6067 00000100:00100000:0.0:1780298642.994547:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b03480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609433472:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:0.0:1780298642.994981:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b03480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2400:x1866777609433472:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 435us (595us total) trans 4294969651 rc 0/0 00000100:00100000:0.0:1780298642.994987:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6067 00002000:00100000:0.0:1780298642.994992:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (6127616->6133577) req@ffff9cf241b03480 x1866777609433472/t4294969651(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:348/0 lens 6456/448 e 0 to 0 dl 1780298653 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298642.994997:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf241b03480 with x1866777609433472 ext(6127616->6133577) 00002000:00010000:0.0:1780298642.995001:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf241b03480 00000100:00100000:3.0:1780298642.995708:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.995752:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609433600 00000100:00100000:2.0:1780298642.995759:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609433600 00000100:00100000:2.0:1780298642.995797:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9127 00000100:00100000:2.0:1780298642.995808:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2400:x1866777609433600:12345-192.168.203.25@tcp:16:fsx.0 00000001:00080000:1.0:1780298642.996658:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969651 is committed 00000100:00100000:2.0:1780298642.996746:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609433600:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 940us (1040us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.996751:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9127 00000100:00100000:1.0:1780298642.997502:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.997553:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609433728 00000100:00100000:2.0:1780298642.997558:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609433728 00000100:00100000:2.0:1780298642.997592:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61965 00000100:00100000:2.0:1780298642.997598:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+102:29360:x1866777609433728:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298642.997608:0:6477:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298642.997626:0:6477:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3000/0xaf35132e6303391c lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 67 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2aa2 expref: -99 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.997689:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2800/0xaf35132e63033923 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 68 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.997702:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2800/0xaf35132e63033923 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 68 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.997710:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2800/0xaf35132e63033923 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 68 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298642.997725:0:6477:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 10327882 00000004:00200000:2.0:1780298642.997728:0:6477:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298642.997740:0:6477:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2800/0xaf35132e63033923 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 68 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.997758:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3000/0xaf35132e6303391c lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 68 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2aa2 expref: 104 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.997772:0:6477:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2800/0xaf35132e63033923 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 67 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2aa2 expref: 103 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298642.997778:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298642.997780:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298642.997783:0:6477:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2800, rc 0) 00000100:00100000:2.0:1780298642.997809:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+103:29360:x1866777609433728:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 212us (311us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298642.997813:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61965 00000100:00100000:0.0:1780298642.998733:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298642.998763:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298642.998782:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609433984 00000100:00100000:2.0:1780298642.998786:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609433984 00000100:00100000:2.0:1780298642.998808:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609433856 00000100:00100000:2.0:1780298642.998810:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609433856 00000100:00100000:1.0:1780298642.998831:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9128 00000100:00100000:3.0:1780298642.998846:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9129 00000100:00100000:1.0:1780298642.998851:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609433984:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298642.998861:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609433856:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298642.998867:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.998869:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298642.998882:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d828f4f8/0xaf35132e6303392a lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2aa9 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.998883:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e63033931 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2ab0 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298642.998947:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604914560:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298642.998947:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604914688:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298642.999049:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298642.999049:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00002000:00010000:0.0:1780298642.999783:0:12101:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks: 344 -> 360 00000100:00100000:1.0:1780298642.999788:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604914688:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298642.999800:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604914560:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298642.999827:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e63033931 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ab0 expref: 7 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.999830:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d828f4f8/0xaf35132e6303392a lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2aa9 expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.999882:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d828f4f8/0xaf35132e6303392a lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2aa9 expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298642.999885:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e63033931 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ab0 expref: 7 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298642.999908:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2e00, rc 0) 00010000:00010000:1.0:1780298642.999916:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36a00, rc 0) 00000100:00100000:0.0:1780298642.999944:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609433856:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1089us (1181us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298642.999950:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9129 00000100:00100000:1.0:1780298642.999956:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609433984:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1111us (1219us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298642.999969:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9128 00000100:00100000:0.0:1780298643.000862:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.000909:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609434112 00000100:00100000:1.0:1780298643.000916:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609434112 00000100:00080000:2.0:1780298643.000958:0:6473:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1780298643.000969:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.000975:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9130 00000100:00100000:1.0:1780298643.000985:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609434112:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.000994:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.001006:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009cb6f854/0xaf35132e63033938 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2ab7 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.001025:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609434240 00000100:00100000:0.0:1780298643.001033:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609434240 00000100:00100000:1.0:1780298643.001041:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604914816:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.001071:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9131 00000100:00100000:0.0:1780298643.001081:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609434240:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.001094:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298643.001096:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00010000:00010000:0.0:1780298643.001108:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e6303393f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2abe expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.001184:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36d4b5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604914944:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.001255:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417e40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.001699:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604914816:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.001721:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009cb6f854/0xaf35132e63033938 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ab7 expref: 7 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.001737:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009cb6f854/0xaf35132e63033938 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ab7 expref: 7 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.001747:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36e00, rc 0) 00000100:00100000:0.0:1780298643.001791:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36d4b5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604914944:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.001792:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609434112:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 805us (927us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.001820:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9130 00010000:00010000:0.0:1780298643.001832:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e6303393f lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2abe expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.001848:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e6303393f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2abe expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.001857:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baea00, rc 0) 00000100:00100000:0.0:1780298643.001884:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609434240:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 806us (924us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.001889:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9131 00000100:00100000:3.0:1780298643.002582:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.002633:0:7411:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609434368 00000100:00100000:2.0:1780298643.002638:0:7411:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609434368 00000100:00100000:2.0:1780298643.002669:0:7411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61966 00000100:00100000:2.0:1780298643.002675:0:7411:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343967b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+130:29360:x1866777609434368:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.002687:0:7411:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.002698:0:7411:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3000/0xaf35132e63033946 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 68 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2ac5 expref: -99 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.002764:0:7411:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3200/0xaf35132e6303394d lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 69 type: IBT flags: 0x40000000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.002775:0:7411:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3200/0xaf35132e6303394d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 69 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.002783:0:7411:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3200/0xaf35132e6303394d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 69 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298643.002803:0:7411:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 10327882 00000004:00200000:2.0:1780298643.002805:0:7411:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298643.002819:0:7411:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3200/0xaf35132e6303394d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 69 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.002833:0:7411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3000/0xaf35132e63033946 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 69 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ac5 expref: 132 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.002844:0:7411:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3200/0xaf35132e6303394d lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 68 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ac5 expref: 131 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.002849:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298643.002851:0:7411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298643.002853:0:7411:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3200, rc 0) 00000100:00100000:2.0:1780298643.002879:0:7411:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343967b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+131:29360:x1866777609434368:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 204us (298us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.002883:0:7411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61966 00000100:00100000:3.0:1780298643.003812:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.003844:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609434496 00000100:00100000:2.0:1780298643.003848:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609434496 00000100:00100000:2.0:1780298643.003876:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6068 00000100:00100000:2.0:1780298643.003881:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343966300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609434496:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298643.004069:0:19928:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 6133578 -> 6133578 00002000:00010000:2.0:1780298643.004073:0:19928:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:2.0:1780298643.004076:0:19928:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:2.0:1780298643.004078:0:19928:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:2.0:1780298643.004080:0:19928:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 360 -> 376 00000100:00100000:2.0:1780298643.004115:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343966300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609434496:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 233us (302us total) trans 4294969652 rc 0/0 00000100:00100000:2.0:1780298643.004122:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6068 00000100:00100000:1.0:1780298643.004869:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.004900:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609434624 00000100:00100000:2.0:1780298643.004904:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609434624 00000100:00100000:2.0:1780298643.004930:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9132 00000100:00100000:1.0:1780298643.004931:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.004938:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609434624:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.004944:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.004952:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e63033954 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2acc expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.004960:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609434752 00000100:00100000:0.0:1780298643.004968:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609434752 00000100:00100000:2.0:1780298643.004981:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b00000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604915072:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.005017:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9133 00000100:00100000:0.0:1780298643.005037:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ada300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609434752:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.005049:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.005062:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e6303395b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2ad3 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.005099:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36d4b5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604915200:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.005105:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.005197:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417e40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.005693:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b00000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604915072:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.005715:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e63033954 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2acc expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.005732:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e63033954 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2acc expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.005744:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2a00, rc 0) 00000100:00100000:0.0:1780298643.005754:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36d4b5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604915200:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.005772:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e6303395b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ad3 expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.005788:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e6303395b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ad3 expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.005795:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baec00, rc 0) 00000100:00100000:3.0:1780298643.005812:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609434624:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 874us (941us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.005817:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ada300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609434752:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 790us (889us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.005826:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9132 00000100:00100000:0.0:1780298643.005833:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9133 00000100:00100000:1.0:1780298643.006623:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.006658:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609434880 00000100:00100000:2.0:1780298643.006662:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609434880 00000100:00100000:1.0:1780298643.006675:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.006699:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609435008 00000100:00100000:2.0:1780298643.006702:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609435008 00000100:00100000:2.0:1780298643.006721:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9134 00000100:00100000:0.0:1780298643.006729:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9135 00000100:00100000:2.0:1780298643.006730:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609434880:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.006738:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298643.006742:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609435008:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.006748:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e63033962 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2ada expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.006749:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.006759:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c37b15a2/0xaf35132e63033969 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2ae1 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.006774:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b03b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604915328:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.006787:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36d4b5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604915456:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.006816:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.006816:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417e40 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.007367:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b03b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604915328:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.007382:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e63033962 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ada expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.007391:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e63033962 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ada expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.007397:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3600, rc 0) 00000100:00100000:0.0:1780298643.007422:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609434880:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 695us (800us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.007428:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9134 00000100:00100000:2.0:1780298643.007461:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36d4b5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604915456:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.007483:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c37b15a2/0xaf35132e63033969 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ae1 expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.007497:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c37b15a2/0xaf35132e63033969 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ae1 expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.007507:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baee00, rc 0) 00000100:00100000:2.0:1780298643.007532:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609435008:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 796us (859us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.007538:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9135 00000100:00100000:3.0:1780298643.008315:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.008347:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609435136 00000100:00100000:1.0:1780298643.008352:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609435136 00000100:00100000:1.0:1780298643.008380:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9136 00000100:00100000:1.0:1780298643.008388:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:29360:x1866777609435136:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.008395:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.008402:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63033970 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2ae8 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.008419:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63033835 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29ec expref: 10 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.008427:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298642 v=(92 130) 00010000:00010000:1.0:1780298643.008431:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(93 130) 00010000:00010000:1.0:1780298643.008435:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63033970 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3194880->3198975] (req 3194880->3198975) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ae8 expref: 7 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.008455:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63033835 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29ec expref: 10 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.008468:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63033835 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29ec expref: 10 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.008478:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f755500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604915584:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298643.008515:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.009015:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f755500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604915584:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298643.009029:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63033970 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3194880->3198975] (req 3194880->3198975) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ae8 expref: 7 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.009040:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37000, rc 0) 00000100:00100000:1.0:1780298643.009066:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609435136:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 679us (751us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.009072:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9136 00000100:00100000:2.0:1780298643.012304:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.012344:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609435264 00000100:00100000:0.0:1780298643.012349:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609435264 00010000:00010000:0.0:1780298643.012370:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63033835 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29ec expref: 11 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.012393:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15050 00000100:00100000:0.0:1780298643.012399:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b03100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609435264:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298643.012406:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298643.012415:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63033835 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29ec expref: 11 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.012424:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63033835 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29ec expref: 11 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.012437:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63033835 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d29ec expref: 11 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.012444:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(93 130) 00010000:00010000:0.0:1780298643.012447:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(93 130) 00010000:00010000:0.0:1780298643.012452:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63033970 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3194880->36749311] (req 3194880->3198975) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ae8 expref: 6 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.012458:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63033970 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3194880->36749311] (req 3194880->3198975) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ae8 expref: 6 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.012475:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63033970 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3194880->36749311] (req 3194880->3198975) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ae8 expref: 6 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.012482:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36d4b6680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:8374:1866777604915712:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298643.012512:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417a80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.013157:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36d4b6680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:8374:1866777604915712:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298643.013167:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298643.013190:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b03100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609435264:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 792us (888us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298643.013195:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15050 00000100:00100000:2.0:1780298643.013392:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.013431:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609435392 00000100:00100000:1.0:1780298643.013435:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609435392 00002000:00100000:1.0:1780298643.013464:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (3194880->3198975) req@ffff9cf241b00a80 x1866777609435392/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.013472:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf241b00a80 with x1866777609435392 ext(3194880->3198975) 00002000:00010000:1.0:1780298643.013475:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf241b00a80 00000100:00100000:1.0:1780298643.013491:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6069 00000100:00100000:1.0:1780298643.013496:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b00a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609435392:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298643.013607:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b00a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609435392:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 111us (214us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298643.013612:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6069 00002000:00100000:1.0:1780298643.013618:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (3194880->3198975) req@ffff9cf241b00a80 x1866777609435392/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.013624:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf241b00a80 with x1866777609435392 ext(3194880->3198975) 00002000:00010000:1.0:1780298643.013626:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf241b00a80 00000100:00100000:2.0:1780298643.014537:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.014568:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609435520 00000100:00100000:0.0:1780298643.014572:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609435520 00000100:00100000:0.0:1780298643.014595:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9137 00000100:00100000:0.0:1780298643.014601:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b03800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609435520:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.014607:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.014615:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e63033977 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2aef expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.014643:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36d4b6680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604915840:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.014672:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417a80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.015237:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36d4b6680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604915840:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.015250:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e63033977 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2aef expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.015264:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e63033977 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2aef expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.015271:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baec00, rc 0) 00000100:00100000:0.0:1780298643.015308:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b03800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609435520:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 709us (772us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.015313:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9137 00000100:00100000:2.0:1780298643.016044:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.016079:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609435648 00000100:00100000:1.0:1780298643.016083:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609435648 00000100:00100000:1.0:1780298643.016104:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9138 00000100:00100000:1.0:1780298643.016112:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b01880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609435648:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.016118:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.016144:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009cb6f854/0xaf35132e6303397e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2af6 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.016166:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f755500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604915968:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.016195:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.016864:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f755500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604915968:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.016876:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009cb6f854/0xaf35132e6303397e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2af6 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.016885:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009cb6f854/0xaf35132e6303397e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2af6 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.016894:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36e00, rc 0) 00000100:00100000:1.0:1780298643.016924:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b01880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609435648:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 810us (879us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.016930:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9138 00000100:00100000:3.0:1780298643.017783:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.017823:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609435776 00000100:00100000:2.0:1780298643.017828:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609435776 00000100:00100000:2.0:1780298643.017852:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9139 00000100:00100000:2.0:1780298643.017857:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609435776:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.017863:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.017871:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c47b04a/0xaf35132e63033985 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2afd expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.017897:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b03b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604916096:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.017940:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.018595:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b03b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604916096:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.018610:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005c47b04a/0xaf35132e63033985 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2afd expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.018620:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c47b04a/0xaf35132e63033985 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2afd expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.018627:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3400, rc 0) 00000100:00100000:0.0:1780298643.018655:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609435776:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 798us (878us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.018663:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9139 00000100:00100000:1.0:1780298643.019513:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.019562:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609435904 00000100:00100000:3.0:1780298643.019570:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609435904 00000100:00100000:3.0:1780298643.019614:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9140 00000100:00100000:3.0:1780298643.019630:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f755500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609435904:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.019646:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.019659:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa41763f/0xaf35132e6303398c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2b04 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.019705:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604916224:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.019772:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.020410:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604916224:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.020431:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000aa41763f/0xaf35132e6303398c lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b04 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.020454:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa41763f/0xaf35132e6303398c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b04 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.020471:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3600, rc 0) 00000100:00100000:1.0:1780298643.020508:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f755500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609435904:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 881us (995us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.020516:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9140 00000100:00100000:2.0:1780298643.021316:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.021350:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609436032 00000100:00100000:0.0:1780298643.021356:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609436032 00000100:00100000:0.0:1780298643.021383:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9141 00000100:00100000:0.0:1780298643.021392:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b00000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609436032:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.021403:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.021413:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e63033993 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2b0b expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.021431:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(93 130) 00010000:00010000:0.0:1780298643.021435:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(93 130) 00010000:00010000:0.0:1780298643.021443:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e63033993 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1347584->3194879] (req 1347584->1355775) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b0b expref: 11 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.021454:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e63033993 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1347584->3194879] (req 1347584->1355775) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b0b expref: 11 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.021460:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baea00, rc 0) 00000100:00100000:0.0:1780298643.021487:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b00000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:29360:x1866777609436032:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 98us (172us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298643.021491:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9141 00000100:00100000:2.0:1780298643.022183:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.022225:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609436160 00000100:00100000:1.0:1780298643.022230:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609436160 00002000:00100000:1.0:1780298643.022252:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x12:0x0] (1347584->1351679) req@ffff9cf241b03b80 x1866777609436160/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.022260:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf241b03b80 with x1866777609436160 ext(1347584->1351679) 00002000:00010000:1.0:1780298643.022263:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf241b03b80 00000100:00100000:1.0:1780298643.022279:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6070 00000100:00100000:1.0:1780298643.022284:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b03b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609436160:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298643.022392:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b03b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609436160:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 108us (209us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298643.022397:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6070 00002000:00100000:1.0:1780298643.022403:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x12:0x0] (1347584->1351679) req@ffff9cf241b03b80 x1866777609436160/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.022409:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf241b03b80 with x1866777609436160 ext(1347584->1351679) 00002000:00010000:1.0:1780298643.022411:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf241b03b80 00000100:00100000:3.0:1780298643.023164:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.023188:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609436288 00000100:00100000:1.0:1780298643.023192:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609436288 00002000:00100000:1.0:1780298643.023209:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (1351680->1355775) req@ffff9cf24304d880 x1866777609436288/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.023220:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24304d880 with x1866777609436288 ext(1351680->1355775) 00002000:00010000:1.0:1780298643.023223:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24304d880 00000100:00100000:1.0:1780298643.023240:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6071 00000100:00100000:1.0:1780298643.023251:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609436288:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298643.023319:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609436288:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 73us (210us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298643.023324:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6071 00002000:00100000:1.0:1780298643.023330:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (1351680->1355775) req@ffff9cf24304d880 x1866777609436288/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.023335:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24304d880 with x1866777609436288 ext(1351680->1355775) 00002000:00010000:1.0:1780298643.023339:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24304d880 00000100:00100000:2.0:1780298643.024181:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.024203:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.024213:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609436544 00000100:00100000:1.0:1780298643.024217:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609436544 00000100:00100000:1.0:1780298643.024245:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609436416 00000100:00100000:1.0:1780298643.024248:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609436416 00000100:00100000:3.0:1780298643.024258:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9142 00000100:00100000:0.0:1780298643.024269:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9143 00000100:00100000:3.0:1780298643.024275:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609436544:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298643.024284:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609436416:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.024302:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.024303:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.024314:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e6303399a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2b19 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.024316:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630339a1 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2b12 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.024430:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604916480:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.024430:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36d4b6680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604916352:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.024497:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856900 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.024501:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417a80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.025252:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36d4b6680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604916352:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.025280:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630339a1 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b12 expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.025306:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604916480:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.025312:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630339a1 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b12 expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.025324:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf000, rc 0) 00010000:00010000:3.0:1780298643.025337:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e6303399a lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b19 expref: 8 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.025355:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609436416:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1077us (1149us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.025360:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9143 00010000:00010000:3.0:1780298643.025364:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e6303399a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b19 expref: 8 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.025379:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3200, rc 0) 00000100:00100000:3.0:1780298643.025405:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609436544:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1139us (1225us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.025410:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9142 00000100:00100000:2.0:1780298643.026267:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.026316:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.026329:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609436672 00000100:00100000:1.0:1780298643.026336:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609436672 00000100:00100000:0.0:1780298643.026362:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609436800 00000100:00100000:0.0:1780298643.026368:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609436800 00000100:00100000:1.0:1780298643.026381:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9144 00000100:00100000:1.0:1780298643.026393:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609436672:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.026402:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298643.026413:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9145 00010000:00010000:1.0:1780298643.026414:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e630339a8 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2b20 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.026427:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609436800:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.026436:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.026446:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e630339af lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2b27 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.026455:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f757b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604916608:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.026479:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36d4b6680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604916736:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.026516:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.026521:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439c00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.027221:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f757b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604916608:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.027246:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e630339a8 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b20 expref: 7 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.027252:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36d4b6680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604916736:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.027270:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000078c7f460/0xaf35132e630339a8 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b20 expref: 7 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.027277:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e630339af lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b27 expref: 8 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.027285:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36c00, rc 0) 00010000:00010000:0.0:1780298643.027306:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e630339af lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b27 expref: 8 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.027324:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae400, rc 0) 00000100:00100000:1.0:1780298643.027336:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609436672:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 946us (1070us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.027347:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9144 00000100:00100000:0.0:1780298643.027354:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609436800:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 930us (1040us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.027363:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9145 00000100:00100000:2.0:1780298643.028398:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.028441:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609436928 00000100:00100000:1.0:1780298643.028447:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609436928 00000100:00100000:1.0:1780298643.028482:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9146 00000100:00100000:1.0:1780298643.028492:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609436928:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.028501:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.028510:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e630339b6 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2b2e expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.028541:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f757b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604916864:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.028584:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439c00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.029226:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f757b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604916864:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.029242:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e630339b6 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b2e expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.029259:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e630339b6 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b2e expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.029268:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36a00, rc 0) 00000100:00100000:1.0:1780298643.029310:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609436928:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 821us (913us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.029316:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9146 00000100:00100000:2.0:1780298643.030193:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.030235:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609437056 00000100:00100000:3.0:1780298643.030242:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609437056 00000100:00100000:3.0:1780298643.030301:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9147 00000100:00100000:3.0:1780298643.030312:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:29360:x1866777609437056:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.030327:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.030340:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002c458c77/0xaf35132e630339bd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2b35 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.030365:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63033970 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3194880->36749311] (req 3194880->3198975) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ae8 expref: 6 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.030374:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(93 130) 00010000:00010000:3.0:1780298643.030378:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(94 130) 00010000:00010000:3.0:1780298643.030383:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002c458c77/0xaf35132e630339bd lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3489792->3493887] (req 3489792->3493887) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b35 expref: 12 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.030411:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63033970 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3194880->36749311] (req 3194880->3198975) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ae8 expref: 6 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.030426:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63033970 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3194880->36749311] (req 3194880->3198975) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ae8 expref: 6 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.030439:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604916992:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298643.030495:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856900 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.031175:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604916992:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298643.031199:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002c458c77/0xaf35132e630339bd lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3489792->3493887] (req 3489792->3493887) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b35 expref: 12 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.031272:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2a00, rc 0) 00000100:00100000:3.0:1780298643.031343:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:29360:x1866777609437056:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1032us (1152us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.031343:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.031351:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9147 00000100:00100000:0.0:1780298643.031393:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609437184 00000100:00100000:0.0:1780298643.031400:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609437184 00002000:00100000:0.0:1780298643.031427:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (3194880->3198975) req@ffff9cf348007100 x1866777609437184/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 4584/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.031435:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf348007100 with x1866777609437184 ext(3194880->3198975) 00010000:00010000:0.0:1780298643.031441:0:8364:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63033970 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3194880->36749311] (req 3194880->3198975) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ae8 expref: 7 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.031460:0:8364:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63033970 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3194880->36749311] (req 3194880->3198975) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ae8 expref: 7 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1780298643.031466:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9cf348007100 00000100:00100000:0.0:1780298643.031485:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 190 00000100:00100000:0.0:1780298643.031497:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348007100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609437184:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:0.0:1780298643.032058:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348007100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609437184:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 565us (719us total) trans 4294969212 rc 0/0 00000100:00100000:0.0:1780298643.032063:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 190 00002000:00100000:0.0:1780298643.032068:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (3194880->3198975) req@ffff9cf348007100 x1866777609437184/t4294969212(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 4584/448 e 0 to 0 dl 1780298654 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.032078:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf348007100 with x1866777609437184 ext(3194880->3198975) 00010000:00010000:0.0:1780298643.032082:0:8364:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63033970 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3194880->36749311] (req 3194880->3198975) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ae8 expref: 9 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.032093:0:8364:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63033970 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3194880->36749311] (req 3194880->3198975) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ae8 expref: 9 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1780298643.032099:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9cf348007100 00000100:00100000:1.0:1780298643.033467:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.033521:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609437312 00000100:00100000:2.0:1780298643.033526:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609437312 00010000:00010000:2.0:1780298643.033543:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63033970 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3194880->36749311] (req 3194880->3198975) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ae8 expref: 9 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.033568:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15051 00000100:00100000:2.0:1780298643.033574:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f757b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609437312:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298643.033582:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:2.0:1780298643.033589:0:6461:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 16 -> 24 00010000:00010000:2.0:1780298643.033594:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63033970 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [3194880->36749311] (req 3194880->3198975) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ae8 expref: 9 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.033602:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63033970 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [3194880->36749311] (req 3194880->3198975) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ae8 expref: 9 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.033615:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003789a116/0xaf35132e63033970 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [3194880->36749311] (req 3194880->3198975) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ae8 expref: 9 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.033624:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(94 130) 00010000:00010000:2.0:1780298643.033627:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(94 130) 00010000:00010000:2.0:1780298643.033633:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000002c458c77/0xaf35132e630339bd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3489792->37044223] (req 3489792->3493887) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b35 expref: 11 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.033639:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002c458c77/0xaf35132e630339bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3489792->37044223] (req 3489792->3493887) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b35 expref: 11 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.033658:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000002c458c77/0xaf35132e630339bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3489792->37044223] (req 3489792->3493887) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b35 expref: 11 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.033666:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf348007480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6461:1866777604917120:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298643.033705:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.034310:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348007480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6461:1866777604917120:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298643.034321:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298643.034342:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f757b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609437312:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 769us (878us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.034348:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15051 00000100:00100000:0.0:1780298643.034528:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.034557:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609437440 00000100:00100000:2.0:1780298643.034561:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609437440 00002000:00100000:2.0:1780298643.034579:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x12:0x0] (3489792->3493887) req@ffff9cf36d4b6680 x1866777609437440/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.034586:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36d4b6680 with x1866777609437440 ext(3489792->3493887) 00002000:00010000:2.0:1780298643.034589:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf36d4b6680 00000100:00100000:2.0:1780298643.034606:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6072 00000100:00100000:2.0:1780298643.034613:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609437440:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298643.034689:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609437440:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 78us (162us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298643.034694:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6072 00002000:00100000:2.0:1780298643.034698:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x12:0x0] (3489792->3493887) req@ffff9cf36d4b6680 x1866777609437440/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.034705:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36d4b6680 with x1866777609437440 ext(3489792->3493887) 00002000:00010000:2.0:1780298643.034707:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf36d4b6680 00000100:00100000:2.0:1780298643.035817:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.035865:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609437568 00000100:00100000:1.0:1780298643.035868:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609437568 00002000:00100000:1.0:1780298643.035884:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (3493888->3497983) req@ffff9cf348006680 x1866777609437568/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.035892:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf348006680 with x1866777609437568 ext(3493888->3497983) 00002000:00010000:1.0:1780298643.035896:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf348006680 00000100:00100000:1.0:1780298643.035915:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6073 00000100:00100000:1.0:1780298643.035924:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348006680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609437568:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298643.035997:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348006680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609437568:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 76us (180us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298643.036005:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6073 00002000:00100000:1.0:1780298643.036010:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (3493888->3497983) req@ffff9cf348006680 x1866777609437568/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.036017:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf348006680 with x1866777609437568 ext(3493888->3497983) 00002000:00010000:1.0:1780298643.036020:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf348006680 00000100:00100000:1.0:1780298643.037012:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.037056:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609437696 00000100:00100000:2.0:1780298643.037061:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609437696 00002000:00100000:2.0:1780298643.037080:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x12:0x0] (3497984->3502079) req@ffff9cf37f755880 x1866777609437696/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.037090:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f755880 with x1866777609437696 ext(3497984->3502079) 00002000:00010000:2.0:1780298643.037093:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f755880 00000100:00100000:2.0:1780298643.037113:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6074 00000100:00100000:2.0:1780298643.037119:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f755880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609437696:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298643.037204:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f755880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609437696:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 84us (192us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298643.037211:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6074 00002000:00100000:2.0:1780298643.037219:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x12:0x0] (3497984->3502079) req@ffff9cf37f755880 x1866777609437696/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.037226:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f755880 with x1866777609437696 ext(3497984->3502079) 00002000:00010000:2.0:1780298643.037229:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f755880 00000100:00100000:0.0:1780298643.038180:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.038207:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609437824 00000100:00100000:1.0:1780298643.038212:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609437824 00002000:00100000:1.0:1780298643.038232:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (1347584->3502079) req@ffff9cf36d4b4e00 x1866777609437824/t0(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 504/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.038241:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36d4b4e00 with x1866777609437824 ext(1347584->3502079) 00002000:00010000:1.0:1780298643.038250:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf36d4b4e00 00000100:00100000:1.0:1780298643.038272:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6075 00000100:00100000:1.0:1780298643.038282:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609437824:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:3.0:1780298643.039402:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2397:x1866777609437824:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 1115us (1217us total) trans 4294969213 rc 0/0 00000100:00100000:3.0:1780298643.039414:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6075 00002000:00100000:3.0:1780298643.039427:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (1347584->3502079) req@ffff9cf36d4b4e00 x1866777609437824/t4294969213(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 504/448 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298643.039438:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36d4b4e00 with x1866777609437824 ext(1347584->3502079) 00002000:00010000:3.0:1780298643.039449:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf36d4b4e00 00000100:00100000:1.0:1780298643.040042:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.040083:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609437952 00000100:00100000:2.0:1780298643.040088:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609437952 00000100:00100000:2.0:1780298643.040114:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9148 00000100:00100000:2.0:1780298643.040120:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e583c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609437952:12345-192.168.203.25@tcp:16:fsx.0 00000001:00080000:0.0:1780298643.040896:0:8351:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0000: transno 4294969213 is committed 00000001:00080000:0.0:1780298643.040909:0:8351:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0000: transno 4294969212 is committed 00000100:00100000:2.0:1780298643.041005:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e583c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609437952:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 882us (963us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.041015:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9148 00000100:00100000:2.0:1780298643.041885:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.041915:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.041937:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609438080 00000100:00100000:0.0:1780298643.041944:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609438080 00000100:00100000:3.0:1780298643.041970:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609438208 00000100:00100000:3.0:1780298643.041977:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609438208 00000100:00100000:0.0:1780298643.041993:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9149 00000100:00100000:0.0:1780298643.042004:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348004e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609438080:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.042018:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298643.042029:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9150 00010000:00010000:0.0:1780298643.042035:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e630339c4 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2b3c expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.042046:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348007480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609438208:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.042103:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298643.042104:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36d4b6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604917248:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.042115:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ced6368f/0xaf35132e630339cb lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2b43 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.042219:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b40a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604917376:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.042251:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.042257:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00002000:00010000:3.0:1780298643.042945:0:9128:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks: 24 -> 64 00000100:00100000:2.0:1780298643.042969:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36d4b6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604917248:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.042970:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b40a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604917376:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.043026:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ced6368f/0xaf35132e630339cb lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b43 expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.043031:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e630339c4 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b3c expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.043072:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ced6368f/0xaf35132e630339cb lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b43 expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.043079:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e630339c4 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b3c expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.043088:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2a00, rc 0) 00010000:00010000:2.0:1780298643.043101:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae000, rc 0) 00000100:00100000:3.0:1780298643.043174:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348007480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609438208:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1135us (1257us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.043180:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9150 00000100:00100000:2.0:1780298643.043189:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348004e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609438080:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1184us (1301us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.043200:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9149 00000100:00100000:1.0:1780298643.044162:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.044191:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609438336 00000100:00100000:3.0:1780298643.044195:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609438336 00000100:00100000:1.0:1780298643.044205:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.044240:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609438464 00000100:00100000:3.0:1780298643.044243:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609438464 00000100:00100000:3.0:1780298643.044272:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9151 00000100:00100000:3.0:1780298643.044282:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f756680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609438336:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298643.044300:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9152 00010000:00010000:3.0:1780298643.044308:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298643.044319:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f755500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609438464:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.044323:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000045ae2ef5/0xaf35132e630339d2 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2b4a expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.044331:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.044343:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e630339d9 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2b51 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.044361:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604917504:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.044385:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b42300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604917632:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.044414:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856900 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.044426:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4900 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.045183:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b42300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604917632:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.045215:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e630339d9 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b51 expref: 7 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.045221:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604917504:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.045250:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e630339d9 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b51 expref: 7 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.045255:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000045ae2ef5/0xaf35132e630339d2 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b4a expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.045311:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3000, rc 0) 00010000:00010000:3.0:1780298643.045316:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000045ae2ef5/0xaf35132e630339d2 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b4a expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.045326:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3200, rc 0) 00000100:00100000:3.0:1780298643.045359:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f756680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609438336:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1073us (1193us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.045359:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f755500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609438464:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1040us (1149us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.045380:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9151 00000100:00100000:2.0:1780298643.045389:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9152 00000100:00100000:0.0:1780298643.046368:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.046407:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609438592 00000100:00100000:1.0:1780298643.046413:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609438592 00000100:00100000:1.0:1780298643.046449:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9153 00000100:00100000:1.0:1780298643.046459:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609438592:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.046467:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.046475:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e630339e0 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2b58 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.046505:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e56d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604917760:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.046557:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.047179:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e56d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604917760:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.047201:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e630339e0 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b58 expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.047219:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e630339e0 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b58 expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.047228:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36a00, rc 0) 00000100:00100000:2.0:1780298643.047253:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609438592:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 798us (887us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.047259:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9153 00000100:00100000:2.0:1780298643.048200:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.048234:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609438720 00000100:00100000:3.0:1780298643.048238:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609438720 00000100:00100000:3.0:1780298643.048264:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9154 00000100:00100000:3.0:1780298643.048269:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609438720:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.048278:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.048297:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003540747e/0xaf35132e630339e7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2b5f expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.048327:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604917888:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.048364:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856900 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.049428:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604917888:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.049444:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003540747e/0xaf35132e630339e7 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b5f expref: 12 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.049459:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003540747e/0xaf35132e630339e7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b5f expref: 12 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.049466:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3c00, rc 0) 00000100:00100000:3.0:1780298643.049492:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609438720:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1224us (1299us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.049497:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9154 00000100:00100000:1.0:1780298643.050348:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.050402:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609438848 00000100:00100000:0.0:1780298643.050409:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609438848 00000100:00100000:0.0:1780298643.050447:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9155 00000100:00100000:0.0:1780298643.050458:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609438848:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.050470:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.050479:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b35ce980/0xaf35132e630339ee lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2b66 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.050515:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36d4b4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604918016:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.050569:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.051201:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36d4b4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604918016:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.051223:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b35ce980/0xaf35132e630339ee lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b66 expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.051238:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b35ce980/0xaf35132e630339ee lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b66 expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.051247:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bafa00, rc 0) 00000100:00100000:0.0:1780298643.051277:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609438848:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 823us (933us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.051283:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9155 00000100:00100000:2.0:1780298643.052164:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.052218:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609438976 00000100:00100000:1.0:1780298643.052222:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609438976 00000100:00100000:1.0:1780298643.052258:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9156 00000100:00100000:1.0:1780298643.052268:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b41880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609438976:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.052277:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.052298:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078c7f460/0xaf35132e630339f5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2b6d expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.052328:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e56d180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604918144:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.052367:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.052978:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e56d180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604918144:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.052990:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000078c7f460/0xaf35132e630339f5 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b6d expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.053003:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078c7f460/0xaf35132e630339f5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b6d expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.053015:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36c00, rc 0) 00000100:00100000:2.0:1780298643.053042:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b41880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609438976:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 778us (880us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.053047:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9156 00000100:00100000:2.0:1780298643.054059:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.054113:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609439104 00000100:00100000:1.0:1780298643.054119:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609439104 00000100:00100000:1.0:1780298643.054203:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9157 00000100:00100000:1.0:1780298643.054211:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b40000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609439104:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.054220:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.054231:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e630339fc lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2b74 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.054240:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.054264:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e56ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604918272:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.054273:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609439232 00000100:00100000:3.0:1780298643.054324:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609439232 00000100:00100000:1.0:1780298643.054330:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439c00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.054368:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9158 00000100:00100000:3.0:1780298643.054376:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609439232:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.054388:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.054398:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e63033a03 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2b7b expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.054481:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604918400:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.054521:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856900 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.054966:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e56ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604918272:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.054983:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e630339fc lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b74 expref: 7 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.055001:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e630339fc lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b74 expref: 7 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.055016:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36a00, rc 0) 00000100:00100000:1.0:1780298643.055051:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b40000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609439104:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 841us (992us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.055059:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9157 00000100:00100000:3.0:1780298643.055194:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604918400:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.055248:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e63033a03 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b7b expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.055260:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e63033a03 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b7b expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.055271:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3200, rc 0) 00000100:00100000:3.0:1780298643.055311:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609439232:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 937us (1071us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.055316:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9158 00000100:00100000:2.0:1780298643.056232:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.056265:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609439360 00000100:00100000:0.0:1780298643.056272:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609439360 00000100:00100000:0.0:1780298643.056323:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9159 00000100:00100000:0.0:1780298643.056337:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609439360:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.056347:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.056358:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63033a0a lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2b82 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.056387:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.056396:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36d4b4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604918528:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.056431:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609439488 00000100:00100000:2.0:1780298643.056437:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609439488 00000100:00100000:0.0:1780298643.056452:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.056478:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9160 00000100:00100000:2.0:1780298643.056491:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609439488:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.056500:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.056509:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e63033a11 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2b89 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.056538:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b41f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604918656:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.056571:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4900 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.057057:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36d4b4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604918528:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.057077:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63033a0a lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b82 expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.057091:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63033a0a lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b82 expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.057100:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf200, rc 0) 00000100:00100000:0.0:1780298643.057170:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609439360:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 835us (935us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.057177:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9159 00000100:00100000:2.0:1780298643.057329:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b41f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604918656:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.057347:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e63033a11 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b89 expref: 7 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.057359:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e63033a11 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b89 expref: 7 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.057365:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3e00, rc 0) 00000100:00100000:2.0:1780298643.057387:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609439488:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 904us (1002us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.057394:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9160 00000100:00100000:3.0:1780298643.058316:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.058354:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609439616 00000100:00100000:0.0:1780298643.058360:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609439616 00002000:00100000:0.0:1780298643.058390:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (2236416->2240511) req@ffff9cf24304e300 x1866777609439616/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.058400:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24304e300 with x1866777609439616 ext(2236416->2240511) 00002000:00010000:0.0:1780298643.058404:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24304e300 00000100:00100000:0.0:1780298643.058429:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6076 00000100:00100000:0.0:1780298643.058439:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609439616:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298643.058590:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609439616:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 154us (275us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298643.058599:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6076 00002000:00100000:0.0:1780298643.058605:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (2236416->2240511) req@ffff9cf24304e300 x1866777609439616/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.058613:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24304e300 with x1866777609439616 ext(2236416->2240511) 00002000:00010000:0.0:1780298643.058616:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24304e300 00000100:00100000:1.0:1780298643.059404:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.059450:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609439744 00000100:00100000:2.0:1780298643.059455:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609439744 00002000:00100000:2.0:1780298643.059480:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x12:0x0] (2240512->2244607) req@ffff9cf36e56f800 x1866777609439744/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.059488:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e56f800 with x1866777609439744 ext(2240512->2244607) 00002000:00010000:2.0:1780298643.059491:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf36e56f800 00000100:00100000:2.0:1780298643.059515:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6077 00000100:00100000:2.0:1780298643.059521:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609439744:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298643.059594:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609439744:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 72us (190us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298643.059600:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6077 00002000:00100000:2.0:1780298643.059606:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x12:0x0] (2240512->2244607) req@ffff9cf36e56f800 x1866777609439744/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.059614:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e56f800 with x1866777609439744 ext(2240512->2244607) 00002000:00010000:2.0:1780298643.059617:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf36e56f800 00000100:00100000:1.0:1780298643.060588:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.060671:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609439872 00000100:00100000:3.0:1780298643.060679:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609439872 00000100:00100000:1.0:1780298643.060714:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.060739:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609440000 00000100:00100000:2.0:1780298643.060746:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609440000 00000100:00100000:0.0:1780298643.060766:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9161 00000100:00100000:0.0:1780298643.060775:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609439872:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298643.060782:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9162 00010000:00010000:0.0:1780298643.060784:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298643.060795:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609440000:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.060796:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63033a18 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2b90 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.060822:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.060832:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003540747e/0xaf35132e63033a1f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2b97 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.060839:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36d4b4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604918784:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.060873:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604918912:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.060884:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417f00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.060908:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856900 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.061713:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604918912:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.061716:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36d4b4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604918784:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.061740:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003540747e/0xaf35132e63033a1f lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b97 expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.061749:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63033a18 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b90 expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.061754:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003540747e/0xaf35132e63033a1f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b97 expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.061792:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3c00, rc 0) 00010000:00010000:0.0:1780298643.061803:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63033a18 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b90 expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.061814:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bafa00, rc 0) 00000100:00100000:3.0:1780298643.061825:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609440000:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1035us (1111us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.061832:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9162 00000100:00100000:0.0:1780298643.061843:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609439872:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1068us (1255us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.061851:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9161 00000100:00100000:2.0:1780298643.062835:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.062871:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.062888:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609440128 00000100:00100000:1.0:1780298643.062894:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609440128 00000100:00100000:0.0:1780298643.062903:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609440256 00000100:00100000:0.0:1780298643.062907:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609440256 00000100:00100000:1.0:1780298643.062935:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9163 00000100:00100000:1.0:1780298643.062948:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b41f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609440128:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298643.062953:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9164 00010000:00010000:1.0:1780298643.062963:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298643.062968:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b40a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609440256:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.062976:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63033a26 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2b9e expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.062976:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.062986:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a3f03a/0xaf35132e63033a2d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2ba5 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.063013:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e56d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604919040:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.063025:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36d4b4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604919168:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.063063:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417f00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.063064:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439c00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.063818:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36d4b4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604919168:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.063833:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000001a3f03a/0xaf35132e63033a2d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ba5 expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.063846:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e56d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604919040:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.063846:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a3f03a/0xaf35132e63033a2d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ba5 expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.063855:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae600, rc 0) 00010000:00010000:2.0:1780298643.063859:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63033a26 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b9e expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.063871:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bd7ce2a2/0xaf35132e63033a26 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b9e expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.063879:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36a00, rc 0) 00000100:00100000:0.0:1780298643.063885:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b40a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609440256:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 923us (1014us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.063891:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9164 00000100:00100000:2.0:1780298643.063896:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b41f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609440128:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 954us (1069us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.063901:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9163 00000100:00100000:2.0:1780298643.064964:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.065008:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609440384 00000100:00100000:1.0:1780298643.065013:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609440384 00000100:00100000:1.0:1780298643.065043:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9165 00000100:00100000:1.0:1780298643.065052:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343966a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:29360:x1866777609440384:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.065062:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.065071:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e63033a34 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2bac expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.065090:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000003a994673/0xaf35132e630338e4 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a78 expref: 9 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.065100:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298642 v=(130 177) 00010000:00010000:1.0:1780298643.065104:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(131 177) 00010000:00010000:1.0:1780298643.065109:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e63033a34 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2826240->2830335] (req 2826240->2830335) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bac expref: 7 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.065163:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000003a994673/0xaf35132e630338e4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a78 expref: 9 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.065178:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000003a994673/0xaf35132e630338e4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a78 expref: 9 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.065189:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e56d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604919296:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298643.065233:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439c00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.065837:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e56d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604919296:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298643.065852:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e63033a34 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2826240->2830335] (req 2826240->2830335) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bac expref: 7 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.065870:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b37000, rc 0) 00000100:00100000:1.0:1780298643.065901:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343966a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609440384:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 851us (937us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.065909:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9165 00000100:00100000:0.0:1780298643.068569:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.068612:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609440512 00000100:00100000:3.0:1780298643.068622:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609440512 00010000:00010000:3.0:1780298643.068652:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000003a994673/0xaf35132e630338e4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a78 expref: 10 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.068679:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15052 00000100:00100000:3.0:1780298643.068685:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609440512:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298643.068695:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298643.068709:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000003a994673/0xaf35132e630338e4 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a78 expref: 10 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.068718:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000003a994673/0xaf35132e630338e4 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a78 expref: 10 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.068733:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003a994673/0xaf35132e630338e4 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2a78 expref: 10 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.068744:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(131 177) 00010000:00010000:3.0:1780298643.068747:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(131 177) 00010000:00010000:3.0:1780298643.068753:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e63033a34 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2826240->2830335) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bac expref: 6 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.068760:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e63033a34 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2826240->2830335) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bac expref: 6 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.068781:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e63033a34 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2826240->2830335) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bac expref: 6 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.068790:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304c380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777604919424:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298643.068839:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856900 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.069477:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304c380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777604919424:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298643.069489:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298643.069512:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609440512:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 828us (945us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298643.069518:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15052 00000100:00100000:2.0:1780298643.069717:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.069766:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609440640 00000100:00100000:1.0:1780298643.069772:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609440640 00002000:00100000:1.0:1780298643.069802:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x2c0000401:0x11:0x0] (2826240->2830335) req@ffff9cf3481ffb80 x1866777609440640/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.069811:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481ffb80 with x1866777609440640 ext(2826240->2830335) 00002000:00010000:1.0:1780298643.069816:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481ffb80 00000100:00100000:1.0:1780298643.069839:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6078 00000100:00100000:1.0:1780298643.069847:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481ffb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609440640:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298643.069974:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481ffb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609440640:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 128us (257us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298643.069983:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6078 00002000:00100000:1.0:1780298643.069990:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x2c0000401:0x11:0x0] (2826240->2830335) req@ffff9cf3481ffb80 x1866777609440640/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.070000:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481ffb80 with x1866777609440640 ext(2826240->2830335) 00002000:00010000:1.0:1780298643.070004:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481ffb80 00000100:00100000:0.0:1780298643.070820:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.070852:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609440768 00000100:00100000:3.0:1780298643.070857:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609440768 00000100:00100000:3.0:1780298643.070883:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9166 00000100:00100000:3.0:1780298643.070888:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609440768:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.070897:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.070907:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e63033a3b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2bb3 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.070932:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604919552:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.070965:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856900 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.071551:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604919552:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.071565:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e63033a3b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bb3 expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.071578:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e63033a3b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bb3 expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.071588:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3200, rc 0) 00000100:00100000:3.0:1780298643.071620:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609440768:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 731us (801us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.071627:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9166 00000100:00100000:2.0:1780298643.072378:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.072436:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609440896 00000100:00100000:1.0:1780298643.072451:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609440896 00000100:00100000:1.0:1780298643.072483:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9167 00000100:00100000:1.0:1780298643.072492:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609440896:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.072504:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.072514:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078c7f460/0xaf35132e63033a42 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2bba expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.072549:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e56d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604919680:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.072604:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.073340:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e56d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604919680:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.073360:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000078c7f460/0xaf35132e63033a42 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bba expref: 7 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.073376:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078c7f460/0xaf35132e63033a42 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bba expref: 7 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.073386:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36c00, rc 0) 00000100:00100000:1.0:1780298643.073420:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609440896:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 929us (1044us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.073427:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9167 00000100:00100000:2.0:1780298643.074329:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.074369:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609441024 00000100:00100000:1.0:1780298643.074374:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609441024 00000100:00100000:1.0:1780298643.074407:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9168 00000100:00100000:1.0:1780298643.074417:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609441024:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.074427:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.074436:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e63033a49 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2bc1 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.074471:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e56d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604919808:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.074512:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.075241:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e56d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604919808:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.075259:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e63033a49 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bc1 expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.075273:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd7ce2a2/0xaf35132e63033a49 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bc1 expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.075283:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244b36a00, rc 0) 00000100:00100000:1.0:1780298643.075325:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609441024:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 910us (997us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.075332:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9168 00000100:00100000:0.0:1780298643.076183:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.076225:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609441152 00000100:00100000:1.0:1780298643.076231:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609441152 00000100:00100000:1.0:1780298643.076266:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9169 00000100:00100000:1.0:1780298643.076275:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609441152:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.076285:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.076310:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bb478d6d/0xaf35132e63033a50 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2bc8 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.076330:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e63033a34 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2826240->2830335) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bac expref: 8 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.076343:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(131 177) 00010000:00010000:1.0:1780298643.076347:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(131 177) 00010000:00010000:1.0:1780298643.076353:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000bb478d6d/0xaf35132e63033a50 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [4968448->4972543] (req 4968448->4972543) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bc8 expref: 8 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.076374:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e63033a34 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2826240->2830335) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bac expref: 8 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.076387:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e63033a34 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2826240->2830335) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bac expref: 8 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.076398:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e56d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604919936:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298643.076441:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.077041:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e56d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604919936:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298643.077057:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bb478d6d/0xaf35132e63033a50 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [4968448->4972543] (req 4968448->4972543) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bc8 expref: 8 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.077069:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341498c00, rc 0) 00000100:00100000:1.0:1780298643.077100:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:29360:x1866777609441152:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 827us (921us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.077108:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9169 00000100:00100000:0.0:1780298643.077240:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.077296:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609441280 00000100:00100000:1.0:1780298643.077301:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609441280 00010000:00010000:1.0:1780298643.077321:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e63033a34 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2826240->2830335) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bac expref: 8 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.077349:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15053 00000100:00100000:1.0:1780298643.077356:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b7b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609441280:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298643.077366:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298643.077377:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e63033a34 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2826240->2830335) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bac expref: 8 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.077388:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e63033a34 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2826240->2830335) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bac expref: 8 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.077403:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003789a116/0xaf35132e63033a34 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2826240->2830335) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bac expref: 8 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.077413:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(131 177) 00010000:00010000:1.0:1780298643.077416:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(131 177) 00010000:00010000:1.0:1780298643.077422:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000bb478d6d/0xaf35132e63033a50 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [4968448->38522879] (req 4968448->4972543) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bc8 expref: 7 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.077430:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000bb478d6d/0xaf35132e63033a50 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [4968448->38522879] (req 4968448->4972543) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bc8 expref: 7 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.077450:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000bb478d6d/0xaf35132e63033a50 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [4968448->38522879] (req 4968448->4972543) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bc8 expref: 7 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.077464:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e56d880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777604920064:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298643.077500:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.078175:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e56d880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777604920064:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298643.078195:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298643.078225:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b7b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609441280:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 870us (988us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.078232:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15053 00000100:00100000:2.0:1780298643.078503:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.078542:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609441408 00000100:00100000:1.0:1780298643.078547:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609441408 00002000:00100000:1.0:1780298643.078571:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (4968448->4972543) req@ffff9cf3481fdf80 x1866777609441408/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.078579:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fdf80 with x1866777609441408 ext(4968448->4972543) 00002000:00010000:1.0:1780298643.078590:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481fdf80 00000100:00100000:1.0:1780298643.078609:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6079 00000100:00100000:1.0:1780298643.078617:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609441408:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298643.078738:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609441408:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 123us (237us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298643.078744:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6079 00002000:00100000:1.0:1780298643.078750:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (4968448->4972543) req@ffff9cf3481fdf80 x1866777609441408/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.078758:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fdf80 with x1866777609441408 ext(4968448->4972543) 00002000:00010000:1.0:1780298643.078762:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481fdf80 00000100:00100000:0.0:1780298643.079874:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.079902:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609441536 00000100:00100000:3.0:1780298643.079907:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609441536 00002000:00100000:3.0:1780298643.079925:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (4972544->4976639) req@ffff9cf36d4b5180 x1866777609441536/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298643.079933:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36d4b5180 with x1866777609441536 ext(4972544->4976639) 00002000:00010000:3.0:1780298643.079935:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf36d4b5180 00000100:00100000:3.0:1780298643.079953:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6080 00000100:00100000:3.0:1780298643.079958:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609441536:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298643.080031:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609441536:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 71us (156us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1780298643.080039:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6080 00002000:00100000:3.0:1780298643.080046:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (4972544->4976639) req@ffff9cf36d4b5180 x1866777609441536/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298643.080052:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36d4b5180 with x1866777609441536 ext(4972544->4976639) 00002000:00010000:3.0:1780298643.080055:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf36d4b5180 00000100:00100000:2.0:1780298643.081098:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.081164:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609441664 00000100:00100000:0.0:1780298643.081170:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609441664 00002000:00100000:0.0:1780298643.081192:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (4968448->4976639) req@ffff9cf3481fce00 x1866777609441664/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 8680/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.081204:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fce00 with x1866777609441664 ext(4968448->4976639) 00002000:00010000:0.0:1780298643.081207:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481fce00 00000100:00100000:0.0:1780298643.081225:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6081 00000100:00100000:0.0:1780298643.081233:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609441664:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:0.0:1780298643.081692:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609441664:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 460us (593us total) trans 4294969653 rc 0/0 00000100:00100000:0.0:1780298643.081700:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6081 00002000:00100000:0.0:1780298643.081707:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (4968448->4976639) req@ffff9cf3481fce00 x1866777609441664/t4294969653(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 8680/448 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.081715:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fce00 with x1866777609441664 ext(4968448->4976639) 00002000:00010000:0.0:1780298643.081718:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481fce00 00000100:00100000:2.0:1780298643.082488:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.082519:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609441792 00000100:00100000:3.0:1780298643.082524:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609441792 00000100:00100000:3.0:1780298643.082556:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9170 00000100:00100000:3.0:1780298643.082562:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609441792:12345-192.168.203.25@tcp:16:fsx.0 00000001:00080000:1.0:1780298643.083233:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969653 is committed 00000001:00080000:1.0:1780298643.083242:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969652 is committed 00000100:00100000:3.0:1780298643.083315:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609441792:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 751us (826us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298643.083322:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9170 00000100:00100000:1.0:1780298643.084069:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.084101:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.084119:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609441920 00000100:00100000:2.0:1780298643.084142:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609441920 00000100:00100000:3.0:1780298643.084160:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609442048 00000100:00100000:3.0:1780298643.084166:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609442048 00000100:00100000:2.0:1780298643.084178:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9171 00000100:00100000:1.0:1780298643.084186:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9172 00000100:00100000:2.0:1780298643.084191:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609441920:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298643.084201:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5dbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609442048:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.084212:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.084212:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.084228:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63033a57 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2bcf expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.084228:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ac4408d/0xaf35132e63033a5e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2bd6 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.084307:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604920320:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.084307:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604920192:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.084371:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.084371:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.085072:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604920192:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.085099:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000ac4408d/0xaf35132e63033a5e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bd6 expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.085119:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ac4408d/0xaf35132e63033a5e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bd6 expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.085152:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341499200, rc 0) 00002000:00010000:2.0:1780298643.085157:0:8356:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks: 376 -> 392 00000100:00100000:2.0:1780298643.085169:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604920320:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.085185:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5dbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609442048:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 986us (1080us total) trans 0 rc 301/301 00010000:00010000:2.0:1780298643.085191:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63033a57 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bcf expref: 8 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.085198:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9172 00010000:00010000:2.0:1780298643.085213:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63033a57 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bcf expref: 8 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.085225:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2600, rc 0) 00000100:00100000:2.0:1780298643.085250:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609441920:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1065us (1181us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.085257:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9171 00000100:00100000:3.0:1780298643.086040:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.086064:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.086079:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609442176 00000100:00100000:1.0:1780298643.086084:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609442176 00000100:00100000:1.0:1780298643.086112:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609442304 00000100:00100000:1.0:1780298643.086115:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609442304 00000100:00100000:0.0:1780298643.086119:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9173 00000100:00100000:0.0:1780298643.086150:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609442176:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298643.086157:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9174 00010000:00010000:0.0:1780298643.086161:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298643.086169:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609442304:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.086171:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e63033a65 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2bdd expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.086178:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.086189:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003540747e/0xaf35132e63033a6c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2be4 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.086202:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36d4b7100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604920448:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.086220:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604920576:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.086242:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417f00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.086245:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.086963:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604920576:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.086969:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36d4b7100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604920448:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.086987:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003540747e/0xaf35132e63033a6c lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2be4 expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.086995:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e63033a65 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bdd expref: 8 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.087006:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003540747e/0xaf35132e63033a6c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2be4 expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.087015:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e63033a65 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bdd expref: 8 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.087019:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3c00, rc 0) 00010000:00010000:2.0:1780298643.087036:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf000, rc 0) 00000100:00100000:0.0:1780298643.087053:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609442304:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 887us (987us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.087061:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9174 00000100:00100000:2.0:1780298643.087066:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609442176:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 917us (1023us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.087075:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9173 00000100:00100000:2.0:1780298643.088086:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.088150:0:6476:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609442560 00000100:00100000:1.0:1780298643.088156:0:6476:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609442560 00000100:00100000:1.0:1780298643.088193:0:6476:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61967 00000100:00100000:1.0:1780298643.088200:0:6476:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+131:29360:x1866777609442560:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:1.0:1780298643.088240:0:6476:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298643.088254:0:6476:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3200/0xaf35132e6303394d lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 69 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ac5 expref: 131 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.088267:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298643.088270:0:6476:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298643.088417:0:6476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499c00/0xaf35132e63033a73 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 68 type: IBT flags: 0x40000000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.088428:0:6476:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2800/0xaf35132e63033923 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 68 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2aa2 expref: 102 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.088445:0:6476:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499c00/0xaf35132e63033a73 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 68 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.088461:0:6476:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2800/0xaf35132e63033923 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 68 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2aa2 expref: 102 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.088470:0:6476:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2800/0xaf35132e63033923 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 68 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2aa2 expref: 102 pid: 6477 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.088478:0:6476:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6476:1866777604920704:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298643.088510:0:6476:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439c00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.089106:0:6476:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6476:1866777604920704:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298643.089115:0:6476:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499c00/0xaf35132e63033a73 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 68 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.089230:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.089270:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609442688 00000100:00100000:3.0:1780298643.089274:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609442688 00010000:00010000:3.0:1780298643.089335:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2800/0xaf35132e63033923 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 68 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2aa2 expref: 103 pid: 6477 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.089358:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15054 00000100:00100000:3.0:1780298643.089363:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+103:2399:x1866777609442688:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298643.089372:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298643.089376:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2800/0xaf35132e63033923 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 69 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2aa2 expref: 103 pid: 6477 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.089384:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2800/0xaf35132e63033923 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 69 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2aa2 expref: 103 pid: 6477 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.089400:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2800/0xaf35132e63033923 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 69 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2aa2 expref: 103 pid: 6477 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.089409:0:20253:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298643.089412:0:20253:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499c00/0xaf35132e63033a73 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 68 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.089420:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499c00/0xaf35132e63033a73 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 68 type: IBT flags: 0x40290000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.089425:0:20253:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499c00/0xaf35132e63033a73 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 68 type: IBT flags: 0x40290000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.089439:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298643.089453:0:6476:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499c00/0xaf35132e63033a73 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 67 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.089459:0:6476:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499c00/0xaf35132e63033a73 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 67 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.089462:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+102:2399:x1866777609442688:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 97us (234us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298643.089472:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15054 00000001:00080000:1.0:1780298643.089503:0:6476:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991038 00000004:00010000:1.0:1780298643.089525:0:6476:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000ac2d6e73 reply state 00000000bd97671a transno 4294991039 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499c00/0xaf35132e63033a73 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 67 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.089534:0:6476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499c00/0xaf35132e63033a73 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 67 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.089538:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298643.089540:0:6476:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1780298643.089542:0:6476:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1780298643.089592:0:6476:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+133:29360:x1866777609442560:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1392us (1507us total) trans 4294991039 rc 0/0 00000100:00100000:1.0:1780298643.089597:0:6476:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61967 00000020:00080000:1.0:1780298643.089600:0:6476:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1780298643.090195:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.090227:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609442816 00000100:00100000:0.0:1780298643.090232:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609442816 00000100:00100000:0.0:1780298643.090264:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9175 00000100:00100000:0.0:1780298643.090272:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:29360:x1866777609442816:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.090280:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.090300:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e63033a7a lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2beb expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.090320:0:19992:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000bb478d6d/0xaf35132e63033a50 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [4968448->38522879] (req 4968448->4972543) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bc8 expref: 6 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.090330:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(131 177) 00010000:00010000:0.0:1780298643.090334:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(132 177) 00010000:00010000:0.0:1780298643.090339:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e63033a7a lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2beb expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.090359:0:19992:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000bb478d6d/0xaf35132e63033a50 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [4968448->38522879] (req 4968448->4972543) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bc8 expref: 6 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.090370:0:19992:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000bb478d6d/0xaf35132e63033a50 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [4968448->38522879] (req 4968448->4972543) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bc8 expref: 6 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.090381:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36d4b7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604920832:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298643.090417:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417f00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.090946:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36d4b7800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604920832:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298643.090960:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e63033a7a lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2beb expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.090971:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae400, rc 0) 00000100:00100000:0.0:1780298643.090998:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609442816:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 729us (806us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298643.091006:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9175 00000100:00100000:2.0:1780298643.091183:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.091211:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609442944 00000100:00100000:0.0:1780298643.091214:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609442944 00010000:00010000:0.0:1780298643.091226:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000bb478d6d/0xaf35132e63033a50 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [4968448->38522879] (req 4968448->4972543) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bc8 expref: 7 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.091245:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15055 00000100:00100000:0.0:1780298643.091250:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b00450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609442944:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298643.091256:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298643.091265:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000bb478d6d/0xaf35132e63033a50 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [4968448->38522879] (req 4968448->4972543) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bc8 expref: 7 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.091271:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000bb478d6d/0xaf35132e63033a50 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [4968448->38522879] (req 4968448->4972543) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bc8 expref: 7 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.091281:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bb478d6d/0xaf35132e63033a50 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [4968448->38522879] (req 4968448->4972543) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bc8 expref: 7 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.091302:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(132 177) 00010000:00010000:0.0:1780298643.091305:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(132 177) 00010000:00010000:0.0:1780298643.091309:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e63033a7a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2beb expref: 8 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.091315:0:6463:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e63033a7a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2beb expref: 8 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.091331:0:6463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e63033a7a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2beb expref: 8 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.091337:0:6463:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36d4b7800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6463:1866777604920960:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298643.091363:0:6463:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417f00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.091979:0:6463:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36d4b7800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6463:1866777604920960:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298643.091989:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298643.092014:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b00450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609442944:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 763us (832us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298643.092021:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15055 00000100:00100000:3.0:1780298643.092111:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.092161:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609443072 00000100:00100000:1.0:1780298643.092165:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609443072 00000100:00100000:1.0:1780298643.092192:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9176 00000100:00100000:1.0:1780298643.092200:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:29360:x1866777609443072:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.092208:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.092226:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e90122d2/0xaf35132e63033a81 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2bf2 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.092247:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e63033993 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1347584->3194879] (req 1347584->1355775) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b0b expref: 12 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.092256:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000002c458c77/0xaf35132e630339bd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3489792->37044223] (req 3489792->3493887) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b35 expref: 12 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.092261:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 2 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(94 130) 00010000:00010000:1.0:1780298643.092264:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(96 130) 00010000:00010000:1.0:1780298643.092267:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e90122d2/0xaf35132e63033a81 lrc: 5/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bf2 expref: 12 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.092281:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000002c458c77/0xaf35132e630339bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3489792->37044223] (req 3489792->3493887) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b35 expref: 12 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.092298:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000002c458c77/0xaf35132e630339bd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3489792->37044223] (req 3489792->3493887) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b35 expref: 12 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.092305:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604921088:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298643.092350:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e63033993 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1347584->3194879] (req 1347584->1355775) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b0b expref: 12 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.092360:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e63033993 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1347584->3194879] (req 1347584->1355775) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b0b expref: 12 pid: 12101 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.092377:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604921216:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298643.092419:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.092894:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604921088:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298643.093009:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604921216:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298643.093029:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e90122d2/0xaf35132e63033a81 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bf2 expref: 12 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.093041:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93c00, rc 0) 00000100:00100000:1.0:1780298643.093072:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:29360:x1866777609443072:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 873us (961us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.093079:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9176 00000100:00100000:3.0:1780298643.093168:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.093193:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609443200 00000100:00100000:0.0:1780298643.093199:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609443200 00010000:00010000:0.0:1780298643.093218:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000002c458c77/0xaf35132e630339bd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3489792->37044223] (req 3489792->3493887) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b35 expref: 12 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.093244:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15056 00000100:00100000:0.0:1780298643.093252:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2399:x1866777609443200:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298643.093263:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298643.093274:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000002c458c77/0xaf35132e630339bd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [3489792->37044223] (req 3489792->3493887) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b35 expref: 12 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.093284:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000002c458c77/0xaf35132e630339bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [3489792->37044223] (req 3489792->3493887) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b35 expref: 12 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.093312:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002c458c77/0xaf35132e630339bd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [3489792->37044223] (req 3489792->3493887) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b35 expref: 12 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.093323:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298643.093349:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.093356:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609443200:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 102us (187us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298643.093364:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15056 00000100:00100000:1.0:1780298643.093390:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609443328 00000100:00100000:1.0:1780298643.093395:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609443328 00002000:00100000:1.0:1780298643.093423:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x12:0x0] (2236416->2244607) req@ffff9cf24519fb80 x1866777609443328/t0(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 8680/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.093434:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24519fb80 with x1866777609443328 ext(2236416->2244607) 00010000:00010000:1.0:1780298643.093441:0:19928:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e63033993 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1347584->3194879] (req 1347584->1355775) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b0b expref: 11 pid: 12101 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.093455:0:19928:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e63033993 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1347584->3194879] (req 1347584->1355775) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b0b expref: 11 pid: 12101 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1780298643.093462:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9cf24519fb80 00000100:00100000:1.0:1780298643.093484:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 191 00000100:00100000:1.0:1780298643.093491:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609443328:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:1.0:1780298643.094009:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609443328:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 517us (660us total) trans 4294969214 rc 0/0 00000100:00100000:1.0:1780298643.094016:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 191 00002000:00100000:1.0:1780298643.094023:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x12:0x0] (2236416->2244607) req@ffff9cf24519fb80 x1866777609443328/t4294969214(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 8680/448 e 0 to 0 dl 1780298654 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.094031:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24519fb80 with x1866777609443328 ext(2236416->2244607) 00010000:00010000:1.0:1780298643.094037:0:19928:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e63033993 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1347584->3194879] (req 1347584->1355775) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b0b expref: 13 pid: 12101 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.094051:0:19928:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e63033993 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1347584->3194879] (req 1347584->1355775) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b0b expref: 13 pid: 12101 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1780298643.094057:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9cf24519fb80 00000100:00100000:2.0:1780298643.094671:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.094711:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609443456 00000100:00100000:1.0:1780298643.094716:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609443456 00010000:00010000:1.0:1780298643.094732:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e63033993 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1347584->3194879] (req 1347584->1355775) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b0b expref: 13 pid: 12101 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.094757:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15057 00000100:00100000:1.0:1780298643.094765:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609443456:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298643.094776:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:1.0:1780298643.094783:0:6461:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 64 -> 88 00010000:00010000:1.0:1780298643.094787:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e63033993 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1347584->3194879] (req 1347584->1355775) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b0b expref: 13 pid: 12101 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.094797:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e63033993 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1347584->3194879] (req 1347584->1355775) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b0b expref: 13 pid: 12101 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.094810:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e63033993 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1347584->3194879] (req 1347584->1355775) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2b0b expref: 13 pid: 12101 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.094820:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(96 130) 00010000:00010000:1.0:1780298643.094824:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(96 130) 00010000:00010000:1.0:1780298643.094829:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000e90122d2/0xaf35132e63033a81 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bf2 expref: 12 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.094838:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e90122d2/0xaf35132e63033a81 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bf2 expref: 12 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.094855:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000e90122d2/0xaf35132e63033a81 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bf2 expref: 12 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.094862:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d8e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6461:1866777604921344:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298643.094896:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.095481:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d8e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6461:1866777604921344:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298643.095492:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298643.095520:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609443456:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 757us (850us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.095526:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15057 00000100:00100000:3.0:1780298643.095671:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.095695:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.095742:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609443584 00000100:00100000:2.0:1780298643.095743:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609443712 00000100:00100000:1.0:1780298643.095746:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609443584 00000100:00100000:2.0:1780298643.095749:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609443712 00002000:00010000:1.0:1780298643.095764:0:19951:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (6243739->18446744073709551615) 00002000:00100000:1.0:1780298643.095768:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24304fb80 with x1866777609443584 ext(6243739->18446744073709551615) 00002000:00010000:2.0:1780298643.095769:0:8363:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4194304->18446744073709551615) 00002000:00100000:2.0:1780298643.095774:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24304ed80 with x1866777609443712 ext(4194304->18446744073709551615) 00002000:00010000:1.0:1780298643.095774:0:19951:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24304fb80. 00002000:00010000:2.0:1780298643.095782:0:8363:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24304ed80. 00000100:00100000:1.0:1780298643.095801:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6082 00000100:00100000:3.0:1780298643.095813:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6083 00000100:00100000:1.0:1780298643.095814:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609443584:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:3.0:1780298643.095825:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609443712:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:3.0:1780298643.095906:0:16972:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:3.0:1780298643.095910:0:16972:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:3.0:1780298643.095913:0:16972:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:3.0:1780298643.095915:0:16972:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:3.0:1780298643.095918:0:16972:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 88 -> 88 00000100:00100000:3.0:1780298643.095947:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609443712:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 126us (251us total) trans 4294969215 rc 0/0 00000100:00100000:3.0:1780298643.095952:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6083 00002000:00010000:3.0:1780298643.095955:0:16972:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4194304->18446744073709551615) 00002000:00100000:3.0:1780298643.095957:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24304ed80 with x1866777609443712 ext(4194304->18446744073709551615) 00002000:00010000:3.0:1780298643.095962:0:16972:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24304ed80. 00002000:00010000:1.0:1780298643.095989:0:19951:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 6133578 -> 6243739 00002000:00010000:1.0:1780298643.095993:0:19951:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298643.095995:0:19951:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298643.095998:0:19951:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298643.096000:0:19951:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 392 -> 392 00000100:00100000:1.0:1780298643.096028:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609443584:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 220us (359us total) trans 4294969654 rc 0/0 00000100:00100000:1.0:1780298643.096032:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6082 00002000:00010000:1.0:1780298643.096035:0:19951:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (6243739->18446744073709551615) 00002000:00100000:1.0:1780298643.096037:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24304fb80 with x1866777609443584 ext(6243739->18446744073709551615) 00002000:00010000:1.0:1780298643.096040:0:19951:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24304fb80. 00000100:00100000:3.0:1780298643.097082:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.097118:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609443840 00000100:00100000:1.0:1780298643.097122:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609443840 00002000:00100000:1.0:1780298643.097154:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (6135808->6246399) req@ffff9cf24304ca80 x1866777609443840/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.097165:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24304ca80 with x1866777609443840 ext(6135808->6246399) 00002000:00010000:1.0:1780298643.097168:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24304ca80 00000100:00100000:1.0:1780298643.097192:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6084 00000100:00100000:1.0:1780298643.097201:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609443840:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298643.098202:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609443840:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 1001us (1118us total) trans 0 rc 110592/110592 00000100:00100000:1.0:1780298643.098210:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6084 00002000:00100000:1.0:1780298643.098217:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (6135808->6246399) req@ffff9cf24304ca80 x1866777609443840/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/440 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 110592/110592 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.098225:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24304ca80 with x1866777609443840 ext(6135808->6246399) 00002000:00010000:1.0:1780298643.098228:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24304ca80 00000100:00100000:3.0:1780298643.099425:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.099456:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609443968 00000100:00100000:0.0:1780298643.099460:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609443968 00002000:00100000:0.0:1780298643.099478:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (6234112->6243738) req@ffff9cf24304df80 x1866777609443968/t0(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 10120/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.099485:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24304df80 with x1866777609443968 ext(6234112->6243738) 00002000:00010000:0.0:1780298643.099488:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24304df80 00000100:00100000:0.0:1780298643.099506:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6085 00000100:00100000:0.0:1780298643.099514:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609443968:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:0.0:1780298643.099851:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609443968:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 338us (425us total) trans 4294969655 rc 0/0 00000100:00100000:0.0:1780298643.099858:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6085 00002000:00100000:0.0:1780298643.099864:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (6234112->6243738) req@ffff9cf24304df80 x1866777609443968/t4294969655(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 10120/448 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.099872:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24304df80 with x1866777609443968 ext(6234112->6243738) 00002000:00010000:0.0:1780298643.099875:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24304df80 00000100:00100000:0.0:1780298643.100496:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.100529:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609444096 00000100:00100000:1.0:1780298643.100533:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609444096 00000100:00100000:1.0:1780298643.100558:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9177 00000100:00100000:1.0:1780298643.100564:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609444096:12345-192.168.203.25@tcp:16:fsx.0 00000001:00080000:2.0:1780298643.101392:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969655 is committed 00000100:00100000:1.0:1780298643.101475:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609444096:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 909us (977us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.101481:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9177 00000100:00100000:2.0:1780298643.102370:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.102403:0:9976:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609444224 00000100:00100000:0.0:1780298643.102407:0:9976:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609444224 00000100:00100000:0.0:1780298643.102439:0:9976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61968 00000100:00100000:0.0:1780298643.102445:0:9976:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+133:29360:x1866777609444224:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.102458:0:9976:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.102477:0:9976:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae600/0xaf35132e63033a88 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 68 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2bf9 expref: -99 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.102541:0:9976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafa00/0xaf35132e63033a8f lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 69 type: IBT flags: 0x40000000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.102552:0:9976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafa00/0xaf35132e63033a8f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 69 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.102560:0:9976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafa00/0xaf35132e63033a8f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 69 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1780298643.102577:0:9976:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 10438043 00000004:00200000:0.0:1780298643.102580:0:9976:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1780298643.102594:0:9976:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafa00/0xaf35132e63033a8f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 69 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.102611:0:9976:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae600/0xaf35132e63033a88 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 69 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bf9 expref: 135 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.102623:0:9976:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafa00/0xaf35132e63033a8f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 68 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bf9 expref: 134 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.102628:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298643.102630:0:9976:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1780298643.102632:0:9976:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bafa00, rc 0) 00000100:00100000:0.0:1780298643.102657:0:9976:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+134:29360:x1866777609444224:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 213us (288us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298643.102661:0:9976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61968 00000100:00100000:3.0:1780298643.103809:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.103854:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609444352 00000100:00100000:3.0:1780298643.103862:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.103864:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609444352 00000100:00100000:2.0:1780298643.103897:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609444480 00000100:00100000:2.0:1780298643.103901:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609444480 00000100:00100000:1.0:1780298643.103913:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9178 00000100:00100000:1.0:1780298643.103926:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609444352:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298643.103931:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9179 00010000:00010000:1.0:1780298643.103938:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298643.103943:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609444480:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.103952:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000670f33f0/0xaf35132e63033a96 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2c00 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.103953:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.103965:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e63033a9d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2c07 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.104007:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604921600:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.104008:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36d4b4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604921472:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.104067:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.104068:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484173c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.104805:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36d4b4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604921472:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.104822:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e63033a9d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c07 expref: 7 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.104835:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e63033a9d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c07 expref: 7 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.104842:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf200, rc 0) 00002000:00010000:1.0:1780298643.104845:0:12101:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks: 392 -> 416 00000100:00100000:1.0:1780298643.104856:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604921600:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.104871:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609444480:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 931us (1014us total) trans 0 rc 301/301 00010000:00010000:1.0:1780298643.104875:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000670f33f0/0xaf35132e63033a96 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c00 expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.104880:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9179 00010000:00010000:1.0:1780298643.104895:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000670f33f0/0xaf35132e63033a96 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c00 expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.104904:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92e00, rc 0) 00000100:00100000:1.0:1780298643.104933:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609444352:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1012us (1130us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.104944:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9178 00000100:00100000:2.0:1780298643.105758:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.105790:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609444608 00000100:00100000:1.0:1780298643.105794:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609444608 00000100:00100000:1.0:1780298643.105819:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9180 00000100:00100000:1.0:1780298643.105823:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:29360:x1866777609444608:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.105829:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.105836:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e63033aa4 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2c0e expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.105849:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e63033a7a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2beb expref: 8 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.105855:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(132 177) 00010000:00010000:1.0:1780298643.105857:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(133 177) 00010000:00010000:1.0:1780298643.105861:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e63033aa4 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [6074368->6078463] (req 6074368->6078463) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c0e expref: 7 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.105877:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e63033a7a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2beb expref: 8 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.105886:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e63033a7a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2beb expref: 8 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.105893:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604921728:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298643.105931:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.106416:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604921728:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298643.106431:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e63033aa4 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [6074368->6078463] (req 6074368->6078463) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c0e expref: 7 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.106440:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92600, rc 0) 00000100:00100000:1.0:1780298643.106464:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609444608:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 641us (706us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.106468:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9180 00000100:00100000:3.0:1780298643.106717:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.106757:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609444736 00000100:00100000:2.0:1780298643.106761:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609444736 00010000:00010000:2.0:1780298643.106776:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e63033a7a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2beb expref: 9 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.106807:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15058 00000100:00100000:2.0:1780298643.106815:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609444736:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298643.106825:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298643.106842:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e63033a7a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2beb expref: 9 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.106850:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e63033a7a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2beb expref: 9 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.106860:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e63033a7a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2beb expref: 9 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.106867:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(133 177) 00010000:00010000:2.0:1780298643.106869:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(133 177) 00010000:00010000:2.0:1780298643.106874:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e63033aa4 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 6074368->6078463) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c0e expref: 6 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.106880:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e63033aa4 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 6074368->6078463) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c0e expref: 6 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.106895:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e63033aa4 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 6074368->6078463) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c0e expref: 6 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.106906:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519e300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777604921856:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298643.106938:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.107538:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519e300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777604921856:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298643.107555:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298643.107596:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2400:x1866777609444736:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 781us (881us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298643.107606:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15058 00000100:00100000:3.0:1780298643.107752:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.107792:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609444864 00000100:00100000:1.0:1780298643.107797:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609444864 00002000:00100000:1.0:1780298643.107821:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (6074368->6078463) req@ffff9cf24304c700 x1866777609444864/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.107831:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24304c700 with x1866777609444864 ext(6074368->6078463) 00002000:00010000:1.0:1780298643.107835:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24304c700 00000100:00100000:1.0:1780298643.107858:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6086 00000100:00100000:1.0:1780298643.107867:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609444864:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298643.107980:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609444864:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 114us (226us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298643.107987:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6086 00002000:00100000:1.0:1780298643.107993:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (6074368->6078463) req@ffff9cf24304c700 x1866777609444864/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.108002:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24304c700 with x1866777609444864 ext(6074368->6078463) 00002000:00010000:1.0:1780298643.108006:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24304c700 00000100:00100000:0.0:1780298643.108973:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.109013:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609444992 00000100:00100000:2.0:1780298643.109018:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609444992 00002000:00100000:2.0:1780298643.109039:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (6078464->6082559) req@ffff9cf36d4b4380 x1866777609444992/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.109048:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36d4b4380 with x1866777609444992 ext(6078464->6082559) 00002000:00010000:2.0:1780298643.109052:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf36d4b4380 00000100:00100000:2.0:1780298643.109073:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6087 00000100:00100000:2.0:1780298643.109082:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609444992:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298643.109187:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609444992:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 107us (216us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298643.109194:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6087 00002000:00100000:2.0:1780298643.109201:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (6078464->6082559) req@ffff9cf36d4b4380 x1866777609444992/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.109210:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36d4b4380 with x1866777609444992 ext(6078464->6082559) 00002000:00010000:2.0:1780298643.109215:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf36d4b4380 00000100:00100000:2.0:1780298643.109925:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.109960:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609445120 00000100:00100000:1.0:1780298643.109964:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609445120 00000100:00100000:1.0:1780298643.109988:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61969 00000100:00100000:1.0:1780298643.109994:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+102:29360:x1866777609445120:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.110003:0:6477:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.110015:0:6477:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92c00/0xaf35132e63033aab lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 69 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2c15 expref: -99 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.110071:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92400/0xaf35132e63033ab2 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 70 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.110079:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92400/0xaf35132e63033ab2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 70 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.110085:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92400/0xaf35132e63033ab2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 70 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1780298643.110098:0:6477:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 10438043 00000004:00200000:1.0:1780298643.110100:0:6477:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1780298643.110109:0:6477:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92400/0xaf35132e63033ab2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 70 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.110140:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92c00/0xaf35132e63033aab lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 70 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c15 expref: 104 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.110151:0:6477:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92400/0xaf35132e63033ab2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 69 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c15 expref: 103 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.110156:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298643.110158:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1780298643.110160:0:6477:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92400, rc 0) 00000100:00100000:1.0:1780298643.110181:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+103:29360:x1866777609445120:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 189us (258us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.110185:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61969 00000100:00100000:3.0:1780298643.110973:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.111019:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609445248 00000100:00100000:1.0:1780298643.111024:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609445248 00000100:00100000:1.0:1780298643.111054:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9181 00000100:00100000:1.0:1780298643.111061:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609445248:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.111070:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.111078:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd0af3fe/0xaf35132e63033ab9 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2c1c expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.111102:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604921984:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.111150:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.111861:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604921984:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.111878:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000cd0af3fe/0xaf35132e63033ab9 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c1c expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.111889:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd0af3fe/0xaf35132e63033ab9 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c1c expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.111897:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93400, rc 0) 00000100:00100000:1.0:1780298643.111927:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609445248:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 866us (957us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.111933:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9181 00000100:00100000:2.0:1780298643.112778:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.112829:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609445376 00000100:00100000:3.0:1780298643.112838:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609445376 00000100:00100000:3.0:1780298643.112883:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9182 00000100:00100000:3.0:1780298643.112892:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609445376:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.112907:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.112922:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000fa1945a/0xaf35132e63033ac0 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2c23 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.112972:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604922112:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.113030:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.113668:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604922112:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.113685:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000fa1945a/0xaf35132e63033ac0 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c23 expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.113700:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000fa1945a/0xaf35132e63033ac0 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c23 expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.113708:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3000, rc 0) 00000100:00100000:0.0:1780298643.113735:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609445376:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 846us (958us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.113740:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9182 00000100:00100000:0.0:1780298643.114624:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.114653:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609445504 00000100:00100000:1.0:1780298643.114657:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609445504 00000100:00100000:1.0:1780298643.114690:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9183 00000100:00100000:1.0:1780298643.114697:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:29360:x1866777609445504:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.114704:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.114713:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63033ac7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2c2a expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.114730:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e90122d2/0xaf35132e63033a81 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bf2 expref: 12 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.114738:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(96 130) 00010000:00010000:1.0:1780298643.114742:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(97 130) 00010000:00010000:1.0:1780298643.114747:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63033ac7 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1081344->1089535] (req 1081344->1089535) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c2a expref: 7 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.114768:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e90122d2/0xaf35132e63033a81 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bf2 expref: 12 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.114777:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e90122d2/0xaf35132e63033a81 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bf2 expref: 12 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.114785:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604922240:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298643.114822:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.115407:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604922240:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298643.115422:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63033ac7 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1081344->1089535] (req 1081344->1089535) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c2a expref: 7 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.115433:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93000, rc 0) 00000100:00100000:1.0:1780298643.115467:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609445504:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 768us (840us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.115474:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9183 00000100:00100000:3.0:1780298643.115594:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.115628:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609445632 00000100:00100000:1.0:1780298643.115632:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609445632 00010000:00010000:1.0:1780298643.115645:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e90122d2/0xaf35132e63033a81 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bf2 expref: 13 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.115667:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15059 00000100:00100000:1.0:1780298643.115672:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609445632:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298643.115678:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298643.115686:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e90122d2/0xaf35132e63033a81 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bf2 expref: 13 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.115696:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e90122d2/0xaf35132e63033a81 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bf2 expref: 13 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.115707:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e90122d2/0xaf35132e63033a81 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bf2 expref: 13 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.115714:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(97 130) 00010000:00010000:1.0:1780298643.115717:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(97 130) 00010000:00010000:1.0:1780298643.115721:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63033ac7 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1081344->1089535) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c2a expref: 6 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.115726:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63033ac7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1081344->1089535) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c2a expref: 6 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.115740:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63033ac7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1081344->1089535) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c2a expref: 6 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.115747:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d8e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777604922368:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298643.115781:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.116356:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d8e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777604922368:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298643.116364:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298643.116388:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609445632:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 716us (797us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.116396:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15059 00000100:00100000:2.0:1780298643.116743:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.116773:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609445760 00000100:00100000:1.0:1780298643.116777:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609445760 00002000:00100000:1.0:1780298643.116794:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (1081344->1089535) req@ffff9cf244adb100 x1866777609445760/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.116801:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244adb100 with x1866777609445760 ext(1081344->1089535) 00002000:00010000:1.0:1780298643.116803:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf244adb100 00000100:00100000:1.0:1780298643.116819:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6088 00000100:00100000:1.0:1780298643.116824:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609445760:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298643.116928:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609445760:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 103us (185us total) trans 0 rc 8192/8192 00000100:00100000:1.0:1780298643.116932:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6088 00002000:00100000:1.0:1780298643.116936:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (1081344->1089535) req@ffff9cf244adb100 x1866777609445760/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/8632 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 8192/8192 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.116942:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244adb100 with x1866777609445760 ext(1081344->1089535) 00002000:00010000:1.0:1780298643.116945:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf244adb100 00000100:00100000:3.0:1780298643.117926:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.117967:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609445888 00000100:00100000:1.0:1780298643.117972:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609445888 00000100:00100000:3.0:1780298643.117979:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.118016:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609446016 00000100:00100000:1.0:1780298643.118020:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609446016 00000100:00100000:0.0:1780298643.118040:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9184 00000100:00100000:0.0:1780298643.118051:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2397:x1866777609445888:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298643.118058:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9185 00010000:00010000:0.0:1780298643.118061:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298643.118072:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609446016:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.118073:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63033ace lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2c31 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.118086:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.118090:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(133 177) 00010000:00010000:0.0:1780298643.118095:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(133 177) 00010000:00010000:1.0:1780298643.118099:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd0af3fe/0xaf35132e63033ad5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2c38 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.118100:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63033ace lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c31 expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.118150:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(97 130) 00010000:00010000:0.0:1780298643.118153:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63033ace lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c31 expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.118156:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(97 130) 00010000:00010000:1.0:1780298643.118169:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd0af3fe/0xaf35132e63033ad5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c38 expref: 13 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.118172:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf200, rc 0) 00010000:00010000:1.0:1780298643.118186:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000cd0af3fe/0xaf35132e63033ad5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c38 expref: 13 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.118196:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93400, rc 0) 00000100:00100000:0.0:1780298643.118201:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609445888:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 154us (278us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298643.118206:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9184 00000100:00100000:1.0:1780298643.118223:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609446016:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 156us (245us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.118231:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9185 00000100:00100000:0.0:1780298643.119218:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.119269:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609446144 00000100:00100000:3.0:1780298643.119277:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609446144 00000100:00100000:3.0:1780298643.119329:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9186 00000100:00100000:3.0:1780298643.119340:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:29360:x1866777609446144:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.119352:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.119363:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e63033adc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2c3f expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.119385:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63033ac7 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1081344->1089535) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c2a expref: 6 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.119396:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000cd0af3fe/0xaf35132e63033ad5 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c38 expref: 14 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.119410:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(97 130) 00010000:00010000:3.0:1780298643.119414:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(97 130) 00010000:00010000:3.0:1780298643.119419:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e63033adc lrc: 5/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [4014080->4022271] (req 4014080->4022271) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c3f expref: 14 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.119443:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000cd0af3fe/0xaf35132e63033ad5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c38 expref: 14 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.119456:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000cd0af3fe/0xaf35132e63033ad5 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c38 expref: 14 pid: 8356 timeout: 932 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.119467:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604922496:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298643.119531:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63033ac7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1081344->1089535) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c2a expref: 6 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.119542:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63033ac7 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1081344->1089535) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c2a expref: 6 pid: 12101 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.119552:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481faa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604922624:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298643.119577:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.120360:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.120389:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609446272 00000100:00100000:1.0:1780298643.120394:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609446272 00010000:00010000:1.0:1780298643.120413:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000cd0af3fe/0xaf35132e63033ad5 lrc: 5/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c38 expref: 15 pid: 8356 timeout: 932 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.120428:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604922496:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298643.120449:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15060 00000100:00100000:3.0:1780298643.120452:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481faa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604922624:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298643.120462:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2400:x1866777609446272:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298643.120472:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298643.120483:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000cd0af3fe/0xaf35132e63033ad5 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c38 expref: 15 pid: 8356 timeout: 932 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.120488:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e63033adc lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [4014080->4022271] (req 4014080->4022271) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c3f expref: 15 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.120504:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:1.0:1780298643.120518:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000cd0af3fe/0xaf35132e63033ad5 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c38 expref: 15 pid: 8356 timeout: 932 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.120535:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3200, rc 0) 00010000:00010000:1.0:1780298643.120535:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd0af3fe/0xaf35132e63033ad5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c38 expref: 15 pid: 8356 timeout: 932 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.120551:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609446400 00010000:00010000:1.0:1780298643.120558:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298643.120558:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609446400 00000100:00100000:3.0:1780298643.120578:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:29360:x1866777609446144:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1239us (1357us total) trans 0 rc 0/0 00010000:00010000:0.0:1780298643.120579:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63033ac7 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1081344->1089535) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c2a expref: 7 pid: 12101 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.120593:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609446272:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 132us (226us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298643.120614:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9186 00000100:00100000:1.0:1780298643.120620:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15060 00000100:00100000:1.0:1780298643.120666:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15061 00000100:00100000:1.0:1780298643.120686:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609446400:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298643.120694:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298643.120702:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63033ac7 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1081344->1089535) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c2a expref: 7 pid: 12101 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.120711:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63033ac7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1081344->1089535) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c2a expref: 7 pid: 12101 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.120721:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63033ac7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1081344->1089535) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c2a expref: 7 pid: 12101 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.120729:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(97 130) 00010000:00010000:1.0:1780298643.120732:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(97 130) 00010000:00010000:1.0:1780298643.120737:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e63033adc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [4014080->37568511] (req 4014080->4022271) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c3f expref: 12 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.120745:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e63033adc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [4014080->37568511] (req 4014080->4022271) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c3f expref: 12 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.120760:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e63033adc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [4014080->37568511] (req 4014080->4022271) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c3f expref: 12 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.120766:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d8e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:20253:1866777604922752:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298643.120798:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.121405:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d8e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:20253:1866777604922752:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298643.121414:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298643.121436:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609446400:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 763us (945us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.121441:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15061 00000100:00100000:0.0:1780298643.121624:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.121658:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609446528 00000100:00100000:1.0:1780298643.121663:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609446528 00002000:00100000:1.0:1780298643.121681:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (4014080->4018175) req@ffff9cf36d4b7100 x1866777609446528/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.121687:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36d4b7100 with x1866777609446528 ext(4014080->4018175) 00002000:00010000:1.0:1780298643.121690:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf36d4b7100 00000100:00100000:1.0:1780298643.121706:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6089 00000100:00100000:1.0:1780298643.121712:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609446528:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298643.121777:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609446528:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 66us (154us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298643.121784:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6089 00002000:00100000:1.0:1780298643.121791:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (4014080->4018175) req@ffff9cf36d4b7100 x1866777609446528/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.121797:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36d4b7100 with x1866777609446528 ext(4014080->4018175) 00002000:00010000:1.0:1780298643.121799:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf36d4b7100 00000100:00100000:2.0:1780298643.122514:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.122545:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609446656 00000100:00100000:0.0:1780298643.122549:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609446656 00002000:00100000:0.0:1780298643.122570:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (4018176->4022271) req@ffff9cf244ada300 x1866777609446656/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.122577:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244ada300 with x1866777609446656 ext(4018176->4022271) 00002000:00010000:0.0:1780298643.122580:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf244ada300 00000100:00100000:0.0:1780298643.122597:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6090 00000100:00100000:0.0:1780298643.122601:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ada300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609446656:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298643.122670:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ada300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609446656:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 68us (156us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298643.122677:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6090 00002000:00100000:0.0:1780298643.122684:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (4018176->4022271) req@ffff9cf244ada300 x1866777609446656/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.122692:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244ada300 with x1866777609446656 ext(4018176->4022271) 00002000:00010000:0.0:1780298643.122696:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf244ada300 00000100:00100000:2.0:1780298643.123611:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.123645:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609446784 00000100:00100000:1.0:1780298643.123649:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609446784 00000100:00100000:1.0:1780298643.123674:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9187 00000100:00100000:1.0:1780298643.123679:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609446784:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.123685:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.123694:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f8ed852/0xaf35132e63033ae3 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2c46 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.123721:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604922880:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.123765:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.124377:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604922880:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.124390:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002f8ed852/0xaf35132e63033ae3 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c46 expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.124400:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f8ed852/0xaf35132e63033ae3 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c46 expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.124406:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92c00, rc 0) 00000100:00100000:1.0:1780298643.124431:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609446784:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 752us (823us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.124436:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9187 00000100:00100000:2.0:1780298643.125269:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.125325:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609446912 00000100:00100000:3.0:1780298643.125332:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609446912 00000100:00100000:3.0:1780298643.125374:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9188 00000100:00100000:3.0:1780298643.125384:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609446912:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.125397:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.125410:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000075e99a4b/0xaf35132e63033aea lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2c4d expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.125453:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481faa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604923008:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.125513:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.126057:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481faa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604923008:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.126079:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000075e99a4b/0xaf35132e63033aea lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c4d expref: 7 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.126096:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000075e99a4b/0xaf35132e63033aea lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c4d expref: 7 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.126108:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3800, rc 0) 00000100:00100000:3.0:1780298643.126179:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609446912:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 796us (910us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.126187:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9188 00000100:00100000:0.0:1780298643.127255:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.127303:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609447040 00000100:00100000:1.0:1780298643.127308:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609447040 00002000:00100000:1.0:1780298643.127327:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (954368->962559) req@ffff9cf37f757b80 x1866777609447040/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.127335:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f757b80 with x1866777609447040 ext(954368->962559) 00002000:00010000:1.0:1780298643.127339:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f757b80 00000100:00100000:1.0:1780298643.127361:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6091 00000100:00100000:1.0:1780298643.127371:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f757b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609447040:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298643.127472:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f757b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609447040:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 103us (218us total) trans 0 rc 8192/8192 00000100:00100000:1.0:1780298643.127477:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6091 00002000:00100000:1.0:1780298643.127481:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (954368->962559) req@ffff9cf37f757b80 x1866777609447040/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/8632 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 8192/8192 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.127488:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f757b80 with x1866777609447040 ext(954368->962559) 00002000:00010000:1.0:1780298643.127490:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f757b80 00000100:00100000:2.0:1780298643.128354:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.128384:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609447168 00000100:00100000:1.0:1780298643.128389:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609447168 00000100:00100000:1.0:1780298643.128415:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9189 00000100:00100000:1.0:1780298643.128420:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b03100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609447168:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.128427:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.128435:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e90122d2/0xaf35132e63033af1 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2c54 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.128466:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604923136:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.128503:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.129077:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604923136:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.129091:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e90122d2/0xaf35132e63033af1 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c54 expref: 14 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.129102:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e90122d2/0xaf35132e63033af1 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c54 expref: 14 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.129108:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93c00, rc 0) 00000100:00100000:1.0:1780298643.129150:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b03100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609447168:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 730us (798us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.129155:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9189 00000100:00100000:0.0:1780298643.129931:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.129973:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609447296 00000100:00100000:2.0:1780298643.129979:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609447296 00000100:00100000:2.0:1780298643.130008:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9190 00000100:00100000:2.0:1780298643.130016:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609447296:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.130024:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.130033:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e63033af8 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2c5b expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.130060:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b01f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604923264:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.130098:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.130824:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b01f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604923264:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.130849:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e63033af8 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c5b expref: 14 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.130860:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e63033af8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c5b expref: 14 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.130872:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2200, rc 0) 00000100:00100000:0.0:1780298643.130901:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609447296:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 888us (972us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.130906:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9190 00000100:00100000:2.0:1780298643.131734:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.131766:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609447424 00000100:00100000:1.0:1780298643.131771:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609447424 00000100:00100000:1.0:1780298643.131796:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9191 00000100:00100000:1.0:1780298643.131801:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b03800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609447424:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.131808:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.131817:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000670f33f0/0xaf35132e63033aff lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2c62 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.131837:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e63033aa4 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 6074368->6078463) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c0e expref: 6 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.131847:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63033ace lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c31 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.131852:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(133 177) 00010000:00010000:1.0:1780298643.131855:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(133 177) 00010000:00010000:1.0:1780298643.131858:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000670f33f0/0xaf35132e63033aff lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [331776->339967] (req 331776->339967) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c62 expref: 10 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.131872:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63033ace lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c31 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.131880:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63033ace lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c31 expref: 10 pid: 8358 timeout: 932 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.131888:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604923392:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298643.131931:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e63033aa4 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 6074368->6078463) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c0e expref: 6 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.131940:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e63033aa4 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 6074368->6078463) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c0e expref: 6 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.131946:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604923520:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298643.131970:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.132612:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604923392:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298643.132618:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604923520:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298643.132631:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000670f33f0/0xaf35132e63033aff lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [331776->339967] (req 331776->339967) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c62 expref: 10 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.132639:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92e00, rc 0) 00000100:00100000:1.0:1780298643.132661:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b03800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609447424:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 860us (928us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.132665:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9191 00000100:00100000:2.0:1780298643.132822:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.132863:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.132885:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609447552 00000100:00100000:0.0:1780298643.132889:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609447552 00000100:00100000:1.0:1780298643.132908:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609447680 00010000:00010000:0.0:1780298643.132910:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63033ace lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c31 expref: 10 pid: 8358 timeout: 932 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.132913:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609447680 00010000:00010000:1.0:1780298643.132929:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e63033aa4 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 6074368->6078463) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c0e expref: 7 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.132941:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15062 00000100:00100000:0.0:1780298643.132948:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b01050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609447552:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298643.132957:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1780298643.132968:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15063 00010000:00010000:0.0:1780298643.132972:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63033ace lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c31 expref: 10 pid: 8358 timeout: 932 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.132984:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b01880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609447680:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298643.132985:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63033ace lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c31 expref: 10 pid: 8358 timeout: 932 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.132996:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298643.133005:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63033ace lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c31 expref: 10 pid: 8358 timeout: 932 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.133008:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e63033aa4 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 6074368->6078463) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c0e expref: 7 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.133025:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e63033aa4 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 6074368->6078463) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c0e expref: 7 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.133031:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298643.133037:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e63033aa4 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 6074368->6078463) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c0e expref: 7 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.133050:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(133 177) 00010000:00010000:1.0:1780298643.133053:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(133 177) 00000100:00100000:0.0:1780298643.133058:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b01050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609447552:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 108us (237us total) trans 0 rc 0/0 00010000:00010000:1.0:1780298643.133062:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000670f33f0/0xaf35132e63033aff lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [331776->33886207] (req 331776->339967) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c62 expref: 9 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.133077:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000670f33f0/0xaf35132e63033aff lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [331776->33886207] (req 331776->339967) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c62 expref: 9 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.133077:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15062 00010000:00010000:1.0:1780298643.133100:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000670f33f0/0xaf35132e63033aff lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [331776->33886207] (req 331776->339967) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c62 expref: 8 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.133111:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777604923648:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298643.133181:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.133867:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777604923648:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298643.133878:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298643.133911:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b01880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609447680:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 933us (1047us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.133926:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15063 00000100:00100000:3.0:1780298643.133978:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.134018:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609447808 00000100:00100000:0.0:1780298643.134021:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609447808 00000100:00100000:0.0:1780298643.134049:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6092 00000100:00100000:0.0:1780298643.134056:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481faa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609447808:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298643.134213:0:8363:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 6243739 -> 6243739 00002000:00010000:0.0:1780298643.134216:0:8363:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:0.0:1780298643.134218:0:8363:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:0.0:1780298643.134219:0:8363:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:0.0:1780298643.134221:0:8363:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 416 -> 416 00000100:00100000:0.0:1780298643.134244:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481faa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609447808:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 189us (326us total) trans 4294969656 rc 0/0 00000100:00100000:0.0:1780298643.134248:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6092 00000100:00100000:2.0:1780298643.135037:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.135062:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.135090:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609447936 00000100:00100000:1.0:1780298643.135095:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609447936 00000100:00100000:0.0:1780298643.135105:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609448064 00000100:00100000:0.0:1780298643.135111:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609448064 00000100:00100000:1.0:1780298643.135167:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9192 00000100:00100000:1.0:1780298643.135174:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b00000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609447936:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.135182:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.135203:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002f8ed852/0xaf35132e63033b06 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2c69 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.135210:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9193 00000100:00100000:0.0:1780298643.135224:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b01f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609448064:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.135231:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298643.135237:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604923776:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.135240:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c37b15a2/0xaf35132e63033b0d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2c70 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.135267:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604923904:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.135281:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.135308:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417e40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.136051:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604923776:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.136061:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604923904:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.136068:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002f8ed852/0xaf35132e63033b06 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c69 expref: 7 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.136081:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002f8ed852/0xaf35132e63033b06 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c69 expref: 7 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.136085:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c37b15a2/0xaf35132e63033b0d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c70 expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.136090:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92c00, rc 0) 00010000:00010000:0.0:1780298643.136104:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c37b15a2/0xaf35132e63033b0d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c70 expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.136115:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baee00, rc 0) 00000100:00100000:1.0:1780298643.136152:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b00000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609447936:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 950us (1087us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.136160:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b01f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609448064:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 939us (1095us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.136166:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9192 00000100:00100000:0.0:1780298643.136174:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9193 00000100:00100000:3.0:1780298643.137182:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.137211:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609448192 00000100:00100000:0.0:1780298643.137215:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609448192 00000100:00100000:0.0:1780298643.137251:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9194 00000100:00100000:0.0:1780298643.137265:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609448192:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.137273:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298643.137277:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:0.0:1780298643.137282:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005d10d3aa/0xaf35132e63033b14 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2c77 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.137309:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609448320 00000100:00100000:1.0:1780298643.137314:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609448320 00000100:00100000:0.0:1780298643.137317:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604924032:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.137348:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9195 00000100:00100000:1.0:1780298643.137362:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e580c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609448320:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.137369:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298643.137376:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417e40 going to sleep for 11 seconds 00010000:00010000:1.0:1780298643.137378:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63033b1b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2c7e expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.137402:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604924160:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.137445:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.138075:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604924032:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.138092:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604924160:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.138095:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005d10d3aa/0xaf35132e63033b14 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c77 expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.138110:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63033b1b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c7e expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.138112:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005d10d3aa/0xaf35132e63033b14 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c77 expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.138152:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63033b1b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c7e expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.138156:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baec00, rc 0) 00010000:00010000:1.0:1780298643.138162:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93000, rc 0) 00000100:00100000:1.0:1780298643.138196:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e580c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609448320:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 838us (921us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.138198:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609448192:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 935us (1069us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.138206:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9195 00000100:00100000:0.0:1780298643.138214:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9194 00000100:00100000:2.0:1780298643.139225:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.139247:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.139270:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609448448 00000100:00100000:1.0:1780298643.139278:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609448448 00000100:00100000:3.0:1780298643.139297:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609448576 00000100:00100000:3.0:1780298643.139302:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609448576 00000100:00100000:1.0:1780298643.139328:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9196 00000100:00100000:1.0:1780298643.139339:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b03100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609448448:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.139348:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298643.139354:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9197 00010000:00010000:1.0:1780298643.139354:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e90122d2/0xaf35132e63033b22 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2c85 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.139367:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b03800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609448576:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.139377:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298643.139383:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604924288:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.139387:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000075e99a4b/0xaf35132e63033b29 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2c8c expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.139413:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604924416:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.139438:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.139442:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.140108:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604924288:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.140156:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e90122d2/0xaf35132e63033b22 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c85 expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.140172:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e90122d2/0xaf35132e63033b22 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c85 expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.140179:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93c00, rc 0) 00000100:00100000:3.0:1780298643.140202:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604924416:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.140209:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b03100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609448448:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 875us (984us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.140214:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9196 00010000:00010000:3.0:1780298643.140226:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000075e99a4b/0xaf35132e63033b29 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c8c expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.140243:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000075e99a4b/0xaf35132e63033b29 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c8c expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.140253:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3800, rc 0) 00000100:00100000:3.0:1780298643.140390:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b03800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609448576:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1026us (1141us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.140401:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9197 00000100:00100000:2.0:1780298643.141332:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.141368:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609448704 00000100:00100000:1.0:1780298643.141372:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609448704 00002000:00100000:1.0:1780298643.141394:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x2c0000401:0x11:0x0] (1347584->1351679) req@ffff9cf241b00000 x1866777609448704/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.141405:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf241b00000 with x1866777609448704 ext(1347584->1351679) 00002000:00010000:1.0:1780298643.141409:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf241b00000 00000100:00100000:1.0:1780298643.141433:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6093 00000100:00100000:1.0:1780298643.141441:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b00000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609448704:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298643.141565:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b00000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609448704:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 124us (234us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298643.141573:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6093 00002000:00100000:1.0:1780298643.141582:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x2c0000401:0x11:0x0] (1347584->1351679) req@ffff9cf241b00000 x1866777609448704/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.141591:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf241b00000 with x1866777609448704 ext(1347584->1351679) 00002000:00010000:1.0:1780298643.141594:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf241b00000 00000100:00100000:2.0:1780298643.142753:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.142777:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.142801:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609448832 00000100:00100000:1.0:1780298643.142805:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609448832 00000100:00100000:3.0:1780298643.142819:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609448960 00000100:00100000:3.0:1780298643.142826:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609448960 00000100:00100000:1.0:1780298643.142831:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9198 00000100:00100000:1.0:1780298643.142838:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b01f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609448832:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.142844:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.142851:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000cd0af3fe/0xaf35132e63033b30 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2c93 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.142874:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9199 00000100:00100000:1.0:1780298643.142883:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604924544:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.142889:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609448960:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.142900:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.142911:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000024bbcf41/0xaf35132e63033b37 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2c9a expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.142926:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.142954:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604924672:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.143048:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.143739:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604924672:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.143740:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604924544:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.143763:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000cd0af3fe/0xaf35132e63033b30 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c93 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.143771:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000024bbcf41/0xaf35132e63033b37 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c9a expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.143775:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000cd0af3fe/0xaf35132e63033b30 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c93 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.143792:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93400, rc 0) 00010000:00010000:3.0:1780298643.143802:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000024bbcf41/0xaf35132e63033b37 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c9a expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.143815:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2c00, rc 0) 00000100:00100000:1.0:1780298643.143820:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b01f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609448832:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 983us (1069us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.143825:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9198 00000100:00100000:3.0:1780298643.143851:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609448960:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 967us (1072us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.143860:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9199 00000100:00100000:2.0:1780298643.144869:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.144890:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.144921:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609449088 00000100:00100000:1.0:1780298643.144923:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609449216 00000100:00100000:0.0:1780298643.144928:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609449088 00000100:00100000:1.0:1780298643.144929:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609449216 00000100:00100000:1.0:1780298643.144955:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9200 00000100:00100000:1.0:1780298643.144964:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609449216:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.144974:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298643.144975:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9201 00010000:00010000:1.0:1780298643.144985:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e63033b3e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2ca8 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.144996:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609449088:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.145006:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298643.145011:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604924800:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.145017:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63033b45 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2ca1 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.145057:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.145057:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f3100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604924928:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.145096:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417e40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.145815:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604924800:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.145830:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e63033b3e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ca8 expref: 14 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.145839:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e63033b3e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ca8 expref: 14 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.145847:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92a00, rc 0) 00000100:00100000:0.0:1780298643.145852:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f3100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604924928:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.145875:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63033b45 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ca1 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.145876:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609449216:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 914us (983us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.145890:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9200 00010000:00010000:0.0:1780298643.145904:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63033b45 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ca1 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.145917:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf200, rc 0) 00000100:00100000:0.0:1780298643.145944:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609449088:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 959us (1075us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.145955:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9201 00000100:00100000:2.0:1780298643.147044:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.147108:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609449344 00000100:00100000:1.0:1780298643.147113:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609449344 00002000:00100000:1.0:1780298643.147171:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (4358144->4362239) req@ffff9cf37f7bdf80 x1866777609449344/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.147181:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f7bdf80 with x1866777609449344 ext(4358144->4362239) 00002000:00010000:1.0:1780298643.147186:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f7bdf80 00000100:00100000:1.0:1780298643.147212:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6094 00000100:00100000:1.0:1780298643.147220:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609449344:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298643.147325:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609449344:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 107us (282us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298643.147331:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6094 00002000:00100000:1.0:1780298643.147336:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (4358144->4362239) req@ffff9cf37f7bdf80 x1866777609449344/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.147343:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f7bdf80 with x1866777609449344 ext(4358144->4362239) 00002000:00010000:1.0:1780298643.147346:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f7bdf80 00000100:00100000:2.0:1780298643.148138:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.148166:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609449472 00000100:00100000:0.0:1780298643.148170:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609449472 00002000:00100000:0.0:1780298643.148187:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (4362240->4366335) req@ffff9cf37f7bed80 x1866777609449472/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.148194:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f7bed80 with x1866777609449472 ext(4362240->4366335) 00002000:00010000:0.0:1780298643.148203:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f7bed80 00000100:00100000:0.0:1780298643.148218:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6095 00000100:00100000:0.0:1780298643.148223:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609449472:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298643.148313:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609449472:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 89us (190us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298643.148320:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6095 00002000:00100000:0.0:1780298643.148327:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (4362240->4366335) req@ffff9cf37f7bed80 x1866777609449472/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.148334:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f7bed80 with x1866777609449472 ext(4362240->4366335) 00002000:00010000:0.0:1780298643.148338:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f7bed80 00000100:00100000:2.0:1780298643.149153:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.149182:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609449600 00000100:00100000:0.0:1780298643.149185:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609449600 00000100:00100000:0.0:1780298643.149209:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9202 00000100:00100000:0.0:1780298643.149215:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609449600:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.149221:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.149228:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005d10d3aa/0xaf35132e63033b4c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2caf expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.149257:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f3100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604925056:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.149264:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.149314:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.149319:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609449728 00000100:00100000:3.0:1780298643.149324:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609449728 00000100:00100000:3.0:1780298643.149359:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9203 00000100:00100000:3.0:1780298643.149369:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255092680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609449728:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.149413:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.149422:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000075e99a4b/0xaf35132e63033b53 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2cb6 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.149450:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604925184:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.149486:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.149873:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f3100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604925056:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.149886:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005d10d3aa/0xaf35132e63033b4c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2caf expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.149895:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005d10d3aa/0xaf35132e63033b4c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2caf expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.149908:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baec00, rc 0) 00000100:00100000:0.0:1780298643.149927:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609449600:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 713us (776us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.149932:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9202 00000100:00100000:1.0:1780298643.150057:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604925184:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.150071:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000075e99a4b/0xaf35132e63033b53 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2cb6 expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.150080:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000075e99a4b/0xaf35132e63033b53 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2cb6 expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.150088:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3800, rc 0) 00000100:00100000:1.0:1780298643.150113:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255092680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609449728:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 748us (852us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.150118:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9203 00000100:00100000:2.0:1780298643.150898:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.150933:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609449856 00000100:00100000:0.0:1780298643.150939:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609449856 00000100:00100000:0.0:1780298643.150970:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9204 00000100:00100000:0.0:1780298643.150979:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255091500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609449856:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.150986:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.150992:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63033b5a lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2cbd expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.151010:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f3100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604925312:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.151042:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.151054:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.151088:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609449984 00000100:00100000:0.0:1780298643.151093:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609449984 00000100:00100000:0.0:1780298643.151143:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9205 00000100:00100000:0.0:1780298643.151150:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609449984:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.151155:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.151160:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e63033b61 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2cc4 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.151179:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604925440:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.151209:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417540 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.151654:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f3100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604925312:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.151666:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63033b5a lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2cbd expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.151675:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63033b5a lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2cbd expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.151683:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baee00, rc 0) 00000100:00100000:0.0:1780298643.151712:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255091500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609449856:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 734us (814us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.151720:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9204 00000100:00100000:0.0:1780298643.151892:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604925440:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.151904:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e63033b61 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2cc4 expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.151922:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e63033b61 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2cc4 expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.151932:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf600, rc 0) 00000100:00100000:0.0:1780298643.151954:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609449984:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 805us (904us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.151958:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9205 00000100:00100000:2.0:1780298643.152673:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.152710:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609450112 00000100:00100000:1.0:1780298643.152715:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609450112 00000100:00100000:1.0:1780298643.152754:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6096 00000100:00100000:1.0:1780298643.152763:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255090700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609450112:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298643.153002:0:19928:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 6243739 -> 6243739 00002000:00010000:1.0:1780298643.153007:0:19928:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298643.153009:0:19928:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298643.153012:0:19928:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298643.153014:0:19928:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 416 -> 432 00000100:00100000:1.0:1780298643.153044:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255090700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609450112:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 283us (370us total) trans 4294969657 rc 0/0 00000100:00100000:1.0:1780298643.153051:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6096 00000100:00100000:2.0:1780298643.153763:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.153798:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609450240 00000100:00100000:1.0:1780298643.153802:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609450240 00000100:00100000:1.0:1780298643.153831:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9206 00000100:00100000:1.0:1780298643.153840:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b6bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609450240:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.153848:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.153857:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000cd0af3fe/0xaf35132e63033b68 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2ccb expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.153885:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.153889:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604925568:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.153916:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609450368 00000100:00100000:3.0:1780298643.153922:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609450368 00000100:00100000:1.0:1780298643.153941:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.153962:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9207 00000100:00100000:3.0:1780298643.153974:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b68700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609450368:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.153983:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.154031:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000024bbcf41/0xaf35132e63033b6f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2cd2 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.154061:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604925696:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.154095:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.154471:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604925568:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.154488:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000cd0af3fe/0xaf35132e63033b68 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ccb expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.154502:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000cd0af3fe/0xaf35132e63033b68 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ccb expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.154511:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93400, rc 0) 00000100:00100000:1.0:1780298643.154539:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b6bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609450240:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 701us (775us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.154546:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9206 00000100:00100000:1.0:1780298643.154731:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604925696:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.154749:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000024bbcf41/0xaf35132e63033b6f lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2cd2 expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.154764:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000024bbcf41/0xaf35132e63033b6f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2cd2 expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.154775:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2c00, rc 0) 00000100:00100000:1.0:1780298643.154802:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b68700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609450368:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 833us (917us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.154807:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9207 00000100:00100000:2.0:1780298643.155456:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.155491:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.155494:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609450496 00000100:00100000:1.0:1780298643.155499:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609450496 00000100:00100000:1.0:1780298643.155527:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609450624 00000100:00100000:1.0:1780298643.155530:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609450624 00000100:00100000:0.0:1780298643.155540:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9208 00000100:00100000:2.0:1780298643.155549:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9209 00000100:00100000:0.0:1780298643.155550:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609450496:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.155559:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298643.155564:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254919880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609450624:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.155566:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63033b76 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2cd9 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.155577:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.155587:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e63033b7d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2ce0 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.155596:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604925824:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.155627:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25491b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604925952:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.155645:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417540 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.155662:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.156432:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604925824:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.156455:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63033b76 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2cd9 expref: 12 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.156466:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63033b76 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2cd9 expref: 12 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.156474:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf200, rc 0) 00000100:00100000:0.0:1780298643.156504:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609450496:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 955us (1047us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.156505:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25491b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604925952:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.156514:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9208 00010000:00010000:3.0:1780298643.156559:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e63033b7d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ce0 expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.156582:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e63033b7d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ce0 expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.156629:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2200, rc 0) 00000100:00100000:3.0:1780298643.156676:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254919880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609450624:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1118us (1185us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.156688:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9209 00000100:00100000:2.0:1780298643.157433:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.157474:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609450752 00000100:00100000:0.0:1780298643.157480:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609450752 00000100:00100000:0.0:1780298643.157519:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9210 00000100:00100000:0.0:1780298643.157531:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609450752:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.157541:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298643.157544:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:0.0:1780298643.157551:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e63033b84 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2ce7 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.157577:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609450880 00000100:00100000:0.0:1780298643.157587:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604926080:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.157593:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609450880 00000100:00100000:1.0:1780298643.157629:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9211 00000100:00100000:1.0:1780298643.157644:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254919180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609450880:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.157653:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298643.157654:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417540 going to sleep for 11 seconds 00010000:00010000:1.0:1780298643.157664:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e63033b8b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2cee expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.157697:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604926208:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.157737:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.158208:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604926080:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.158233:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e63033b84 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ce7 expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.158244:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e63033b84 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ce7 expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.158252:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf000, rc 0) 00000100:00100000:0.0:1780298643.158279:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609450752:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 753us (848us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.158295:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9210 00000100:00100000:1.0:1780298643.158311:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604926208:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.158330:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e63033b8b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2cee expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.158345:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e63033b8b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2cee expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.158354:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92a00, rc 0) 00000100:00100000:1.0:1780298643.158382:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254919180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609450880:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 745us (839us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.158389:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9211 00000100:00100000:2.0:1780298643.159056:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.159084:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609451008 00000100:00100000:3.0:1780298643.159091:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609451008 00000100:00100000:3.0:1780298643.159148:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9212 00000100:00100000:3.0:1780298643.159163:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:29360:x1866777609451008:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.159203:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.159214:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e63033b92 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2cf5 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.159228:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(97 130) 00010000:00010000:2.0:1780298643.159232:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(97 130) 00010000:00010000:2.0:1780298643.159239:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e63033b92 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2551808->4014079] (req 2551808->2555903) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2cf5 expref: 7 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.159254:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e63033b92 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2551808->4014079] (req 2551808->2555903) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2cf5 expref: 7 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.159260:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3e00, rc 0) 00000100:00100000:2.0:1780298643.159284:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609451008:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 128us (229us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.159300:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9212 00000100:00100000:2.0:1780298643.159958:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.159995:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609451136 00000100:00100000:0.0:1780298643.160004:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609451136 00002000:00100000:0.0:1780298643.160035:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x12:0x0] (2551808->2555903) req@ffff9cf25491b100 x1866777609451136/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.160042:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25491b100 with x1866777609451136 ext(2551808->2555903) 00002000:00010000:0.0:1780298643.160045:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf25491b100 00000100:00100000:0.0:1780298643.160062:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6097 00000100:00100000:0.0:1780298643.160069:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609451136:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298643.160204:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609451136:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 136us (245us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298643.160209:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6097 00002000:00100000:0.0:1780298643.160214:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x12:0x0] (2551808->2555903) req@ffff9cf25491b100 x1866777609451136/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.160219:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25491b100 with x1866777609451136 ext(2551808->2555903) 00002000:00010000:0.0:1780298643.160221:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf25491b100 00000100:00100000:0.0:1780298643.161272:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.161304:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609451264 00000100:00100000:1.0:1780298643.161308:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609451264 00002000:00100000:1.0:1780298643.161326:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (2555904->2559999) req@ffff9cf2425f3800 x1866777609451264/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.161333:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2425f3800 with x1866777609451264 ext(2555904->2559999) 00002000:00010000:1.0:1780298643.161337:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2425f3800 00000100:00100000:1.0:1780298643.161353:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6098 00000100:00100000:1.0:1780298643.161359:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609451264:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298643.161426:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609451264:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 66us (153us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298643.161433:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6098 00002000:00100000:1.0:1780298643.161439:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (2555904->2559999) req@ffff9cf2425f3800 x1866777609451264/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.161446:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2425f3800 with x1866777609451264 ext(2555904->2559999) 00002000:00010000:1.0:1780298643.161449:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2425f3800 00000100:00100000:2.0:1780298643.162384:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.162418:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609451392 00000100:00100000:1.0:1780298643.162423:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609451392 00002000:00100000:1.0:1780298643.162441:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (2560000->2564095) req@ffff9cf37f734000 x1866777609451392/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.162447:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f734000 with x1866777609451392 ext(2560000->2564095) 00002000:00010000:1.0:1780298643.162450:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f734000 00000100:00100000:1.0:1780298643.162467:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6099 00000100:00100000:1.0:1780298643.162473:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f734000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609451392:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298643.162545:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f734000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609451392:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 72us (161us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298643.162551:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6099 00002000:00100000:1.0:1780298643.162556:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (2560000->2564095) req@ffff9cf37f734000 x1866777609451392/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.162562:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f734000 with x1866777609451392 ext(2560000->2564095) 00002000:00010000:1.0:1780298643.162565:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f734000 00000100:00100000:2.0:1780298643.163548:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.163582:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609451520 00000100:00100000:0.0:1780298643.163587:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609451520 00002000:00100000:0.0:1780298643.163608:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (2551808->2564095) req@ffff9cf37f734700 x1866777609451520/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 12776/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.163615:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f734700 with x1866777609451520 ext(2551808->2564095) 00002000:00010000:0.0:1780298643.163617:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f734700 00000100:00100000:0.0:1780298643.163632:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6100 00000100:00100000:0.0:1780298643.163638:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f734700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609451520:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:0.0:1780298643.164174:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f734700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609451520:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 528us (620us total) trans 4294969216 rc 0/0 00000100:00100000:0.0:1780298643.164185:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6100 00002000:00100000:0.0:1780298643.164198:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (2551808->2564095) req@ffff9cf37f734700 x1866777609451520/t4294969216(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 12776/448 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.164209:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f734700 with x1866777609451520 ext(2551808->2564095) 00002000:00010000:0.0:1780298643.164215:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f734700 00000100:00100000:0.0:1780298643.164830:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.164864:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609451648 00000100:00100000:3.0:1780298643.164870:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609451648 00000100:00100000:3.0:1780298643.164899:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9213 00000100:00100000:3.0:1780298643.164904:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609451648:12345-192.168.203.25@tcp:16:fsx.0 00000001:00080000:0.0:1780298643.165579:0:8351:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0000: transno 4294969216 is committed 00000001:00080000:0.0:1780298643.165587:0:8351:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0000: transno 4294969215 is committed 00000100:00100000:3.0:1780298643.165654:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609451648:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 748us (822us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298643.165660:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9213 00000100:00100000:2.0:1780298643.166503:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.166547:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609451776 00000100:00100000:1.0:1780298643.166554:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609451776 00000100:00100000:1.0:1780298643.166606:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9214 00000100:00100000:0.0:1780298643.166627:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.166628:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f734e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609451776:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.166642:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298643.166656:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609451904 00010000:00010000:1.0:1780298643.166657:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e90122d2/0xaf35132e63033b99 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2cfc expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.166661:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609451904 00000100:00100000:3.0:1780298643.166689:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9215 00000100:00100000:3.0:1780298643.166697:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609451904:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.166705:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298643.166713:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604926336:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.166721:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000052b89172/0xaf35132e63033ba0 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2d03 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.166775:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fb800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604926464:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.166787:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.166825:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856900 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.167456:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604926336:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.167481:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e90122d2/0xaf35132e63033b99 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2cfc expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.167499:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e90122d2/0xaf35132e63033b99 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2cfc expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.167512:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93c00, rc 0) 00000100:00100000:1.0:1780298643.167549:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f734e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609451776:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 934us (1049us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.167557:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9214 00002000:00010000:3.0:1780298643.167658:0:12101:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 88 -> 112 00000100:00100000:3.0:1780298643.167707:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fb800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604926464:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.167730:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000052b89172/0xaf35132e63033ba0 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d03 expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.167748:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000052b89172/0xaf35132e63033ba0 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d03 expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.167759:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2800, rc 0) 00000100:00100000:3.0:1780298643.167793:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609451904:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1098us (1179us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.167800:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9215 00000100:00100000:2.0:1780298643.168659:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.168701:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609452032 00000100:00100000:0.0:1780298643.168706:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609452032 00000100:00100000:2.0:1780298643.168713:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.168735:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609452160 00000100:00100000:0.0:1780298643.168738:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609452160 00000100:00100000:0.0:1780298643.168770:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9216 00000100:00100000:0.0:1780298643.168787:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f735c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609452032:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298643.168791:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9217 00010000:00010000:0.0:1780298643.168797:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298643.168811:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f735f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609452160:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.168811:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e63033ba7 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2d0a expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.168838:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.168851:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63033bae lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2d11 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.168856:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f2300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604926592:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.168906:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604926720:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.168909:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.168947:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.169498:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f2300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604926592:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.169528:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e63033ba7 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d0a expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.169548:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e63033ba7 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d0a expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.169558:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf600, rc 0) 00000100:00100000:0.0:1780298643.169592:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f735c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609452032:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 816us (936us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.169599:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9216 00000100:00100000:1.0:1780298643.169674:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604926720:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.169707:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63033bae lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d11 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.169724:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63033bae lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d11 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.169736:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93000, rc 0) 00000100:00100000:1.0:1780298643.169772:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f735f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609452160:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 971us (1061us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.169780:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9217 00000100:00100000:3.0:1780298643.170635:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.170676:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609452288 00000100:00100000:0.0:1780298643.170681:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609452288 00000100:00100000:0.0:1780298643.170717:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9218 00000100:00100000:0.0:1780298643.170726:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:29360:x1866777609452288:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.170737:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.170747:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63033bb5 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2d18 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.170785:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(133 177) 00010000:00010000:0.0:1780298643.170790:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(133 177) 00010000:00010000:0.0:1780298643.170801:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63033bb5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [282624->331775] (req 282624->290815) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d18 expref: 7 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.170815:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63033bb5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [282624->331775] (req 282624->290815) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d18 expref: 7 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.170822:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baee00, rc 0) 00000100:00100000:0.0:1780298643.170853:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609452288:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 129us (220us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298643.170865:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9218 00000100:00100000:2.0:1780298643.171717:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.171757:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609452416 00000100:00100000:1.0:1780298643.171763:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609452416 00002000:00100000:1.0:1780298643.171791:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (282624->286719) req@ffff9cf37f735880 x1866777609452416/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.171799:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f735880 with x1866777609452416 ext(282624->286719) 00002000:00010000:1.0:1780298643.171803:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f735880 00000100:00100000:1.0:1780298643.171822:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6101 00000100:00100000:1.0:1780298643.171828:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f735880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609452416:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298643.171945:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f735880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609452416:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 117us (229us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298643.171952:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6101 00002000:00100000:1.0:1780298643.171959:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (282624->286719) req@ffff9cf37f735880 x1866777609452416/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.171968:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f735880 with x1866777609452416 ext(282624->286719) 00002000:00010000:1.0:1780298643.171973:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f735880 00000100:00100000:3.0:1780298643.172730:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.172764:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609452544 00000100:00100000:0.0:1780298643.172769:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609452544 00002000:00100000:0.0:1780298643.172792:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (286720->290815) req@ffff9cf3481fa300 x1866777609452544/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.172800:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fa300 with x1866777609452544 ext(286720->290815) 00002000:00010000:0.0:1780298643.172803:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481fa300 00000100:00100000:0.0:1780298643.172822:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6102 00000100:00100000:0.0:1780298643.172828:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609452544:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298643.172938:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609452544:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 110us (211us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298643.172943:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6102 00002000:00100000:0.0:1780298643.172947:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (286720->290815) req@ffff9cf3481fa300 x1866777609452544/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.172953:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fa300 with x1866777609452544 ext(286720->290815) 00002000:00010000:0.0:1780298643.172956:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481fa300 00000100:00100000:3.0:1780298643.173963:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.173997:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.174001:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609452672 00000100:00100000:1.0:1780298643.174006:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609452672 00000100:00100000:1.0:1780298643.174033:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9219 00000100:00100000:1.0:1780298643.174045:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609452672:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.174053:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298643.174065:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609452800 00010000:00010000:1.0:1780298643.174066:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e63033bbc lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2d1f expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.174072:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609452800 00000100:00100000:1.0:1780298643.174103:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604926848:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.174112:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9220 00000100:00100000:2.0:1780298643.174157:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609452800:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.174167:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.174177:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e63033bc3 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2d26 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.174183:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.174222:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604926976:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.174262:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4900 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.174830:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604926848:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.174845:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e63033bbc lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d1f expref: 8 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.174858:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e63033bbc lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d1f expref: 8 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.174865:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92a00, rc 0) 00000100:00100000:1.0:1780298643.174898:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609452672:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 853us (992us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.174907:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9219 00000100:00100000:2.0:1780298643.175034:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604926976:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.175053:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e63033bc3 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d26 expref: 8 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.175067:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e63033bc3 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d26 expref: 8 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.175076:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3800, rc 0) 00000100:00100000:2.0:1780298643.175104:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609452800:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 985us (1108us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.175112:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9220 00000100:00100000:2.0:1780298643.175983:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.176014:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609452928 00000100:00100000:1.0:1780298643.176018:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609452928 00000100:00100000:2.0:1780298643.176032:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.176049:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609453056 00000100:00100000:1.0:1780298643.176052:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609453056 00000100:00100000:1.0:1780298643.176069:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9221 00000100:00100000:1.0:1780298643.176076:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f736d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609452928:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.176083:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298643.176084:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9222 00010000:00010000:1.0:1780298643.176091:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000cd0af3fe/0xaf35132e63033bca lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2d2d expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.176099:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f737b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609453056:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.176108:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298643.176115:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604927104:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.176117:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000010b6573d/0xaf35132e63033bd1 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2d34 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.176193:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604927232:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.176205:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.176234:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4900 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.176747:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604927104:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.176758:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000cd0af3fe/0xaf35132e63033bca lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d2d expref: 8 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.176767:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000cd0af3fe/0xaf35132e63033bca lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d2d expref: 8 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.176773:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93400, rc 0) 00000100:00100000:1.0:1780298643.176796:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f736d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609452928:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 722us (816us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.176801:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9221 00000100:00100000:3.0:1780298643.176977:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604927232:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.177008:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000010b6573d/0xaf35132e63033bd1 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d34 expref: 8 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.177028:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000010b6573d/0xaf35132e63033bd1 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d34 expref: 8 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.177040:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3a00, rc 0) 00000100:00100000:3.0:1780298643.177085:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f737b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609453056:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 991us (1053us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.177094:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9222 00000100:00100000:0.0:1780298643.177894:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.177935:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609453184 00000100:00100000:2.0:1780298643.177941:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609453184 00000100:00100000:2.0:1780298643.177971:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9223 00000100:00100000:2.0:1780298643.177978:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609453184:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.177989:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.177997:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63033bd8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2d3b expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.178013:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(97 130) 00010000:00010000:2.0:1780298643.178018:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(97 130) 00010000:00010000:2.0:1780298643.178027:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63033bd8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1454080->2551807] (req 1454080->1458175) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d3b expref: 8 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.178042:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63033bd8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1454080->2551807] (req 1454080->1458175) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d3b expref: 8 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.178052:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2800, rc 0) 00000100:00100000:2.0:1780298643.178083:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:29360:x1866777609453184:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 105us (191us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.178090:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9223 00000100:00100000:2.0:1780298643.178743:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.178774:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609453312 00000100:00100000:1.0:1780298643.178777:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609453312 00000100:00100000:1.0:1780298643.178805:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6103 00000100:00100000:1.0:1780298643.178812:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f736300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609453312:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298643.178957:0:19951:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:1.0:1780298643.178960:0:19951:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298643.178962:0:19951:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298643.178964:0:19951:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298643.178966:0:19951:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 112 -> 112 00000100:00100000:1.0:1780298643.178989:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f736300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609453312:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 178us (246us total) trans 4294969217 rc 0/0 00000100:00100000:1.0:1780298643.178993:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6103 00000100:00100000:3.0:1780298643.179823:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.179895:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609453440 00000100:00100000:2.0:1780298643.179901:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609453440 00000100:00100000:3.0:1780298643.179925:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.179973:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609453568 00000100:00100000:2.0:1780298643.179978:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609453568 00000100:00100000:0.0:1780298643.179984:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9224 00000100:00100000:1.0:1780298643.180002:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9225 00000100:00100000:0.0:1780298643.180007:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609453440:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298643.180016:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609453568:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.180023:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.180027:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.180030:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63033bdf lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2d49 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.180042:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e63033be6 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2d42 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.180055:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604927360:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.180100:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.180116:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604927488:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.180218:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417d80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.180918:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604927360:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.180936:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63033bdf lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d49 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.180952:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63033bdf lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d49 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.180956:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604927488:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.180970:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93000, rc 0) 00010000:00010000:2.0:1780298643.180980:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e63033be6 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d42 expref: 8 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.180994:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e63033be6 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d42 expref: 8 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.181003:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609453568:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 991us (1077us total) trans 0 rc 301/301 00010000:00010000:2.0:1780298643.181010:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf600, rc 0) 00000100:00100000:1.0:1780298643.181014:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9225 00000100:00100000:2.0:1780298643.181041:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609453440:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1047us (1220us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.181052:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9224 00000100:00100000:2.0:1780298643.181940:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.181970:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.181994:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609453696 00000100:00100000:1.0:1780298643.182001:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609453824 00000100:00100000:3.0:1780298643.182003:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609453696 00000100:00100000:1.0:1780298643.182006:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609453824 00000100:00100000:1.0:1780298643.182036:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9226 00000100:00100000:1.0:1780298643.182043:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f737800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609453824:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.182049:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.182058:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd0af3fe/0xaf35132e63033bed lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2d57 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.182063:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9227 00000100:00100000:2.0:1780298643.182081:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f737480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609453696:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298643.182090:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604927616:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.182091:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.182104:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63033bf4 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2d50 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.182160:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.182174:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604927744:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.182207:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.182759:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604927616:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.182772:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000cd0af3fe/0xaf35132e63033bed lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d57 expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.182785:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd0af3fe/0xaf35132e63033bed lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d57 expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.182793:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93400, rc 0) 00000100:00100000:1.0:1780298643.182827:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f737800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609453824:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 785us (857us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.182834:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9226 00000100:00100000:3.0:1780298643.182987:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604927744:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.183012:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63033bf4 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d50 expref: 8 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.183030:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63033bf4 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d50 expref: 8 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.183040:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3000, rc 0) 00000100:00100000:3.0:1780298643.183077:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f737480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609453696:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1003us (1138us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.183084:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9227 00000100:00100000:0.0:1780298643.183905:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.183943:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609453952 00000100:00100000:2.0:1780298643.183948:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609453952 00000100:00100000:2.0:1780298643.183978:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9228 00000100:00100000:2.0:1780298643.183984:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609453952:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.183993:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.184002:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000039c5f58c/0xaf35132e63033bfb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2d5e expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.184019:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63033bd8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [1454080->2551807] (req 1454080->1458175) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d3b expref: 8 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.184034:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(97 130) 00010000:00010000:2.0:1780298643.184038:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(98 130) 00010000:00010000:2.0:1780298643.184043:0:20252:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000039c5f58c/0xaf35132e63033bfb lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [1761280->1769471] (req 1761280->1769471) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d5e expref: 11 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.184062:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63033bd8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [1454080->2551807] (req 1454080->1458175) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d3b expref: 8 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.184076:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63033bd8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [1454080->2551807] (req 1454080->1458175) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d3b expref: 8 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.184088:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604927872:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298643.184152:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.184737:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604927872:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298643.184752:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000039c5f58c/0xaf35132e63033bfb lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [1761280->1769471] (req 1761280->1769471) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d5e expref: 11 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.184763:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3200, rc 0) 00000100:00100000:0.0:1780298643.184790:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:29360:x1866777609453952:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 806us (887us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298643.184798:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9228 00000100:00100000:3.0:1780298643.184967:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.185005:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609454080 00000100:00100000:0.0:1780298643.185009:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609454080 00010000:00010000:0.0:1780298643.185032:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63033bd8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [1454080->2551807] (req 1454080->1458175) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d3b expref: 9 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.185061:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15064 00000100:00100000:0.0:1780298643.185073:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609454080:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298643.185083:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298643.185094:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63033bd8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [1454080->2551807] (req 1454080->1458175) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d3b expref: 9 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.185102:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63033bd8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [1454080->2551807] (req 1454080->1458175) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d3b expref: 9 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.185115:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63033bd8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [1454080->2551807] (req 1454080->1458175) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d3b expref: 9 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.185140:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(98 130) 00010000:00010000:0.0:1780298643.185147:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(98 130) 00010000:00010000:0.0:1780298643.185158:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000039c5f58c/0xaf35132e63033bfb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [1761280->2547711] (req 1761280->1769471) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d5e expref: 10 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.185169:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000039c5f58c/0xaf35132e63033bfb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [1761280->2547711] (req 1761280->1769471) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d5e expref: 10 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.185199:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000039c5f58c/0xaf35132e63033bfb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [1761280->2547711] (req 1761280->1769471) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d5e expref: 10 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.185212:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f3b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:7400:1866777604928000:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298643.185256:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417d80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.186023:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.186042:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f3b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:7400:1866777604928000:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298643.186057:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298643.186067:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609454208 00000100:00100000:1.0:1780298643.186074:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609454208 00000100:00100000:0.0:1780298643.186095:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609454080:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1021us (1123us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298643.186105:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15064 00000100:00100000:1.0:1780298643.186159:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6104 00000100:00100000:1.0:1780298643.186175:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f736a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609454208:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298643.186317:0:8362:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:1.0:1780298643.186321:0:8362:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298643.186324:0:8362:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298643.186326:0:8362:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298643.186328:0:8362:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 112 -> 112 00000100:00100000:1.0:1780298643.186364:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f736a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609454208:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 197us (345us total) trans 4294969218 rc 0/0 00000100:00100000:1.0:1780298643.186371:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6104 00000100:00100000:2.0:1780298643.187061:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.187099:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609454336 00000100:00100000:0.0:1780298643.187104:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609454336 00000100:00100000:0.0:1780298643.187162:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9229 00000100:00100000:3.0:1780298643.187172:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.187178:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556abb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609454336:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.187188:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298643.187195:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609454464 00010000:00010000:0.0:1780298643.187198:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e63033c02 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2d65 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.187210:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609454464 00000100:00100000:0.0:1780298643.187233:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604928128:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.187258:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9230 00000100:00100000:2.0:1780298643.187267:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609454464:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.187277:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298643.187282:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417d80 going to sleep for 11 seconds 00010000:00010000:2.0:1780298643.187298:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e63033c09 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2d6c expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.187333:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2556a9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604928256:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.187371:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4900 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.187788:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604928128:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.187801:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e63033c02 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d65 expref: 8 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.187812:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e63033c02 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d65 expref: 8 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.187819:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf600, rc 0) 00000100:00100000:0.0:1780298643.187842:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556abb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609454336:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 673us (783us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.187847:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9229 00000100:00100000:0.0:1780298643.187958:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2556a9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604928256:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.187981:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e63033c09 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d6c expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.187995:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e63033c09 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d6c expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.188006:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2e00, rc 0) 00000100:00100000:0.0:1780298643.188034:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609454464:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 767us (866us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.188041:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9230 00000100:00100000:2.0:1780298643.188869:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.188898:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.188899:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609454592 00000100:00100000:3.0:1780298643.188906:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609454592 00000100:00100000:3.0:1780298643.188947:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609454720 00000100:00100000:3.0:1780298643.188950:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609454720 00000100:00100000:1.0:1780298643.188959:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9231 00000100:00100000:0.0:1780298643.188972:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9232 00000100:00100000:1.0:1780298643.188979:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556ab800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609454592:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298643.188988:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556ab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609454720:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.188991:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.188997:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.189003:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000186136f8/0xaf35132e63033c10 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2d73 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.189006:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e63033c17 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2d7a expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.189066:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604928384:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.189071:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604928512:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.189199:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.189201:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417d80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.189775:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604928384:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.189797:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000186136f8/0xaf35132e63033c10 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d73 expref: 8 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.189814:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000186136f8/0xaf35132e63033c10 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d73 expref: 8 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.189832:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93000, rc 0) 00000100:00100000:1.0:1780298643.189872:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556ab800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609454592:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 902us (1002us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.189881:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9231 00000100:00100000:0.0:1780298643.189924:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604928512:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.189942:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e63033c17 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d7a expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.189952:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e63033c17 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d7a expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.189959:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baea00, rc 0) 00000100:00100000:0.0:1780298643.189984:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556ab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609454720:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1001us (1089us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.189990:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9232 00000100:00100000:3.0:1780298643.190841:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.190883:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609454848 00000100:00100000:1.0:1780298643.190889:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609454848 00000100:00100000:3.0:1780298643.190895:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.190935:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609454976 00000100:00100000:0.0:1780298643.190940:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609454976 00000100:00100000:1.0:1780298643.190945:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9233 00000100:00100000:1.0:1780298643.190962:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609454848:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.190972:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298643.190982:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9234 00010000:00010000:1.0:1780298643.190984:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e63033c1e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2d81 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.190993:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609454976:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.191012:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.191022:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63033c25 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2d88 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.191040:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604928640:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.191064:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604928768:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.191098:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.191103:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417d80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.191741:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604928640:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.191770:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e63033c1e lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d81 expref: 8 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.191771:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604928768:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.191799:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e63033c1e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d81 expref: 8 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.191799:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63033c25 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d88 expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.191852:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92a00, rc 0) 00010000:00010000:0.0:1780298643.191853:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63033c25 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d88 expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.191865:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae000, rc 0) 00000100:00100000:1.0:1780298643.191910:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609454848:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 944us (1099us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.191910:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609454976:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 909us (1007us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.191931:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9234 00000100:00100000:1.0:1780298643.191943:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9233 00000100:00100000:2.0:1780298643.192805:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.192846:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609455104 00000100:00100000:1.0:1780298643.192851:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609455104 00000100:00100000:1.0:1780298643.192885:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9235 00000100:00100000:1.0:1780298643.192894:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609455104:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.192903:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.192913:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000cd0af3fe/0xaf35132e63033c2c lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2d8f expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.192931:0:19992:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000670f33f0/0xaf35132e63033aff lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [331776->33886207] (req 331776->339967) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c62 expref: 10 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.192939:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(133 177) 00010000:00010000:1.0:1780298643.192943:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(134 177) 00010000:00010000:1.0:1780298643.192947:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000cd0af3fe/0xaf35132e63033c2c lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [4571136->4579327] (req 4571136->4579327) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d8f expref: 8 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.192966:0:19992:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000670f33f0/0xaf35132e63033aff lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [331776->33886207] (req 331776->339967) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c62 expref: 10 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.192979:0:19992:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000670f33f0/0xaf35132e63033aff lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [331776->33886207] (req 331776->339967) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c62 expref: 10 pid: 12101 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.192989:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604928896:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298643.193032:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.193643:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604928896:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298643.193659:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000cd0af3fe/0xaf35132e63033c2c lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [4571136->4579327] (req 4571136->4579327) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d8f expref: 8 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.193676:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93400, rc 0) 00000100:00100000:1.0:1780298643.193707:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:29360:x1866777609455104:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 815us (903us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.193714:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9235 00000100:00100000:3.0:1780298643.193947:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.193990:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609455232 00000100:00100000:0.0:1780298643.193996:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609455232 00002000:00100000:0.0:1780298643.194028:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (4358144->4366335) req@ffff9cf3481fb480 x1866777609455232/t0(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 8680/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.194039:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fb480 with x1866777609455232 ext(4358144->4366335) 00010000:00010000:0.0:1780298643.194047:0:8364:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000670f33f0/0xaf35132e63033aff lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [331776->33886207] (req 331776->339967) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c62 expref: 11 pid: 12101 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.194062:0:8364:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000670f33f0/0xaf35132e63033aff lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [331776->33886207] (req 331776->339967) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c62 expref: 11 pid: 12101 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1780298643.194070:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf3481fb480 00000100:00100000:0.0:1780298643.194099:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 192 00000100:00100000:0.0:1780298643.194109:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609455232:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:0.0:1780298643.194652:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609455232:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 545us (706us total) trans 4294969658 rc 0/0 00000100:00100000:0.0:1780298643.194661:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 192 00002000:00100000:0.0:1780298643.194669:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (4358144->4366335) req@ffff9cf3481fb480 x1866777609455232/t4294969658(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 8680/448 e 0 to 0 dl 1780298654 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.194679:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fb480 with x1866777609455232 ext(4358144->4366335) 00010000:00010000:0.0:1780298643.194685:0:8364:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000670f33f0/0xaf35132e63033aff lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [331776->33886207] (req 331776->339967) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c62 expref: 13 pid: 12101 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.194700:0:8364:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000670f33f0/0xaf35132e63033aff lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [331776->33886207] (req 331776->339967) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c62 expref: 13 pid: 12101 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1780298643.194708:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf3481fb480 00000100:00100000:2.0:1780298643.195435:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.195479:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609455360 00000100:00100000:0.0:1780298643.195485:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609455360 00010000:00010000:0.0:1780298643.195508:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000670f33f0/0xaf35132e63033aff lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [331776->33886207] (req 331776->339967) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c62 expref: 13 pid: 12101 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.195543:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15065 00000100:00100000:0.0:1780298643.195552:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556aaa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609455360:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298643.195561:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:0.0:1780298643.195570:0:6461:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 432 -> 448 00010000:00010000:0.0:1780298643.195576:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000670f33f0/0xaf35132e63033aff lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [331776->33886207] (req 331776->339967) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c62 expref: 13 pid: 12101 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.195588:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000670f33f0/0xaf35132e63033aff lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [331776->33886207] (req 331776->339967) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c62 expref: 13 pid: 12101 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.195605:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000670f33f0/0xaf35132e63033aff lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [331776->33886207] (req 331776->339967) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c62 expref: 13 pid: 12101 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.195619:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(134 177) 00010000:00010000:0.0:1780298643.195623:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(134 177) 00010000:00010000:0.0:1780298643.195631:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000cd0af3fe/0xaf35132e63033c2c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [4571136->38125567] (req 4571136->4579327) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d8f expref: 7 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.195641:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000cd0af3fe/0xaf35132e63033c2c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [4571136->38125567] (req 4571136->4579327) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d8f expref: 7 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.195669:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000cd0af3fe/0xaf35132e63033c2c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [4571136->38125567] (req 4571136->4579327) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d8f expref: 7 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.195687:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f3b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777604929024:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298643.195735:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417d80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.196377:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f3b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777604929024:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298643.196391:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298643.196420:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556aaa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609455360:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 870us (985us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298643.196428:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15065 00000100:00100000:2.0:1780298643.196558:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.196601:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609455488 00000100:00100000:1.0:1780298643.196606:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609455488 00000100:00100000:1.0:1780298643.196648:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6105 00000100:00100000:1.0:1780298643.196657:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609455488:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298643.196781:0:19928:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 6243739 -> 6243739 00002000:00010000:1.0:1780298643.196784:0:19928:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298643.196787:0:19928:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298643.196789:0:19928:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298643.196791:0:19928:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 448 -> 464 00000100:00100000:1.0:1780298643.196824:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609455488:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 168us (268us total) trans 4294969659 rc 0/0 00000100:00100000:1.0:1780298643.196831:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6105 00000100:00100000:3.0:1780298643.197598:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.197651:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609455616 00000100:00100000:0.0:1780298643.197657:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609455616 00000100:00100000:0.0:1780298643.197688:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9236 00000100:00100000:3.0:1780298643.197693:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.197699:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609455616:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.197707:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.197715:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e63033c33 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2d96 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.197744:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604929152:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.197750:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609455744 00000100:00100000:1.0:1780298643.197757:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609455744 00000100:00100000:0.0:1780298643.197790:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417d80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.197818:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9237 00000100:00100000:1.0:1780298643.197830:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609455744:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.197840:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.197850:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63033c3a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2d9d expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.197887:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604929280:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.197928:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.198338:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604929152:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.198353:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e63033c33 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d96 expref: 13 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.198363:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e63033c33 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d96 expref: 13 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.198374:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf600, rc 0) 00000100:00100000:0.0:1780298643.198400:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609455616:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 705us (806us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.198406:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9236 00000100:00100000:1.0:1780298643.198561:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604929280:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.198581:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63033c3a lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d9d expref: 13 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.198595:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63033c3a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d9d expref: 13 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.198604:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93000, rc 0) 00000100:00100000:1.0:1780298643.198635:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609455744:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 809us (945us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.198642:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9237 00000100:00100000:3.0:1780298643.199368:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.199408:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609456000 00000100:00100000:1.0:1780298643.199414:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609456000 00000100:00100000:3.0:1780298643.199421:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.199462:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609455872 00000100:00100000:1.0:1780298643.199467:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609455872 00000100:00100000:0.0:1780298643.199473:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9238 00000100:00100000:2.0:1780298643.199500:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9239 00000100:00100000:0.0:1780298643.199500:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609456000:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.199516:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298643.199517:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609455872:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.199528:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.199531:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63033c41 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2dab expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.199538:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b93eb96b/0xaf35132e63033c48 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2da4 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.199587:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2556aa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604929408:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.199594:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604929536:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.199642:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.199645:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417d80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.200338:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2556aa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604929408:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.200358:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b93eb96b/0xaf35132e63033c48 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2da4 expref: 13 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.200370:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b93eb96b/0xaf35132e63033c48 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2da4 expref: 13 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.200381:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3800, rc 0) 00000100:00100000:0.0:1780298643.200414:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604929536:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.200416:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609455872:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 907us (996us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.200432:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9239 00010000:00010000:0.0:1780298643.200448:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63033c41 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2dab expref: 13 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.200459:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63033c41 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2dab expref: 13 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.200467:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf400, rc 0) 00000100:00100000:0.0:1780298643.200487:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609456000:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1005us (1121us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.200492:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9238 00000100:00100000:3.0:1780298643.201344:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.201370:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609456128 00000100:00100000:0.0:1780298643.201374:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609456128 00000100:00100000:3.0:1780298643.201379:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.201401:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609456256 00000100:00100000:0.0:1780298643.201404:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609456256 00000100:00100000:2.0:1780298643.201425:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9240 00000100:00100000:0.0:1780298643.201448:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9241 00000100:00100000:2.0:1780298643.201449:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609456128:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.201464:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298643.201470:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609456256:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.201475:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63033c4f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2db2 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.201477:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.201487:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63033c56 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2db9 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.201509:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2556aa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604929664:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.201516:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604929792:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.201568:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.201569:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417d80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.202188:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2556aa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604929664:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.202218:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63033c4f lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2db2 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.202232:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63033c4f lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2db2 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.202243:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2000, rc 0) 00000100:00100000:0.0:1780298643.202247:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604929792:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.202269:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63033c56 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2db9 expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.202281:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609456128:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 842us (932us total) trans 0 rc 301/301 00010000:00010000:0.0:1780298643.202298:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63033c56 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2db9 expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.202307:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9240 00010000:00010000:0.0:1780298643.202322:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae000, rc 0) 00000100:00100000:0.0:1780298643.202346:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609456256:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 885us (966us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.202352:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9241 00000100:00100000:3.0:1780298643.203040:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.203072:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609456384 00000100:00100000:2.0:1780298643.203077:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609456384 00000100:00100000:2.0:1780298643.203103:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9242 00000100:00100000:2.0:1780298643.203108:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e582c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:29360:x1866777609456384:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.203114:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.203122:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e63033c5d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2dc0 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.203155:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(98 130) 00010000:00010000:2.0:1780298643.203158:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(98 130) 00010000:00010000:2.0:1780298643.203165:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e63033c5d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [1036288->1761279] (req 1036288->1044479) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2dc0 expref: 9 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.203175:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e63033c5d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [1036288->1761279] (req 1036288->1044479) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2dc0 expref: 9 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.203181:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2200, rc 0) 00000100:00100000:2.0:1780298643.203206:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e582c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:29360:x1866777609456384:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 97us (166us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.203211:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9242 00000100:00100000:3.0:1780298643.203732:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.203759:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609456512 00000100:00100000:0.0:1780298643.203764:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609456512 00000100:00100000:0.0:1780298643.203793:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6106 00000100:00100000:0.0:1780298643.203799:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609456512:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298643.203920:0:8363:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:0.0:1780298643.203923:0:8363:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:0.0:1780298643.203925:0:8363:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:0.0:1780298643.203926:0:8363:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:0.0:1780298643.203928:0:8363:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 112 -> 128 00000100:00100000:0.0:1780298643.203952:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609456512:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 154us (220us total) trans 4294969219 rc 0/0 00000100:00100000:0.0:1780298643.203956:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6106 00000100:00100000:2.0:1780298643.204632:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.204670:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609456640 00000100:00100000:0.0:1780298643.204674:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609456640 00000100:00100000:2.0:1780298643.204680:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.204708:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9243 00000100:00100000:1.0:1780298643.204721:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609456768 00000100:00100000:0.0:1780298643.204721:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556abb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609456640:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298643.204728:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609456768 00010000:00010000:0.0:1780298643.204734:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.204746:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000058e2d0/0xaf35132e63033c64 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2dc7 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.204773:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9244 00000100:00100000:0.0:1780298643.204783:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604929920:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.204788:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556ab800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609456768:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.204796:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.204805:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000052b89172/0xaf35132e63033c6b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2dce expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.204830:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604930048:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.204834:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417d80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.204857:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856900 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.205418:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604929920:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.205434:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000058e2d0/0xaf35132e63033c64 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2dc7 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.205445:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000058e2d0/0xaf35132e63033c64 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2dc7 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.205458:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baea00, rc 0) 00000100:00100000:0.0:1780298643.205488:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556abb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609456640:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 774us (858us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.205498:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9243 00000100:00100000:1.0:1780298643.205561:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604930048:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.205582:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000052b89172/0xaf35132e63033c6b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2dce expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.205596:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000052b89172/0xaf35132e63033c6b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2dce expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.205607:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2800, rc 0) 00000100:00100000:1.0:1780298643.205636:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556ab800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609456768:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 856us (958us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.205641:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9244 00000100:00100000:3.0:1780298643.206233:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.206278:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609456896 00000100:00100000:0.0:1780298643.206294:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609456896 00000100:00100000:3.0:1780298643.206322:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.206331:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609457024 00000100:00100000:0.0:1780298643.206335:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609457024 00000100:00100000:1.0:1780298643.206362:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9245 00000100:00100000:2.0:1780298643.206371:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9246 00000100:00100000:1.0:1780298643.206376:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609456896:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298643.206385:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609457024:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.206385:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.206394:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.206395:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000186136f8/0xaf35132e63033c72 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2dd5 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.206402:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c47b04a/0xaf35132e63033c79 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2ddc expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.206422:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604930176:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.206429:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2556ab100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604930304:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.206477:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.206478:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.207154:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2556ab100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604930304:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.207169:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005c47b04a/0xaf35132e63033c79 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ddc expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.207178:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604930176:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.207181:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c47b04a/0xaf35132e63033c79 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ddc expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.207192:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3400, rc 0) 00010000:00010000:1.0:1780298643.207200:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000186136f8/0xaf35132e63033c72 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2dd5 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.207211:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000186136f8/0xaf35132e63033c72 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2dd5 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.207219:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93000, rc 0) 00000100:00100000:2.0:1780298643.207226:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609457024:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 844us (905us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.207231:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9246 00000100:00100000:1.0:1780298643.207241:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609456896:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 871us (1009us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.207249:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9245 00000100:00100000:3.0:1780298643.208264:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.208303:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609457152 00000100:00100000:1.0:1780298643.208307:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609457152 00002000:00100000:1.0:1780298643.208328:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (3002368->3010559) req@ffff9cf347cd6a00 x1866777609457152/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.208335:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf347cd6a00 with x1866777609457152 ext(3002368->3010559) 00002000:00010000:1.0:1780298643.208339:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf347cd6a00 00000100:00100000:1.0:1780298643.208355:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6107 00000100:00100000:1.0:1780298643.208361:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609457152:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298643.208482:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609457152:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 122us (219us total) trans 0 rc 8192/8192 00000100:00100000:1.0:1780298643.208487:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6107 00002000:00100000:1.0:1780298643.208490:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (3002368->3010559) req@ffff9cf347cd6a00 x1866777609457152/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/8632 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 8192/8192 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.208496:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf347cd6a00 with x1866777609457152 ext(3002368->3010559) 00002000:00010000:1.0:1780298643.208499:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf347cd6a00 00000100:00100000:3.0:1780298643.209348:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.209373:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.209392:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609457280 00000100:00100000:2.0:1780298643.209398:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609457280 00000100:00100000:0.0:1780298643.209403:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609457408 00000100:00100000:0.0:1780298643.209409:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609457408 00000100:00100000:2.0:1780298643.209443:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9247 00000100:00100000:0.0:1780298643.209456:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9248 00000100:00100000:2.0:1780298643.209472:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609457280:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298643.209472:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609457408:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.209497:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.209500:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.209506:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e63033c80 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2dea expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.209511:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63033c87 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2de3 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.209534:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604930432:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.209554:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2556ab100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604930560:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.209578:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417d80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.209602:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.210340:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604930432:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.210363:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2556ab100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604930560:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.210382:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e63033c80 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2dea expref: 13 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.210384:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63033c87 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2de3 expref: 13 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.210427:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63033c87 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2de3 expref: 13 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.210442:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2600, rc 0) 00010000:00010000:0.0:1780298643.210444:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e63033c80 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2dea expref: 13 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.210465:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baec00, rc 0) 00000100:00100000:2.0:1780298643.210471:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609457280:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1020us (1151us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.210477:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9247 00000100:00100000:0.0:1780298643.210515:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609457408:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1042us (1140us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.210524:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9248 00000100:00100000:3.0:1780298643.211425:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.211461:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609457536 00000100:00100000:1.0:1780298643.211468:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609457536 00000100:00100000:3.0:1780298643.211474:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.211519:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609457664 00000100:00100000:2.0:1780298643.211523:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609457664 00000100:00100000:1.0:1780298643.211525:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9249 00000100:00100000:1.0:1780298643.211541:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609457536:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.211550:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298643.211556:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9250 00010000:00010000:1.0:1780298643.211561:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e63033c8e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2df1 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.211567:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609457664:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.211580:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.211590:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e63033c95 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2df8 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.211598:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604930688:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.211620:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2556ab100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604930816:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.211650:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.211659:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.212364:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604930688:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.212376:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2556ab100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604930816:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.212382:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e63033c8e lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2df1 expref: 13 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.212393:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e63033c8e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2df1 expref: 13 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.212401:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e63033c95 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2df8 expref: 13 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.212401:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92a00, rc 0) 00010000:00010000:2.0:1780298643.212418:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e63033c95 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2df8 expref: 13 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.212430:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3800, rc 0) 00000100:00100000:1.0:1780298643.212433:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609457536:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 898us (1008us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.212440:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9249 00000100:00100000:2.0:1780298643.212464:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609457664:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 899us (988us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.212472:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9250 00000100:00100000:3.0:1780298643.213281:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.213330:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609457792 00000100:00100000:0.0:1780298643.213335:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609457792 00000100:00100000:0.0:1780298643.213369:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9251 00000100:00100000:0.0:1780298643.213375:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244813b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:29360:x1866777609457792:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.213385:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.213393:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63033c9c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2dff expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.213409:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000039c5f58c/0xaf35132e63033bfb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [1761280->2547711] (req 1761280->1769471) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d5e expref: 11 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.213415:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(98 130) 00010000:00010000:0.0:1780298643.213418:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(99 130) 00010000:00010000:0.0:1780298643.213422:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63033c9c lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [2125824->2134015] (req 2125824->2134015) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2dff expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.213437:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000039c5f58c/0xaf35132e63033bfb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [1761280->2547711] (req 1761280->1769471) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d5e expref: 11 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.213447:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000039c5f58c/0xaf35132e63033bfb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [1761280->2547711] (req 1761280->1769471) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d5e expref: 11 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.213455:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604930944:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298643.213491:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417d80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.214076:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604930944:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298643.214089:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63033c9c lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [2125824->2134015] (req 2125824->2134015) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2dff expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.214097:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf400, rc 0) 00000100:00100000:0.0:1780298643.214119:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244813b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:29360:x1866777609457792:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 744us (839us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298643.214157:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9251 00000100:00100000:2.0:1780298643.214306:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.214339:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609457920 00000100:00100000:0.0:1780298643.214343:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609457920 00010000:00010000:0.0:1780298643.214359:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000039c5f58c/0xaf35132e63033bfb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [1761280->2547711] (req 1761280->1769471) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d5e expref: 12 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.214381:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15066 00000100:00100000:0.0:1780298643.214385:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556ab100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609457920:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298643.214393:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298643.214403:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000039c5f58c/0xaf35132e63033bfb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [1761280->2547711] (req 1761280->1769471) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d5e expref: 12 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.214411:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000039c5f58c/0xaf35132e63033bfb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [1761280->2547711] (req 1761280->1769471) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d5e expref: 12 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.214427:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000039c5f58c/0xaf35132e63033bfb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [1761280->2547711] (req 1761280->1769471) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d5e expref: 12 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.214436:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(99 130) 00010000:00010000:0.0:1780298643.214439:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(99 130) 00010000:00010000:0.0:1780298643.214446:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63033c9c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [2125824->2551807] (req 2125824->2134015) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2dff expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.214455:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63033c9c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [2125824->2551807] (req 2125824->2134015) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2dff expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.214477:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63033c9c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [2125824->2551807] (req 2125824->2134015) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2dff expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.214487:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f3b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:8374:1866777604931072:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298643.214525:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417d80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.215058:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f3b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:8374:1866777604931072:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298643.215066:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298643.215088:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556ab100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609457920:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 702us (783us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298643.215093:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15066 00000100:00100000:3.0:1780298643.215343:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.215382:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609458048 00000100:00100000:1.0:1780298643.215388:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609458048 00000100:00100000:1.0:1780298643.215428:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6108 00000100:00100000:1.0:1780298643.215442:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244810a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609458048:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298643.215631:0:8362:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:1.0:1780298643.215635:0:8362:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298643.215638:0:8362:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298643.215640:0:8362:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298643.215643:0:8362:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 128 -> 144 00000100:00100000:1.0:1780298643.215678:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244810a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609458048:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 243us (336us total) trans 4294969220 rc 0/0 00000100:00100000:1.0:1780298643.215686:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6108 00000100:00100000:2.0:1780298643.216483:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.216518:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609458176 00000100:00100000:0.0:1780298643.216523:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609458176 00000100:00100000:0.0:1780298643.216554:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9252 00000100:00100000:0.0:1780298643.216561:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609458176:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.216570:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298643.216575:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:0.0:1780298643.216582:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000058e2d0/0xaf35132e63033ca3 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2e06 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.216600:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609458304 00000100:00100000:1.0:1780298643.216607:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609458304 00000100:00100000:0.0:1780298643.216620:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604931200:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.216646:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9253 00000100:00100000:1.0:1780298643.216658:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244813480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609458304:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.216668:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298643.216674:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417d80 going to sleep for 11 seconds 00010000:00010000:1.0:1780298643.216681:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63033caa lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2e0d expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.216719:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604931328:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.216761:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.217281:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604931200:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.217311:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000058e2d0/0xaf35132e63033ca3 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e06 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.217324:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000058e2d0/0xaf35132e63033ca3 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e06 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.217334:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baea00, rc 0) 00000100:00100000:1.0:1780298643.217340:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604931328:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.217364:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609458176:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 803us (880us total) trans 0 rc 301/301 00010000:00010000:1.0:1780298643.217366:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63033caa lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e0d expref: 13 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.217383:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9252 00010000:00010000:1.0:1780298643.217403:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63033caa lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e0d expref: 13 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.217417:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93000, rc 0) 00000100:00100000:1.0:1780298643.217455:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244813480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609458304:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 801us (884us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.217464:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9253 00000100:00100000:3.0:1780298643.218090:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.218108:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.218150:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609458432 00000100:00100000:2.0:1780298643.218156:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609458432 00000100:00100000:0.0:1780298643.218182:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609458560 00000100:00100000:0.0:1780298643.218189:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609458560 00000100:00100000:2.0:1780298643.218195:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9254 00000100:00100000:2.0:1780298643.218212:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244811c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609458432:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.218220:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.218229:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e63033cb1 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2e14 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.218232:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9255 00000100:00100000:0.0:1780298643.218252:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244811f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609458560:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298643.218264:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2556aa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604931456:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.218265:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.218282:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63033cb8 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2e1b expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.218325:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.218340:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604931584:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.218368:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417d80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.218964:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2556aa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604931456:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.218989:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e63033cb1 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e14 expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.219001:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e63033cb1 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e14 expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.219009:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2a00, rc 0) 00000100:00100000:3.0:1780298643.219042:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244811c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609458432:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 836us (953us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.219052:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9254 00000100:00100000:0.0:1780298643.219092:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604931584:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.219147:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63033cb8 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e1b expref: 13 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.219162:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63033cb8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e1b expref: 13 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.219172:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae000, rc 0) 00000100:00100000:0.0:1780298643.219207:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244811f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609458560:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 960us (1097us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.219212:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9255 00000100:00100000:2.0:1780298643.219915:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.219938:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609458688 00000100:00100000:3.0:1780298643.219943:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609458688 00000100:00100000:3.0:1780298643.219979:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9256 00000100:00100000:3.0:1780298643.219987:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556aa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:29360:x1866777609458688:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.219997:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.220006:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000024bbcf41/0xaf35132e63033cbf lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2e22 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.220019:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(134 177) 00010000:00010000:3.0:1780298643.220022:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(134 177) 00010000:00010000:3.0:1780298643.220029:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000024bbcf41/0xaf35132e63033cbf lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2072576->4571135] (req 2072576->2080767) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e22 expref: 13 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.220043:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000024bbcf41/0xaf35132e63033cbf lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2072576->4571135] (req 2072576->2080767) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e22 expref: 13 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.220052:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2c00, rc 0) 00000100:00100000:3.0:1780298643.220082:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556aa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:29360:x1866777609458688:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 96us (167us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298643.220089:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9256 00000100:00100000:2.0:1780298643.220813:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.220847:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609458816 00000100:00100000:0.0:1780298643.220852:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609458816 00000100:00100000:0.0:1780298643.220883:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6109 00000100:00100000:0.0:1780298643.220887:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609458816:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298643.221044:0:8364:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 6243739 -> 6243739 00002000:00010000:0.0:1780298643.221047:0:8364:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:0.0:1780298643.221049:0:8364:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:0.0:1780298643.221050:0:8364:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:0.0:1780298643.221051:0:8364:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 464 -> 480 00000100:00100000:0.0:1780298643.221076:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2397:x1866777609458816:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 189us (264us total) trans 4294969660 rc 0/0 00000100:00100000:0.0:1780298643.221081:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6109 00000100:00100000:1.0:1780298643.221847:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.221870:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609458944 00000100:00100000:3.0:1780298643.221877:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609458944 00000100:00100000:1.0:1780298643.221888:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.221922:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609459072 00000100:00100000:0.0:1780298643.221927:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609459072 00000100:00100000:3.0:1780298643.221972:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9257 00000100:00100000:3.0:1780298643.221988:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609458944:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298643.221993:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9258 00010000:00010000:3.0:1780298643.222009:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298643.222010:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609459072:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.222021:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.222029:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000075e99a4b/0xaf35132e63033cc6 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2e29 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.222030:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c47b04a/0xaf35132e63033ccd lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2e30 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.222090:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2556aad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604931712:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.222098:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244812680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604931840:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.222218:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.222231:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.222897:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2556aad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604931712:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.222918:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005c47b04a/0xaf35132e63033ccd lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e30 expref: 13 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.222926:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244812680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604931840:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.222958:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c47b04a/0xaf35132e63033ccd lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e30 expref: 13 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.222969:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000075e99a4b/0xaf35132e63033cc6 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e29 expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.222969:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3400, rc 0) 00010000:00010000:3.0:1780298643.222994:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000075e99a4b/0xaf35132e63033cc6 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e29 expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.223005:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3800, rc 0) 00000100:00100000:1.0:1780298643.223018:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609459072:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1011us (1127us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.223024:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9258 00000100:00100000:3.0:1780298643.223043:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609458944:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1058us (1192us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.223055:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9257 00000100:00100000:0.0:1780298643.223851:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.223906:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609459200 00000100:00100000:2.0:1780298643.223912:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609459200 00000100:00100000:2.0:1780298643.223944:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9259 00000100:00100000:3.0:1780298643.223947:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.223957:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609459200:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.223963:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.223972:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b93eb96b/0xaf35132e63033cd4 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2e37 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.223997:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2556a8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604931968:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.223999:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609459328 00000100:00100000:1.0:1780298643.224006:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609459328 00000100:00100000:2.0:1780298643.224034:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.224054:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9260 00000100:00100000:1.0:1780298643.224067:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244812680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609459328:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.224075:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.224083:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63033cdb lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2e3e expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.224108:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604932096:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.224171:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.224641:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2556a8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604931968:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.224656:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b93eb96b/0xaf35132e63033cd4 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e37 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.224667:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b93eb96b/0xaf35132e63033cd4 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e37 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.224675:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3800, rc 0) 00000100:00100000:2.0:1780298643.224704:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609459200:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 754us (853us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.224714:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9259 00000100:00100000:1.0:1780298643.224807:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604932096:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.224821:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63033cdb lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e3e expref: 13 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.224829:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63033cdb lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e3e expref: 13 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.224839:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93000, rc 0) 00000100:00100000:1.0:1780298643.224866:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244812680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609459328:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 806us (928us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.224875:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9260 00000100:00100000:0.0:1780298643.225592:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.225632:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609459456 00000100:00100000:1.0:1780298643.225637:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609459456 00000100:00100000:1.0:1780298643.225673:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9261 00000100:00100000:1.0:1780298643.225681:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:29360:x1866777609459456:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.225689:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.225697:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e63033ce2 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2e45 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.225715:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000024bbcf41/0xaf35132e63033cbf lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2072576->4571135] (req 2072576->2080767) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e22 expref: 13 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.225724:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(134 177) 00010000:00010000:1.0:1780298643.225729:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(135 177) 00010000:00010000:1.0:1780298643.225734:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e63033ce2 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [4386816->4390911] (req 4386816->4390911) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e45 expref: 10 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.225755:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000024bbcf41/0xaf35132e63033cbf lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2072576->4571135] (req 2072576->2080767) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e22 expref: 13 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.225767:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000024bbcf41/0xaf35132e63033cbf lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2072576->4571135] (req 2072576->2080767) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e22 expref: 13 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.225778:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604932224:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298643.225818:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.226355:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604932224:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298643.226381:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e63033ce2 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [4386816->4390911] (req 4386816->4390911) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e45 expref: 10 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.226388:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92a00, rc 0) 00000100:00100000:1.0:1780298643.226411:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:29360:x1866777609459456:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 733us (822us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.226415:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9261 00000100:00100000:3.0:1780298643.226606:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.226647:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609459584 00000100:00100000:1.0:1780298643.226652:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609459584 00010000:00010000:1.0:1780298643.226672:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000024bbcf41/0xaf35132e63033cbf lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2072576->4571135] (req 2072576->2080767) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e22 expref: 14 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.226696:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15067 00000100:00100000:1.0:1780298643.226701:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244811880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609459584:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298643.226709:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298643.226717:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000024bbcf41/0xaf35132e63033cbf lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [2072576->4571135] (req 2072576->2080767) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e22 expref: 14 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.226726:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000024bbcf41/0xaf35132e63033cbf lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [2072576->4571135] (req 2072576->2080767) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e22 expref: 14 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.226741:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000024bbcf41/0xaf35132e63033cbf lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [2072576->4571135] (req 2072576->2080767) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e22 expref: 14 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.226749:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(135 177) 00010000:00010000:1.0:1780298643.226751:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(135 177) 00010000:00010000:1.0:1780298643.226756:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e63033ce2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [4386816->4571135] (req 4386816->4390911) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e45 expref: 9 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.226762:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e63033ce2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [4386816->4571135] (req 4386816->4390911) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e45 expref: 9 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.226777:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e63033ce2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [4386816->4571135] (req 4386816->4390911) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e45 expref: 9 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.226783:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d8380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777604932352:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298643.226812:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.227498:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d8380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777604932352:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298643.227506:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298643.227527:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244811880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609459584:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 826us (924us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.227532:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15067 00000100:00100000:3.0:1780298643.227720:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.227753:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609459712 00000100:00100000:1.0:1780298643.227757:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609459712 00002000:00100000:1.0:1780298643.227777:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (4386816->4390911) req@ffff9cf244813100 x1866777609459712/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.227784:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244813100 with x1866777609459712 ext(4386816->4390911) 00002000:00010000:1.0:1780298643.227787:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf244813100 00000100:00100000:1.0:1780298643.227802:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6110 00000100:00100000:1.0:1780298643.227807:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244813100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609459712:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298643.227918:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244813100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609459712:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 111us (201us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298643.227922:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6110 00002000:00100000:1.0:1780298643.227926:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (4386816->4390911) req@ffff9cf244813100 x1866777609459712/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.227932:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244813100 with x1866777609459712 ext(4386816->4390911) 00002000:00010000:1.0:1780298643.227935:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf244813100 00000100:00100000:0.0:1780298643.228882:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.228910:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609459840 00000100:00100000:0.0:1780298643.228911:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.228918:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609459840 00000100:00100000:1.0:1780298643.228965:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609459968 00000100:00100000:1.0:1780298643.228970:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609459968 00000100:00100000:3.0:1780298643.228977:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9262 00000100:00100000:3.0:1780298643.228991:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609459840:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298643.228993:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9263 00010000:00010000:3.0:1780298643.229010:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298643.229023:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609459968:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.229025:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000fa1945a/0xaf35132e63033ce9 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2e4c expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.229041:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.229053:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63033cf0 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2e53 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.229076:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244813800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604932480:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.229089:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604932608:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.229206:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.229206:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417d80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.229820:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244813800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604932480:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.229854:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000fa1945a/0xaf35132e63033ce9 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e4c expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.229858:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604932608:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.229884:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000fa1945a/0xaf35132e63033ce9 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e4c expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.229891:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63033cf0 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e53 expref: 12 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.229918:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3000, rc 0) 00010000:00010000:1.0:1780298643.229927:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63033cf0 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e53 expref: 12 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.229940:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae400, rc 0) 00000100:00100000:3.0:1780298643.229960:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609459840:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 972us (1077us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.229981:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9262 00000100:00100000:1.0:1780298643.229981:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609459968:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 966us (1060us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.229999:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9263 00000100:00100000:2.0:1780298643.230918:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.230949:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609460096 00000100:00100000:2.0:1780298643.230955:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.230955:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609460096 00000100:00100000:1.0:1780298643.230992:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609460224 00000100:00100000:1.0:1780298643.230996:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609460224 00000100:00100000:3.0:1780298643.231011:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9264 00000100:00100000:3.0:1780298643.231065:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609460096:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298643.231072:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9265 00010000:00010000:3.0:1780298643.231076:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298643.231082:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609460224:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.231088:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000075e99a4b/0xaf35132e63033cf7 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2e5a expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.231090:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.231098:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63033cfe lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2e61 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.231155:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604932736:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.231182:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244813800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604932864:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.231202:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.231251:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.231793:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604932736:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.231808:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63033cfe lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e61 expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.231823:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63033cfe lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e61 expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.231832:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93000, rc 0) 00000100:00100000:1.0:1780298643.231861:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609460224:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 783us (906us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.231867:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9265 00000100:00100000:0.0:1780298643.232022:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244813800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604932864:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.232040:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000075e99a4b/0xaf35132e63033cf7 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e5a expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.232052:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000075e99a4b/0xaf35132e63033cf7 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e5a expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.232060:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3800, rc 0) 00000100:00100000:0.0:1780298643.232087:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609460096:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1028us (1173us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.232093:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9264 00000100:00100000:2.0:1780298643.232844:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.232880:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609460352 00000100:00100000:0.0:1780298643.232885:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609460352 00000100:00100000:2.0:1780298643.232907:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.232923:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609460480 00000100:00100000:0.0:1780298643.232926:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609460480 00000100:00100000:3.0:1780298643.232935:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9266 00000100:00100000:1.0:1780298643.232972:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9267 00000100:00100000:1.0:1780298643.232985:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556ab480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609460480:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298643.232990:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609460352:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.232997:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.233000:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.233010:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e63033d0c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2e68 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.233011:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000038a88423/0xaf35132e63033d05 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2e6f expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.233068:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244810e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604932992:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.233082:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604933120:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.233121:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.233163:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.233857:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244810e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604932992:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.233875:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e63033d0c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e68 expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.233887:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e63033d0c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e68 expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.233895:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2400, rc 0) 00000100:00100000:1.0:1780298643.233926:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609460352:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 983us (1083us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.233932:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9266 00000100:00100000:0.0:1780298643.233957:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604933120:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.233989:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000038a88423/0xaf35132e63033d05 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e6f expref: 13 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.234007:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038a88423/0xaf35132e63033d05 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e6f expref: 13 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.234016:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93e00, rc 0) 00000100:00100000:0.0:1780298643.234053:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556ab480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609460480:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1073us (1148us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.234065:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9267 00000100:00100000:3.0:1780298643.234871:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.234908:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609460608 00000100:00100000:0.0:1780298643.234914:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609460608 00002000:00100000:0.0:1780298643.234941:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (5152768->5156863) req@ffff9cf244810000 x1866777609460608/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.234951:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244810000 with x1866777609460608 ext(5152768->5156863) 00002000:00010000:0.0:1780298643.234955:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf244810000 00000100:00100000:0.0:1780298643.234978:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6111 00000100:00100000:0.0:1780298643.234987:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244810000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609460608:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298643.235105:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244810000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609460608:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 119us (233us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298643.235113:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6111 00002000:00100000:0.0:1780298643.235121:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (5152768->5156863) req@ffff9cf244810000 x1866777609460608/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.235157:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244810000 with x1866777609460608 ext(5152768->5156863) 00002000:00010000:0.0:1780298643.235162:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf244810000 00010000:00010000:1.0:1780298643.235273:0:7850:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1780298643.236161:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.236218:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609460736 00000100:00100000:3.0:1780298643.236224:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609460736 00002000:00100000:3.0:1780298643.236248:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x2c0000401:0x11:0x0] (5128192->5193727) req@ffff9cf2556a8a80 x1866777609460736/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 504/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298643.236257:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2556a8a80 with x1866777609460736 ext(5128192->5193727) 00002000:00010000:3.0:1780298643.236261:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2556a8a80 00000100:00100000:3.0:1780298643.236298:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6112 00000100:00100000:3.0:1780298643.236307:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609460736:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298643.237273:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609460736:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 965us (1112us total) trans 0 rc 61440/61440 00000100:00100000:1.0:1780298643.237279:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6112 00002000:00100000:1.0:1780298643.237296:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x2c0000401:0x11:0x0] (5128192->5193727) req@ffff9cf2556a8a80 x1866777609460736/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 504/440 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 61440/61440 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.237307:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2556a8a80 with x1866777609460736 ext(5128192->5193727) 00002000:00010000:1.0:1780298643.237311:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2556a8a80 00000100:00100000:3.0:1780298643.238225:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.238255:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.238270:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609460864 00000100:00100000:0.0:1780298643.238277:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609460864 00000100:00100000:1.0:1780298643.238296:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609460992 00000100:00100000:1.0:1780298643.238303:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609460992 00000100:00100000:0.0:1780298643.238332:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9268 00000100:00100000:0.0:1780298643.238349:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244812300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609460864:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298643.238357:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9269 00010000:00010000:0.0:1780298643.238359:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.238375:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005d10d3aa/0xaf35132e63033d13 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2e76 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.238376:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244812d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609460992:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.238395:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.238407:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63033d1a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2e7d expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.238426:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f1180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604933248:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.238449:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604933376:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.238492:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.238492:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417540 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.239242:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f1180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604933248:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.239275:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005d10d3aa/0xaf35132e63033d13 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e76 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.239306:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604933376:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.239308:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005d10d3aa/0xaf35132e63033d13 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e76 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.239328:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baec00, rc 0) 00010000:00010000:3.0:1780298643.239338:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63033d1a lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e7d expref: 13 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.239352:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63033d1a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e7d expref: 13 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.239361:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93000, rc 0) 00000100:00100000:1.0:1780298643.239371:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244812300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609460864:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1026us (1144us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.239379:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9268 00000100:00100000:3.0:1780298643.239395:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244812d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609460992:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1025us (1135us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.239404:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9269 00000100:00100000:2.0:1780298643.240220:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.240261:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609461120 00000100:00100000:0.0:1780298643.240267:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609461120 00000100:00100000:0.0:1780298643.240335:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9270 00000100:00100000:0.0:1780298643.240350:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609461120:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298643.240354:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:0.0:1780298643.240365:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.240379:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e63033d21 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2e84 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.240385:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609461248 00000100:00100000:1.0:1780298643.240392:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609461248 00000100:00100000:0.0:1780298643.240422:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604933504:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.240434:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9271 00000100:00100000:1.0:1780298643.240450:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609461248:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.240459:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.240477:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000056a806f/0xaf35132e63033d28 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2e8b expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.240482:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417e40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.240516:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604933632:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.240547:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.241193:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604933632:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.241201:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604933504:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.241222:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000056a806f/0xaf35132e63033d28 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e8b expref: 13 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.241230:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e63033d21 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e84 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.241249:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000056a806f/0xaf35132e63033d28 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e8b expref: 13 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.241257:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e63033d21 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e84 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.241269:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92800, rc 0) 00010000:00010000:0.0:1780298643.241278:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae400, rc 0) 00000100:00100000:2.0:1780298643.241317:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609461248:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 875us (967us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.241328:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609461120:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 979us (1102us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.241335:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9271 00000100:00100000:0.0:1780298643.241348:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9270 00000100:00100000:2.0:1780298643.242254:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.242304:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609461376 00000100:00100000:0.0:1780298643.242310:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609461376 00000100:00100000:2.0:1780298643.242316:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.242346:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609461504 00000100:00100000:0.0:1780298643.242351:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609461504 00000100:00100000:1.0:1780298643.242359:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9272 00000100:00100000:1.0:1780298643.242369:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556aad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609461376:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.242377:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.242385:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e63033d2f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2e92 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.242385:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9273 00000100:00100000:0.0:1780298643.242404:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556aa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609461504:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.242414:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298643.242418:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604933760:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.242426:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e63033d36 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2e99 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.242465:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604933888:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.242468:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.242504:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417e40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.243003:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604933760:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.243032:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e63033d2f lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e92 expref: 14 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.243051:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e63033d2f lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e92 expref: 14 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.243062:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93600, rc 0) 00000100:00100000:1.0:1780298643.243094:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556aad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609461376:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 728us (842us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.243100:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9272 00000100:00100000:0.0:1780298643.243119:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604933888:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.243198:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e63033d36 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e99 expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.243214:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e63033d36 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e99 expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.243226:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf200, rc 0) 00000100:00100000:0.0:1780298643.243261:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556aa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609461504:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 863us (947us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.243269:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9273 00000100:00100000:3.0:1780298643.243925:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.243963:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609461632 00000100:00100000:0.0:1780298643.243967:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609461632 00000100:00100000:0.0:1780298643.243998:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9274 00000100:00100000:0.0:1780298643.244005:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244811180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:29360:x1866777609461632:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.244013:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.244022:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e63033d3d lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2ea0 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.244038:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(135 177) 00010000:00010000:0.0:1780298643.244043:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(135 177) 00010000:00010000:0.0:1780298643.244052:0:20252:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e63033d3d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1523712->4386815] (req 1523712->1535999) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ea0 expref: 14 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.244066:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e63033d3d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1523712->4386815] (req 1523712->1535999) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ea0 expref: 14 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.244081:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf000, rc 0) 00000100:00100000:0.0:1780298643.244112:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244811180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:29360:x1866777609461632:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 108us (189us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298643.244119:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9274 00000100:00100000:2.0:1780298643.244739:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.244772:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609461760 00000100:00100000:1.0:1780298643.244776:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609461760 00000100:00100000:1.0:1780298643.244812:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6113 00000100:00100000:1.0:1780298643.244820:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2397:x1866777609461760:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298643.245037:0:8362:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 6243739 -> 6243739 00002000:00010000:1.0:1780298643.245041:0:8362:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298643.245044:0:8362:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298643.245046:0:8362:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298643.245047:0:8362:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 480 -> 504 00000100:00100000:1.0:1780298643.245074:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2397:x1866777609461760:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 256us (338us total) trans 4294969661 rc 0/0 00000100:00100000:1.0:1780298643.245080:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6113 00000100:00100000:2.0:1780298643.245833:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.245869:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609461888 00000100:00100000:0.0:1780298643.245876:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609461888 00000100:00100000:0.0:1780298643.245919:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9275 00000100:00100000:0.0:1780298643.245931:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b03100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2397:x1866777609461888:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298643.245934:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:0.0:1780298643.245947:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.245957:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e63033d44 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2ea7 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.245983:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604934016:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.246006:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609462016 00000100:00100000:3.0:1780298643.246010:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609462016 00000100:00100000:0.0:1780298643.246028:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417e40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.246050:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9276 00000100:00100000:3.0:1780298643.246062:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b03800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609462016:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.246069:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.246077:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000fa1945a/0xaf35132e63033d4b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2eae expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.246100:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244812a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604934144:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.246194:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.246626:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604934016:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.246641:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e63033d44 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ea7 expref: 16 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.246652:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e63033d44 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ea7 expref: 16 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.246663:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae600, rc 0) 00000100:00100000:0.0:1780298643.246690:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b03100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2397:x1866777609461888:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 765us (858us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.246696:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9275 00000100:00100000:3.0:1780298643.246845:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244812a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604934144:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.246862:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000fa1945a/0xaf35132e63033d4b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2eae expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.246875:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000fa1945a/0xaf35132e63033d4b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2eae expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.246885:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3000, rc 0) 00000100:00100000:3.0:1780298643.246920:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b03800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609462016:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 863us (994us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.246926:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9276 00000100:00100000:2.0:1780298643.247693:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.247738:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609462144 00000100:00100000:0.0:1780298643.247743:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609462144 00000100:00100000:0.0:1780298643.247772:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9277 00000100:00100000:0.0:1780298643.247780:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b01f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2397:x1866777609462144:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.247788:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298643.247793:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:0.0:1780298643.247797:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e63033d52 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2eb5 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.247824:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604934272:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.247835:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609462272 00000100:00100000:0.0:1780298643.247878:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417e40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.247895:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609462272 00000100:00100000:3.0:1780298643.247939:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9278 00000100:00100000:3.0:1780298643.247953:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5db800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609462272:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.247999:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.248008:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002c458c77/0xaf35132e63033d59 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2ebc expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.248035:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244812a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604934400:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.248077:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.248418:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604934272:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.248437:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e63033d52 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2eb5 expref: 16 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.248452:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e63033d52 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2eb5 expref: 16 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.248463:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae400, rc 0) 00000100:00100000:0.0:1780298643.248495:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b01f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2397:x1866777609462144:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 716us (801us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.248502:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9277 00000100:00100000:3.0:1780298643.248662:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244812a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604934400:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.248675:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002c458c77/0xaf35132e63033d59 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ebc expref: 12 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.248689:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002c458c77/0xaf35132e63033d59 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ebc expref: 12 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.248697:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2a00, rc 0) 00000100:00100000:3.0:1780298643.248723:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5db800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609462272:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 778us (933us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.248728:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9278 00000100:00100000:2.0:1780298643.249502:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.249548:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609462400 00000100:00100000:1.0:1780298643.249554:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609462400 00000100:00100000:1.0:1780298643.249601:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9279 00000100:00100000:2.0:1780298643.249610:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.249614:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2398:x1866777609462400:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.249625:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.249637:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004cdc8249/0xaf35132e63033d60 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2ec3 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.249642:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609462528 00000100:00100000:1.0:1780298643.249684:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604934528:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.249685:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609462528 00000100:00100000:3.0:1780298643.249723:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9280 00000100:00100000:3.0:1780298643.249735:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609462528:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.249746:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298643.249750:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00010000:00010000:3.0:1780298643.249760:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003540747e/0xaf35132e63033d67 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2eca expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.249789:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244812a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604934656:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.249874:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.250315:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604934528:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.250334:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004cdc8249/0xaf35132e63033d60 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ec3 expref: 16 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.250349:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004cdc8249/0xaf35132e63033d60 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ec3 expref: 16 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.250359:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92200, rc 0) 00000100:00100000:1.0:1780298643.250389:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2398:x1866777609462400:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 781us (889us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.250397:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9279 00000100:00100000:3.0:1780298643.250565:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244812a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604934656:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.250583:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003540747e/0xaf35132e63033d67 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2eca expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.250597:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003540747e/0xaf35132e63033d67 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2eca expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.250611:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3c00, rc 0) 00000100:00100000:3.0:1780298643.250645:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609462528:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 913us (1035us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.250656:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9280 00000100:00100000:0.0:1780298643.251380:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.251427:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609462656 00000100:00100000:2.0:1780298643.251433:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609462656 00000100:00100000:2.0:1780298643.251474:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9281 00000100:00100000:3.0:1780298643.251490:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.251490:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2397:x1866777609462656:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.251503:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298643.251518:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609462784 00010000:00010000:2.0:1780298643.251519:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63033d6e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2ed1 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.251524:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609462784 00000100:00100000:0.0:1780298643.251554:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9282 00000100:00100000:0.0:1780298643.251561:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244812a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609462784:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298643.251566:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f747800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604934784:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.251569:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.251578:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e63033d75 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2ed8 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.251605:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604934912:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.251628:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.251632:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417e40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.252317:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f747800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604934784:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.252341:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604934912:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.252344:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63033d6e lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ed1 expref: 16 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.252369:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63033d6e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ed1 expref: 16 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.252377:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e63033d75 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ed8 expref: 12 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.252390:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2600, rc 0) 00010000:00010000:3.0:1780298643.252405:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e63033d75 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ed8 expref: 12 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.252421:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baea00, rc 0) 00000100:00100000:2.0:1780298643.252431:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2397:x1866777609462656:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 946us (1049us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.252439:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9281 00000100:00100000:3.0:1780298643.252458:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244812a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609462784:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 894us (967us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.252467:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9282 00000100:00100000:3.0:1780298643.253335:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.253378:0:7411:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609463040 00000100:00100000:2.0:1780298643.253385:0:7411:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609463040 00000100:00100000:2.0:1780298643.253430:0:7411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61970 00000100:00100000:2.0:1780298643.253441:0:7411:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244810380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+103:29360:x1866777609463040:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:2.0:1780298643.253488:0:7411:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298643.253507:0:7411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92400/0xaf35132e63033ab2 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 70 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c15 expref: 103 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.253521:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298643.253526:0:7411:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298643.253686:0:7411:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3200/0xaf35132e63033d7c lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 69 type: IBT flags: 0x40000000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.253702:0:7411:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafa00/0xaf35132e63033a8f lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 69 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bf9 expref: 133 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.253721:0:7411:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3200/0xaf35132e63033d7c lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 69 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.253744:0:7411:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafa00/0xaf35132e63033a8f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 69 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bf9 expref: 133 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.253758:0:7411:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafa00/0xaf35132e63033a8f lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 69 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bf9 expref: 133 pid: 9976 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.253770:0:7411:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f747800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7411:1866777604935040:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298643.253819:0:7411:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.254396:0:7411:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f747800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7411:1866777604935040:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298643.254410:0:7411:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3200/0xaf35132e63033d7c lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 69 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.254636:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.254681:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609463168 00000100:00100000:0.0:1780298643.254687:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609463168 00010000:00010000:0.0:1780298643.254711:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafa00/0xaf35132e63033a8f lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 69 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bf9 expref: 134 pid: 9976 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.254738:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15068 00000100:00100000:0.0:1780298643.254744:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244810700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+134:2400:x1866777609463168:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298643.254751:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298643.254755:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafa00/0xaf35132e63033a8f lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 70 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bf9 expref: 134 pid: 9976 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.254763:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafa00/0xaf35132e63033a8f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 70 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bf9 expref: 134 pid: 9976 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.254774:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafa00/0xaf35132e63033a8f lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 70 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2bf9 expref: 134 pid: 9976 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.254782:0:6463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298643.254786:0:6463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3200/0xaf35132e63033d7c lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 69 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.254795:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3200/0xaf35132e63033d7c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 69 type: IBT flags: 0x40290000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.254800:0:6463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3200/0xaf35132e63033d7c lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 69 type: IBT flags: 0x40290000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.254814:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298643.254837:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244810700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+133:2400:x1866777609463168:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 94us (201us total) trans 0 rc 0/0 00010000:00010000:2.0:1780298643.254839:0:7411:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3200/0xaf35132e63033d7c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 68 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.254843:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15068 00010000:00010000:2.0:1780298643.254852:0:7411:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3200/0xaf35132e63033d7c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 68 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1780298643.254903:0:7411:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991040 00000004:00010000:2.0:1780298643.254929:0:7411:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000e63c021d reply state 000000001443c3ab transno 4294991041 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3200/0xaf35132e63033d7c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 68 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.254948:0:7411:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3200/0xaf35132e63033d7c lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 68 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.254955:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298643.254959:0:7411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1780298643.254962:0:7411:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1780298643.255026:0:7411:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244810380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+105:29360:x1866777609463040:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1587us (1690us total) trans 4294991041 rc 0/0 00000100:00100000:2.0:1780298643.255033:0:7411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61970 00000020:00080000:2.0:1780298643.255038:0:7411:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1780298643.255665:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.255708:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609463296 00000100:00100000:2.0:1780298643.255714:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609463296 00000100:00100000:2.0:1780298643.255749:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9283 00000100:00100000:2.0:1780298643.255758:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244810e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:29360:x1866777609463296:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.255767:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.255777:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e63033d83 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2edf expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.255795:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63033bb5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [282624->331775] (req 282624->290815) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d18 expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.255804:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e63033d3d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [1523712->4386815] (req 1523712->1535999) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ea0 expref: 14 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.255813:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e63033ce2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [4386816->4571135] (req 4386816->4390911) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e45 expref: 11 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.255828:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000cd0af3fe/0xaf35132e63033c2c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [4571136->38125567] (req 4571136->4579327) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d8f expref: 11 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.255835:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 4 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(135 177) 00010000:00010000:2.0:1780298643.255839:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(139 177) 00010000:00010000:2.0:1780298643.255844:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e63033d83 lrc: 7/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2edf expref: 11 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.255864:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000cd0af3fe/0xaf35132e63033c2c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [4571136->38125567] (req 4571136->4579327) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d8f expref: 11 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.255874:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000cd0af3fe/0xaf35132e63033c2c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [4571136->38125567] (req 4571136->4579327) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d8f expref: 11 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.255885:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f747800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604935168:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298643.255940:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e63033ce2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [4386816->4571135] (req 4386816->4390911) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e45 expref: 11 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.255952:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e63033ce2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [4386816->4571135] (req 4386816->4390911) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e45 expref: 11 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.255961:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bf480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604935296:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298643.256005:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e63033d3d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [1523712->4386815] (req 1523712->1535999) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ea0 expref: 14 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.256016:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e63033d3d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [1523712->4386815] (req 1523712->1535999) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ea0 expref: 14 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.256026:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604935424:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298643.256069:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63033bb5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [282624->331775] (req 282624->290815) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d18 expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.256080:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63033bb5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [282624->331775] (req 282624->290815) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d18 expref: 11 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.256091:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255091500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604935552:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298643.256117:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.256586:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f747800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604935168:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298643.256599:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bf480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604935296:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298643.256777:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.256820:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609463424 00000100:00100000:0.0:1780298643.256833:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609463424 00000100:00100000:2.0:1780298643.256847:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604935424:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298643.256853:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000cd0af3fe/0xaf35132e63033c2c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [4571136->38125567] (req 4571136->4579327) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d8f expref: 12 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.256881:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15069 00000100:00100000:0.0:1780298643.256887:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b01c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609463424:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298643.256894:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298643.256904:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000cd0af3fe/0xaf35132e63033c2c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [4571136->38125567] (req 4571136->4579327) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d8f expref: 12 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.256912:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000cd0af3fe/0xaf35132e63033c2c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [4571136->38125567] (req 4571136->4579327) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d8f expref: 12 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.256924:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000cd0af3fe/0xaf35132e63033c2c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [4571136->38125567] (req 4571136->4579327) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d8f expref: 12 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.256933:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298643.256964:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b01c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609463424:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 76us (187us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298643.256972:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15069 00000100:00100000:2.0:1780298643.257022:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255091500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604935552:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298643.257066:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e63033d83 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2edf expref: 10 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.257081:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2a00, rc 0) 00000100:00100000:1.0:1780298643.257207:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.257214:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244810e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:29360:x1866777609463296:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1456us (1547us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.257225:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9283 00000100:00100000:0.0:1780298643.257246:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609463552 00000100:00100000:0.0:1780298643.257251:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609463552 00002000:00100000:0.0:1780298643.257297:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (282624->4390911) req@ffff9cf36e5d8000 x1866777609463552/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 504/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.257307:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e5d8000 with x1866777609463552 ext(282624->4390911) 00010000:00010000:0.0:1780298643.257316:0:8364:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63033bb5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [282624->331775] (req 282624->290815) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d18 expref: 10 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.257317:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:0.0:1780298643.257327:0:8364:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63033bb5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [282624->331775] (req 282624->290815) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d18 expref: 10 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.257335:0:8364:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e63033ce2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [4386816->4571135] (req 4386816->4390911) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e45 expref: 10 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.257346:0:8364:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e63033ce2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [4386816->4571135] (req 4386816->4390911) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e45 expref: 10 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1780298643.257353:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 2 locks timeout for req ffff9cf36e5d8000 00000100:00100000:3.0:1780298643.257357:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609463680 00000100:00100000:3.0:1780298643.257364:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609463680 00000100:00100000:0.0:1780298643.257380:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 193 00010000:00010000:3.0:1780298643.257388:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e63033d3d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1523712->4386815] (req 1523712->1535999) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ea0 expref: 15 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.257394:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609463552:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:3.0:1780298643.257445:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15070 00000100:00100000:3.0:1780298643.257452:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5db100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2399:x1866777609463680:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298643.257465:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298643.257518:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e63033d3d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [1523712->4386815] (req 1523712->1535999) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ea0 expref: 15 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.257527:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e63033d3d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [1523712->4386815] (req 1523712->1535999) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ea0 expref: 15 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.257541:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e63033d3d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [1523712->4386815] (req 1523712->1535999) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ea0 expref: 15 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.257552:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298643.257633:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5db100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609463680:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 180us (314us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298643.257641:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15070 00000100:00100000:0.0:1780298643.258465:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609463552:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 1075us (1254us total) trans 4294969662 rc 0/0 00000100:00100000:0.0:1780298643.258474:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 193 00002000:00100000:0.0:1780298643.258482:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (282624->4390911) req@ffff9cf36e5d8000 x1866777609463552/t4294969662(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 504/448 e 0 to 0 dl 1780298654 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.258492:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e5d8000 with x1866777609463552 ext(282624->4390911) 00010000:00010000:0.0:1780298643.258502:0:8364:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63033bb5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [282624->331775] (req 282624->290815) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d18 expref: 12 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.258516:0:8364:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63033bb5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [282624->331775] (req 282624->290815) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d18 expref: 12 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.258526:0:8364:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e63033ce2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [4386816->4571135] (req 4386816->4390911) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e45 expref: 12 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.258537:0:8364:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e63033ce2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [4386816->4571135] (req 4386816->4390911) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e45 expref: 12 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1780298643.258545:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 2 locks timeout for req ffff9cf36e5d8000 00000100:00100000:1.0:1780298643.259200:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.259238:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609463936 00000100:00100000:2.0:1780298643.259243:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609463936 00000100:00100000:1.0:1780298643.259249:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:2.0:1780298643.259259:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63033bb5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [282624->331775] (req 282624->290815) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d18 expref: 12 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.259282:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609463808 00000100:00100000:2.0:1780298643.259295:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609463808 00000100:00100000:0.0:1780298643.259302:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15071 00010000:00010000:2.0:1780298643.259307:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e63033ce2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [4386816->4571135] (req 4386816->4390911) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e45 expref: 13 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.259315:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d9500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2398:x1866777609463936:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298643.259325:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1780298643.259329:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15072 00002000:00010000:0.0:1780298643.259335:0:6463:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 504 -> 528 00000100:00100000:2.0:1780298643.259343:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2398:x1866777609463808:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298643.259346:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63033bb5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [282624->331775] (req 282624->290815) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d18 expref: 13 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.259356:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298643.259362:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63033bb5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [282624->331775] (req 282624->290815) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d18 expref: 13 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.259376:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e63033ce2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [4386816->4571135] (req 4386816->4390911) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e45 expref: 13 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.259381:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63033bb5 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [282624->331775] (req 282624->290815) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2d18 expref: 13 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.259397:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298643.259402:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e63033ce2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [4386816->4571135] (req 4386816->4390911) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e45 expref: 12 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.259418:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e63033ce2 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [4386816->4571135] (req 4386816->4390911) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2e45 expref: 12 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.259427:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(139 177) 00000100:00100000:0.0:1780298643.259428:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d9500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609463936:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 115us (226us total) trans 0 rc 0/0 00010000:00010000:2.0:1780298643.259433:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(139 177) 00010000:00010000:2.0:1780298643.259445:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e63033d83 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2edf expref: 11 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.259445:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15071 00010000:00010000:2.0:1780298643.259459:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e63033d83 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2edf expref: 10 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.259485:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e63033d83 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2edf expref: 10 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.259494:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255091500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777604935680:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298643.259531:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.260322:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255091500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777604935680:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298643.260336:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298643.260367:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609463808:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1025us (1112us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298643.260377:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.260382:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15072 00000100:00100000:0.0:1780298643.260432:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609464064 00000100:00100000:0.0:1780298643.260438:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609464064 00000100:00100000:0.0:1780298643.260473:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9284 00000100:00100000:0.0:1780298643.260481:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244813800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:29360:x1866777609464064:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.260537:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.260547:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63033d8a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2ee6 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.260562:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e63033c5d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [1036288->1761279] (req 1036288->1044479) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2dc0 expref: 13 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.260569:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63033c9c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [2125824->2551807] (req 2125824->2134015) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2dff expref: 13 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.260574:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e63033b92 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [2551808->4014079] (req 2551808->2555903) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2cf5 expref: 13 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.260581:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e63033adc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [4014080->37568511] (req 4014080->4022271) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c3f expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.260588:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 4 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(99 130) 00010000:00010000:0.0:1780298643.260592:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(103 130) 00010000:00010000:0.0:1780298643.260596:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63033d8a lrc: 7/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ee6 expref: 13 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.260618:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e63033adc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [4014080->37568511] (req 4014080->4022271) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c3f expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.260629:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e63033adc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [4014080->37568511] (req 4014080->4022271) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c3f expref: 10 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.260639:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604935808:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298643.260678:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e63033b92 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [2551808->4014079] (req 2551808->2555903) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2cf5 expref: 13 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.260685:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e63033b92 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [2551808->4014079] (req 2551808->2555903) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2cf5 expref: 13 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.260694:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604935936:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298643.260716:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63033c9c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [2125824->2551807] (req 2125824->2134015) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2dff expref: 13 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.260726:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63033c9c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [2125824->2551807] (req 2125824->2134015) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2dff expref: 13 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.260734:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604936064:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298643.260758:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e63033c5d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [1036288->1761279] (req 1036288->1044479) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2dc0 expref: 13 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.260765:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e63033c5d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [1036288->1761279] (req 1036288->1044479) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2dc0 expref: 13 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.260772:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604936192:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298643.260784:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.261476:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604935808:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298643.261485:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604935936:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298643.261743:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.261767:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604936064:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298643.261788:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609464192 00000100:00100000:0.0:1780298643.261792:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604936192:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298643.261793:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609464192 00002000:00100000:1.0:1780298643.261818:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x12:0x0] (4014080->4022271) req@ffff9cf244811880 x1866777609464192/t0(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 8680/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00010000:00010000:0.0:1780298643.261819:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63033d8a lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ee6 expref: 13 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00100000:1.0:1780298643.261842:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244811880 with x1866777609464192 ext(4014080->4022271) 00010000:00010000:0.0:1780298643.261844:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae400, rc 0) 00010000:00010000:1.0:1780298643.261850:0:19928:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e63033adc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [4014080->37568511] (req 4014080->4022271) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c3f expref: 11 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.261866:0:19928:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e63033adc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [4014080->37568511] (req 4014080->4022271) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c3f expref: 11 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1780298643.261874:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9cf244811880 00000100:00100000:0.0:1780298643.261882:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244813800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:29360:x1866777609464064:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1400us (1520us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298643.261884:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.261899:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9284 00000100:00100000:1.0:1780298643.261911:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 194 00000100:00100000:1.0:1780298643.261923:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244811880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609464192:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:3.0:1780298643.261996:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.262037:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609464576 00000100:00100000:0.0:1780298643.262043:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609464576 00010000:00010000:0.0:1780298643.262062:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e63033c5d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [1036288->1761279] (req 1036288->1044479) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2dc0 expref: 13 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.262084:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.262086:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609464448 00000100:00100000:0.0:1780298643.262089:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609464448 00010000:00010000:0.0:1780298643.262104:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63033c9c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [2125824->2551807] (req 2125824->2134015) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2dff expref: 14 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.262108:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609464320 00000100:00100000:2.0:1780298643.262115:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609464320 00010000:00010000:2.0:1780298643.262159:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e63033b92 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [2551808->4014079] (req 2551808->2555903) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2cf5 expref: 15 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.262172:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15073 00000100:00100000:0.0:1780298643.262183:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15074 00000100:00100000:3.0:1780298643.262189:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244810700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2397:x1866777609464576:12345-192.168.203.25@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1780298643.262198:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242861180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2400:x1866777609464448:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298643.262207:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1780298643.262209:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15075 00010000:00010000:0.0:1780298643.262209:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1780298643.262221:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242863480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2399:x1866777609464320:12345-192.168.203.25@tcp:103:ldlm_bl.0 00002000:00010000:0.0:1780298643.262221:0:7400:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 144 -> 160 00010000:00010000:0.0:1780298643.262232:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63033c9c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 9 type: EXT [2125824->2551807] (req 2125824->2134015) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2dff expref: 15 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.262235:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298643.262238:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e63033c5d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 9 type: EXT [1036288->1761279] (req 1036288->1044479) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2dc0 expref: 15 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.262250:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63033c9c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 11 type: EXT [2125824->2551807] (req 2125824->2134015) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2dff expref: 15 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.262251:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e63033b92 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 11 type: EXT [2551808->4014079] (req 2551808->2555903) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2cf5 expref: 15 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.262310:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63033c9c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 11 type: EXT [2125824->2551807] (req 2125824->2134015) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2dff expref: 15 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.262319:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e63033c5d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 11 type: EXT [1036288->1761279] (req 1036288->1044479) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2dc0 expref: 15 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.262353:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e63033b92 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 8 type: EXT [2551808->4014079] (req 2551808->2555903) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2cf5 expref: 14 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.262364:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298643.262375:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e63033b92 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [2551808->4014079] (req 2551808->2555903) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2cf5 expref: 14 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.262388:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e63033c5d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [1036288->1761279] (req 1036288->1044479) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2dc0 expref: 13 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.262390:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298643.262400:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242861180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609464448:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 203us (401us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.262401:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244811880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609464192:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 478us (724us total) trans 4294969221 rc 0/0 00000100:00100000:2.0:1780298643.262420:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242863480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2399:x1866777609464320:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 194us (331us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.262433:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 194 00000100:00100000:0.0:1780298643.262434:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15074 00010000:00010000:3.0:1780298643.262439:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00002000:00100000:1.0:1780298643.262444:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x12:0x0] (4014080->4022271) req@ffff9cf244811880 x1866777609464192/t4294969221(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 8680/448 e 0 to 0 dl 1780298654 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00000100:00100000:2.0:1780298643.262446:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15075 00002000:00100000:1.0:1780298643.262454:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244811880 with x1866777609464192 ext(4014080->4022271) 00010000:00010000:1.0:1780298643.262460:0:19928:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e63033adc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [4014080->37568511] (req 4014080->4022271) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c3f expref: 13 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.262473:0:19928:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e63033adc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [4014080->37568511] (req 4014080->4022271) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c3f expref: 13 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1780298643.262481:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9cf244811880 00000100:00100000:3.0:1780298643.262483:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244810700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609464576:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 298us (603us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298643.262497:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15073 00000100:00100000:2.0:1780298643.263270:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.263316:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609464704 00000100:00100000:1.0:1780298643.263321:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609464704 00010000:00010000:1.0:1780298643.263335:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e63033adc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [4014080->37568511] (req 4014080->4022271) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c3f expref: 13 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.263362:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15076 00000100:00100000:1.0:1780298643.263367:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b5d450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609464704:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298643.263374:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298643.263382:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e63033adc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [4014080->37568511] (req 4014080->4022271) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c3f expref: 13 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.263389:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e63033adc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [4014080->37568511] (req 4014080->4022271) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c3f expref: 13 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.263399:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e63033adc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [4014080->37568511] (req 4014080->4022271) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2c3f expref: 13 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.263407:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(103 130) 00010000:00010000:1.0:1780298643.263409:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(103 130) 00010000:00010000:1.0:1780298643.263413:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63033d8a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ee6 expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.263419:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63033d8a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ee6 expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.263435:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63033d8a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ee6 expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.263441:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5db480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:20253:1866777604936320:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298643.263479:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.264214:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5db480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:20253:1866777604936320:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298643.264229:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298643.264258:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b5d450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2399:x1866777609464704:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 891us (991us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.264264:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15076 00000100:00100000:3.0:1780298643.264316:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.264371:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609464832 00000100:00100000:0.0:1780298643.264376:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609464832 00002000:00010000:0.0:1780298643.264393:0:8363:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:0.0:1780298643.264396:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242863b80 with x1866777609464832 ext(4194304->18446744073709551615) 00002000:00010000:0.0:1780298643.264402:0:8363:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf242863b80. 00000100:00100000:0.0:1780298643.264426:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6114 00000100:00100000:0.0:1780298643.264433:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242863b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609464832:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:3.0:1780298643.264442:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.264479:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609464960 00000100:00100000:1.0:1780298643.264485:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609464960 00002000:00010000:1.0:1780298643.264500:0:19951:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (2732463->18446744073709551615) 00002000:00100000:1.0:1780298643.264503:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242862d80 with x1866777609464960 ext(2732463->18446744073709551615) 00002000:00010000:1.0:1780298643.264507:0:19951:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf242862d80. 00000100:00100000:1.0:1780298643.264530:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6115 00000100:00100000:1.0:1780298643.264537:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242862d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609464960:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:1.0:1780298643.266012:0:19951:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4194304 -> 2732463 00002000:00010000:1.0:1780298643.266016:0:19951:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298643.266018:0:19951:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298643.266020:0:19951:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298643.266021:0:19951:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 160 -> 96 00000100:00100000:1.0:1780298643.266050:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242862d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609464960:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 1515us (1609us total) trans 4294969222 rc 0/0 00000100:00100000:1.0:1780298643.266056:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6115 00002000:00010000:1.0:1780298643.266059:0:19951:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (2732463->18446744073709551615) 00002000:00100000:1.0:1780298643.266062:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242862d80 with x1866777609464960 ext(2732463->18446744073709551615) 00002000:00010000:1.0:1780298643.266066:0:19951:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf242862d80. 00002000:00010000:0.0:1780298643.266246:0:8363:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 6243739 -> 4194304 00002000:00010000:0.0:1780298643.266250:0:8363:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:0.0:1780298643.266251:0:8363:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:0.0:1780298643.266253:0:8363:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:0.0:1780298643.266254:0:8363:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 528 -> 432 00000100:00100000:0.0:1780298643.266280:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242863b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609464832:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 1848us (1966us total) trans 4294969663 rc 0/0 00000100:00100000:0.0:1780298643.266296:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6114 00002000:00010000:0.0:1780298643.266299:0:8363:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:0.0:1780298643.266301:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242863b80 with x1866777609464832 ext(4194304->18446744073709551615) 00002000:00010000:0.0:1780298643.266305:0:8363:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf242863b80. 00000100:00100000:3.0:1780298643.266988:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.267025:0:6476:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609465088 00000100:00100000:1.0:1780298643.267030:0:6476:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609465088 00000100:00100000:1.0:1780298643.267058:0:6476:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61971 00000100:00100000:1.0:1780298643.267063:0:6476:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242862300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+105:29360:x1866777609465088:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.267076:0:6476:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.267088:0:6476:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93c00/0xaf35132e63033d91 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 69 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2eed expref: -99 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.267172:0:6476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92e00/0xaf35132e63033d98 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 70 type: IBT flags: 0x40000000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.267183:0:6476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92e00/0xaf35132e63033d98 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 70 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.267190:0:6476:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92e00/0xaf35132e63033d98 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 70 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1780298643.267210:0:6476:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 6926767 00000004:00200000:1.0:1780298643.267213:0:6476:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1780298643.267227:0:6476:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92e00/0xaf35132e63033d98 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 70 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.267244:0:6476:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93c00/0xaf35132e63033d91 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 70 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2eed expref: 107 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.267253:0:6476:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92e00/0xaf35132e63033d98 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 69 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2eed expref: 106 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.267258:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298643.267260:0:6476:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1780298643.267262:0:6476:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92e00, rc 0) 00000100:00100000:1.0:1780298643.267296:0:6476:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242862300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+106:29360:x1866777609465088:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 234us (311us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.267301:0:6476:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61971 00000100:00100000:2.0:1780298643.268868:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.268907:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609465216 00000100:00100000:1.0:1780298643.268912:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609465216 00002000:00100000:1.0:1780298643.268936:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (3334144->3338239) req@ffff9cf255091500 x1866777609465216/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.268945:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255091500 with x1866777609465216 ext(3334144->3338239) 00002000:00010000:1.0:1780298643.268949:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255091500 00000100:00100000:1.0:1780298643.268971:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6116 00000100:00100000:1.0:1780298643.268979:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255091500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609465216:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298643.269099:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255091500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609465216:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 122us (231us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298643.269106:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6116 00002000:00100000:1.0:1780298643.269114:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (3334144->3338239) req@ffff9cf255091500 x1866777609465216/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.269155:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255091500 with x1866777609465216 ext(3334144->3338239) 00002000:00010000:1.0:1780298643.269160:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255091500 00000100:00100000:3.0:1780298643.270619:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.270665:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609465344 00000100:00100000:0.0:1780298643.270670:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609465344 00002000:00100000:0.0:1780298643.270689:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (2174976->2179071) req@ffff9cf242861c00 x1866777609465344/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.270696:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242861c00 with x1866777609465344 ext(2174976->2179071) 00002000:00010000:0.0:1780298643.270699:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf242861c00 00000100:00100000:0.0:1780298643.270715:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6117 00000100:00100000:0.0:1780298643.270720:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242861c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609465344:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298643.270789:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242861c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609465344:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 68us (175us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298643.270794:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6117 00002000:00100000:0.0:1780298643.270797:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (2174976->2179071) req@ffff9cf242861c00 x1866777609465344/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.270803:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242861c00 with x1866777609465344 ext(2174976->2179071) 00002000:00010000:0.0:1780298643.270806:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf242861c00 00000100:00100000:3.0:1780298643.271721:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.271759:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609465472 00000100:00100000:1.0:1780298643.271763:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609465472 00002000:00100000:1.0:1780298643.271781:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (2179072->2183167) req@ffff9cf242863800 x1866777609465472/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.271788:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242863800 with x1866777609465472 ext(2179072->2183167) 00002000:00010000:1.0:1780298643.271790:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf242863800 00000100:00100000:1.0:1780298643.271806:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6118 00000100:00100000:1.0:1780298643.271810:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242863800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609465472:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298643.271878:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242863800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609465472:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 66us (157us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298643.271885:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6118 00002000:00100000:1.0:1780298643.271893:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (2179072->2183167) req@ffff9cf242863800 x1866777609465472/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.271901:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242863800 with x1866777609465472 ext(2179072->2183167) 00002000:00010000:1.0:1780298643.271905:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf242863800 00000100:00100000:2.0:1780298643.272796:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.272843:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609465600 00000100:00100000:0.0:1780298643.272847:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609465600 00002000:00100000:0.0:1780298643.272864:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (2174976->3338239) req@ffff9cf255092680 x1866777609465600/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 504/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.272869:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255092680 with x1866777609465600 ext(2174976->3338239) 00002000:00010000:0.0:1780298643.272872:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255092680 00000100:00100000:0.0:1780298643.272889:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6119 00000100:00100000:0.0:1780298643.272893:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255092680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609465600:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:0.0:1780298643.273858:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255092680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2398:x1866777609465600:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 963us (1064us total) trans 4294969664 rc 0/0 00000100:00100000:0.0:1780298643.273865:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6119 00002000:00100000:0.0:1780298643.273870:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (2174976->3338239) req@ffff9cf255092680 x1866777609465600/t4294969664(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 504/448 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.273881:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255092680 with x1866777609465600 ext(2174976->3338239) 00002000:00010000:0.0:1780298643.273883:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255092680 00000100:00100000:3.0:1780298643.274559:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.274616:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609465728 00000100:00100000:1.0:1780298643.274622:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609465728 00000100:00100000:1.0:1780298643.274653:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9285 00000100:00100000:1.0:1780298643.274662:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242861f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2397:x1866777609465728:12345-192.168.203.25@tcp:16:fsx.0 00000001:00080000:2.0:1780298643.275424:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969664 is committed 00000001:00080000:2.0:1780298643.275436:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969661 is committed 00000100:00100000:1.0:1780298643.275511:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242861f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609465728:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 850us (953us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.275516:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9285 00000100:00100000:2.0:1780298643.276259:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.276308:0:9976:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609465856 00000100:00100000:0.0:1780298643.276312:0:9976:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609465856 00000100:00100000:0.0:1780298643.276340:0:9976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61972 00000100:00100000:0.0:1780298643.276349:0:9976:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+133:29360:x1866777609465856:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.276361:0:9976:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.276380:0:9976:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf400/0xaf35132e63033d9f lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 70 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2ef4 expref: -99 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.276451:0:9976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baee00/0xaf35132e63033da6 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 71 type: IBT flags: 0x40000000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.276461:0:9976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baee00/0xaf35132e63033da6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 71 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.276472:0:9976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baee00/0xaf35132e63033da6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 71 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1780298643.276496:0:9976:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 6926767 00000004:00200000:0.0:1780298643.276501:0:9976:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1780298643.276520:0:9976:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baee00/0xaf35132e63033da6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 71 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.276540:0:9976:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf400/0xaf35132e63033d9f lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 71 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ef4 expref: 135 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.276553:0:9976:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baee00/0xaf35132e63033da6 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 70 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ef4 expref: 134 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.276559:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298643.276561:0:9976:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1780298643.276563:0:9976:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baee00, rc 0) 00000100:00100000:0.0:1780298643.276588:0:9976:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+134:29360:x1866777609465856:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 242us (332us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298643.276592:0:9976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61972 00000100:00100000:3.0:1780298643.277448:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.277491:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609465984 00000100:00100000:0.0:1780298643.277497:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609465984 00000100:00100000:0.0:1780298643.277529:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9286 00000100:00100000:3.0:1780298643.277538:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.277545:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242861880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2398:x1866777609465984:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.277553:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.277564:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63033dad lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2efb expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.277570:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609466112 00000100:00100000:1.0:1780298643.277576:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609466112 00000100:00100000:0.0:1780298643.277606:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604936448:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.277611:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9287 00000100:00100000:1.0:1780298643.277617:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242860000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609466112:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.277624:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.277636:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e90122d2/0xaf35132e63033db4 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2f02 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.277674:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5db100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604936576:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.277675:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417540 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.277711:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439840 going to sleep for 11 seconds 00002000:00010000:0.0:1780298643.278461:0:20252:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks: 432 -> 456 00000100:00100000:1.0:1780298643.278473:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5db100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604936576:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.278473:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604936448:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.278493:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e90122d2/0xaf35132e63033db4 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f02 expref: 13 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.278493:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63033dad lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2efb expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.278511:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e90122d2/0xaf35132e63033db4 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f02 expref: 13 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.278511:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63033dad lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2efb expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.278521:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93c00, rc 0) 00010000:00010000:0.0:1780298643.278534:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bafa00, rc 0) 00000100:00100000:1.0:1780298643.278557:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242860000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609466112:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 936us (1021us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.278561:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242861880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2398:x1866777609465984:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1022us (1118us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.278567:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9287 00000100:00100000:0.0:1780298643.278573:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9286 00000100:00100000:2.0:1780298643.279632:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.279666:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609466240 00000100:00100000:2.0:1780298643.279672:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.279673:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609466240 00000100:00100000:0.0:1780298643.279711:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609466368 00000100:00100000:0.0:1780298643.279715:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609466368 00000100:00100000:3.0:1780298643.279732:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9288 00000100:00100000:1.0:1780298643.279749:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9289 00000100:00100000:3.0:1780298643.279750:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2398:x1866777609466240:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298643.279761:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e583850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609466368:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.279767:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.279769:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.279788:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e63033dbb lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2f10 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.279791:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000aa41763f/0xaf35132e63033dc2 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2f09 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.279855:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5db100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604936704:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.279874:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242861180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604936832:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.279908:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439840 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.279926:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.280735:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5db100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604936704:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.280753:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e63033dbb lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f10 expref: 13 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.280758:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242861180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604936832:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.280770:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e63033dbb lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f10 expref: 13 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.280779:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92a00, rc 0) 00010000:00010000:2.0:1780298643.280787:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000aa41763f/0xaf35132e63033dc2 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f09 expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.280803:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000aa41763f/0xaf35132e63033dc2 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f09 expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.280813:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e583850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609466368:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1056us (1143us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.280825:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9289 00010000:00010000:2.0:1780298643.280834:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3600, rc 0) 00000100:00100000:2.0:1780298643.280865:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2398:x1866777609466240:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1124us (1234us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.280872:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9288 00000100:00100000:0.0:1780298643.282248:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.282279:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609466624 00000100:00100000:1.0:1780298643.282293:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609466624 00000100:00100000:1.0:1780298643.282331:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61973 00000100:00100000:1.0:1780298643.282341:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+106:29360:x1866777609466624:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:1.0:1780298643.282382:0:6477:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298643.282400:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92e00/0xaf35132e63033d98 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 71 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2eed expref: 106 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.282410:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298643.282413:0:6477:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298643.282557:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93400/0xaf35132e63033dc9 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 70 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.282569:0:6477:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baee00/0xaf35132e63033da6 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 70 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ef4 expref: 133 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.282585:0:6477:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93400/0xaf35132e63033dc9 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 70 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.282605:0:6477:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baee00/0xaf35132e63033da6 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 70 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ef4 expref: 133 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.282618:0:6477:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baee00/0xaf35132e63033da6 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 70 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ef4 expref: 133 pid: 9976 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.282626:0:6477:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5db100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6477:1866777604936960:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298643.282663:0:6477:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393d80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.283267:0:6477:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5db100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6477:1866777604936960:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298643.283279:0:6477:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93400/0xaf35132e63033dc9 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 70 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.283491:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.283525:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609466752 00000100:00100000:2.0:1780298643.283529:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609466752 00010000:00010000:2.0:1780298643.283545:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baee00/0xaf35132e63033da6 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 70 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ef4 expref: 134 pid: 9976 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.283570:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15077 00000100:00100000:2.0:1780298643.283577:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+134:2400:x1866777609466752:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298643.283585:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298643.283588:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baee00/0xaf35132e63033da6 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 71 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ef4 expref: 134 pid: 9976 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.283596:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baee00/0xaf35132e63033da6 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 71 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ef4 expref: 134 pid: 9976 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.283606:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baee00/0xaf35132e63033da6 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 71 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ef4 expref: 134 pid: 9976 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.283615:0:6463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298643.283618:0:6463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93400/0xaf35132e63033dc9 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 70 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.283630:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93400/0xaf35132e63033dc9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 70 type: IBT flags: 0x40290000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.283637:0:6463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93400/0xaf35132e63033dc9 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 70 type: IBT flags: 0x40290000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.283658:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298643.283667:0:6477:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93400/0xaf35132e63033dc9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 69 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.283674:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93400/0xaf35132e63033dc9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 69 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.283686:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f3480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+133:2400:x1866777609466752:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 110us (196us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.283696:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15077 00000001:00080000:1.0:1780298643.283713:0:6477:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991042 00000004:00010000:1.0:1780298643.283736:0:6477:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000459d4a15 reply state 00000000ce435b99 transno 4294991043 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93400/0xaf35132e63033dc9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 69 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.283744:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93400/0xaf35132e63033dc9 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 69 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.283749:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298643.283752:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1780298643.283754:0:6477:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1780298643.283802:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+108:29360:x1866777609466624:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1465us (1554us total) trans 4294991043 rc 0/0 00000100:00100000:1.0:1780298643.283807:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61973 00000020:00080000:1.0:1780298643.283811:0:6477:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1780298643.284636:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.284675:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609466880 00000100:00100000:2.0:1780298643.284679:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609466880 00002000:00010000:2.0:1780298643.284697:0:8363:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4983467->18446744073709551615) 00002000:00100000:2.0:1780298643.284701:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2425f1500 with x1866777609466880 ext(4983467->18446744073709551615) 00002000:00010000:2.0:1780298643.284709:0:8363:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2425f1500. 00000100:00100000:2.0:1780298643.284732:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6120 00000100:00100000:2.0:1780298643.284740:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609466880:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:3.0:1780298643.284806:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.284838:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609467008 00000100:00100000:1.0:1780298643.284842:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609467008 00002000:00010000:1.0:1780298643.284855:0:8362:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4194304->18446744073709551615) 00002000:00100000:1.0:1780298643.284858:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242863480 with x1866777609467008 ext(4194304->18446744073709551615) 00002000:00010000:1.0:1780298643.284863:0:8362:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf242863480. 00000100:00100000:1.0:1780298643.284888:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6121 00000100:00100000:1.0:1780298643.284899:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242863480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609467008:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:2.0:1780298643.284919:0:8363:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4983467 00002000:00010000:2.0:1780298643.284924:0:8363:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:2.0:1780298643.284927:0:8363:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:2.0:1780298643.284929:0:8363:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:2.0:1780298643.284932:0:8363:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 456 -> 456 00000100:00100000:2.0:1780298643.284967:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609466880:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 227us (329us total) trans 4294969665 rc 0/0 00000100:00100000:2.0:1780298643.284978:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6120 00002000:00010000:2.0:1780298643.284982:0:8363:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4983467->18446744073709551615) 00002000:00100000:2.0:1780298643.284985:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2425f1500 with x1866777609466880 ext(4983467->18446744073709551615) 00002000:00010000:2.0:1780298643.284989:0:8363:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2425f1500. 00002000:00010000:1.0:1780298643.284998:0:8362:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 2732463 -> 4194304 00002000:00010000:1.0:1780298643.285001:0:8362:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298643.285004:0:8362:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298643.285007:0:8362:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298643.285009:0:8362:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 96 -> 96 00000100:00100000:1.0:1780298643.285039:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242863480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609467008:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 145us (238us total) trans 4294969223 rc 0/0 00000100:00100000:1.0:1780298643.285044:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6121 00002000:00010000:1.0:1780298643.285046:0:8362:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4194304->18446744073709551615) 00002000:00100000:1.0:1780298643.285048:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242863480 with x1866777609467008 ext(4194304->18446744073709551615) 00002000:00010000:1.0:1780298643.285052:0:8362:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf242863480. 00000100:00100000:3.0:1780298643.286420:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.286451:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609467136 00000100:00100000:0.0:1780298643.286455:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609467136 00002000:00100000:0.0:1780298643.286480:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (4980736->4984831) req@ffff9cf242861f80 x1866777609467136/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.286491:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242861f80 with x1866777609467136 ext(4980736->4984831) 00002000:00010000:0.0:1780298643.286495:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf242861f80 00000100:00100000:0.0:1780298643.286516:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6122 00000100:00100000:0.0:1780298643.286523:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242861f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609467136:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298643.286628:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242861f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609467136:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 107us (210us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298643.286632:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6122 00002000:00100000:0.0:1780298643.286637:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (4980736->4984831) req@ffff9cf242861f80 x1866777609467136/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.286648:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242861f80 with x1866777609467136 ext(4980736->4984831) 00002000:00010000:0.0:1780298643.286650:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf242861f80 00000100:00100000:0.0:1780298643.287878:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.287921:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609467264 00000100:00100000:1.0:1780298643.287926:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609467264 00002000:00100000:1.0:1780298643.287948:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (4980736->4983466) req@ffff9cf2425f0a80 x1866777609467264/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 3224/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.287955:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2425f0a80 with x1866777609467264 ext(4980736->4983466) 00002000:00010000:1.0:1780298643.287958:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2425f0a80 00000100:00100000:1.0:1780298643.287987:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6123 00000100:00100000:1.0:1780298643.287997:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609467264:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:1.0:1780298643.288398:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609467264:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 404us (521us total) trans 4294969666 rc 0/0 00000100:00100000:1.0:1780298643.288404:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6123 00002000:00100000:1.0:1780298643.288409:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (4980736->4983466) req@ffff9cf2425f0a80 x1866777609467264/t4294969666(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 3224/448 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.288415:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2425f0a80 with x1866777609467264 ext(4980736->4983466) 00002000:00010000:1.0:1780298643.288418:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2425f0a80 00000100:00100000:3.0:1780298643.289038:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.289083:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609467392 00000100:00100000:2.0:1780298643.289088:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609467392 00000100:00100000:2.0:1780298643.289115:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9290 00000100:00100000:2.0:1780298643.289122:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242860000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609467392:12345-192.168.203.25@tcp:16:fsx.0 00000001:00080000:1.0:1780298643.289816:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969666 is committed 00000100:00100000:2.0:1780298643.289881:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242860000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609467392:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 759us (845us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.289886:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9290 00000100:00100000:3.0:1780298643.290650:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.290689:0:7411:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609467520 00000100:00100000:2.0:1780298643.290695:0:7411:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609467520 00000100:00100000:2.0:1780298643.290739:0:7411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61974 00000100:00100000:2.0:1780298643.290745:0:7411:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242861880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+133:29360:x1866777609467520:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.290755:0:7411:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.290767:0:7411:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2800/0xaf35132e63033dd0 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 70 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2f17 expref: -99 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.290828:0:7411:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3000/0xaf35132e63033dd7 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 71 type: IBT flags: 0x40000000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.290837:0:7411:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3000/0xaf35132e63033dd7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 71 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.290843:0:7411:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3000/0xaf35132e63033dd7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 71 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298643.290859:0:7411:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 9177771 00000004:00200000:2.0:1780298643.290862:0:7411:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298643.290875:0:7411:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3000/0xaf35132e63033dd7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 71 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.290893:0:7411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2800/0xaf35132e63033dd0 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 71 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f17 expref: 135 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.290905:0:7411:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3000/0xaf35132e63033dd7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 70 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f17 expref: 134 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.290911:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298643.290913:0:7411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298643.290915:0:7411:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3000, rc 0) 00000100:00100000:2.0:1780298643.290940:0:7411:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242861880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+134:29360:x1866777609467520:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 196us (293us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.290944:0:7411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61974 00000100:00100000:0.0:1780298643.291804:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.291871:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609467648 00000100:00100000:0.0:1780298643.291902:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.291910:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609467648 00000100:00100000:3.0:1780298643.291959:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609467776 00000100:00100000:3.0:1780298643.291962:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609467776 00000100:00100000:1.0:1780298643.291970:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9291 00000100:00100000:1.0:1780298643.291985:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2397:x1866777609467648:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298643.291988:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9292 00010000:00010000:1.0:1780298643.291999:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298643.292003:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609467776:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.292012:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.292015:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002f8ed852/0xaf35132e63033dde lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2f1e expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.292021:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000010b6573d/0xaf35132e63033de5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2f25 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.292065:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f745180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604937088:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.292067:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5db100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604937216:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.292160:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.292178:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439840 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.292994:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f745180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604937088:192.168.203.25@tcp:106: 00002000:00010000:1.0:1780298643.292997:0:8358:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks: 456 -> 464 00000100:00100000:1.0:1780298643.293007:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5db100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604937216:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.293013:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000010b6573d/0xaf35132e63033de5 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f25 expref: 13 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.293022:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002f8ed852/0xaf35132e63033dde lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f1e expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.293033:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000010b6573d/0xaf35132e63033de5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f25 expref: 13 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.293037:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002f8ed852/0xaf35132e63033dde lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f1e expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.293047:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3a00, rc 0) 00010000:00010000:1.0:1780298643.293057:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92c00, rc 0) 00000100:00100000:2.0:1780298643.293075:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609467776:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1078us (1174us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.293078:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2397:x1866777609467648:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1098us (1274us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.293089:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9292 00000100:00100000:1.0:1780298643.293098:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9291 00000100:00100000:2.0:1780298643.294039:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.294112:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609467904 00000100:00100000:3.0:1780298643.294117:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609467904 00000100:00100000:0.0:1780298643.294189:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.294228:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609468032 00000100:00100000:3.0:1780298643.294232:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609468032 00000100:00100000:1.0:1780298643.294238:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9293 00000100:00100000:1.0:1780298643.294247:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2398:x1866777609467904:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.294255:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.294262:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e90122d2/0xaf35132e63033dec lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2f2c expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.294271:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9294 00000100:00100000:3.0:1780298643.294280:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609468032:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298643.294295:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5db100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604937344:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.294306:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.294322:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003540747e/0xaf35132e63033df3 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2f33 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.294335:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439840 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.294369:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242861180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604937472:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.294426:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.295015:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5db100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604937344:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.295038:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e90122d2/0xaf35132e63033dec lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f2c expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.295052:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e90122d2/0xaf35132e63033dec lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f2c expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.295063:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93c00, rc 0) 00000100:00100000:1.0:1780298643.295100:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2398:x1866777609467904:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 853us (1062us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.295107:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9293 00000100:00100000:2.0:1780298643.295278:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242861180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604937472:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.295306:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003540747e/0xaf35132e63033df3 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f33 expref: 13 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.295319:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003540747e/0xaf35132e63033df3 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f33 expref: 13 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.295326:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3c00, rc 0) 00000100:00100000:2.0:1780298643.295348:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609468032:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1071us (1163us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.295353:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9294 00000100:00100000:0.0:1780298643.296152:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.296181:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609468160 00000100:00100000:1.0:1780298643.296186:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609468160 00000100:00100000:1.0:1780298643.296219:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9295 00000100:00100000:1.0:1780298643.296227:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:29360:x1866777609468160:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.296237:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.296244:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000670f33f0/0xaf35132e63033dfa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2f3a expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.296257:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63033d8a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ee6 expref: 11 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.296264:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(103 130) 00010000:00010000:1.0:1780298643.296267:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(104 130) 00010000:00010000:1.0:1780298643.296270:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000670f33f0/0xaf35132e63033dfa lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2113536->2121727] (req 2113536->2121727) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f3a expref: 13 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.296296:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63033d8a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ee6 expref: 11 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.296308:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63033d8a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ee6 expref: 11 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.296316:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5db100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604937600:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298643.296347:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.296909:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5db100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604937600:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298643.296919:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000670f33f0/0xaf35132e63033dfa lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2113536->2121727] (req 2113536->2121727) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f3a expref: 13 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.296927:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92e00, rc 0) 00000100:00100000:1.0:1780298643.296947:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:29360:x1866777609468160:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 723us (800us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.296952:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9295 00000100:00100000:2.0:1780298643.297094:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.297158:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609468288 00000100:00100000:0.0:1780298643.297162:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609468288 00010000:00010000:0.0:1780298643.297177:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63033d8a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ee6 expref: 12 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.297202:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15078 00000100:00100000:0.0:1780298643.297209:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f747800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609468288:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298643.297217:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298643.297226:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63033d8a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ee6 expref: 12 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.297234:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63033d8a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ee6 expref: 12 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.297246:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63033d8a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ee6 expref: 12 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.297254:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(104 130) 00010000:00010000:0.0:1780298643.297257:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(104 130) 00010000:00010000:0.0:1780298643.297262:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000670f33f0/0xaf35132e63033dfa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2113536->35667967] (req 2113536->2121727) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f3a expref: 12 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.297268:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000670f33f0/0xaf35132e63033dfa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2113536->35667967] (req 2113536->2121727) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f3a expref: 12 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.297301:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000670f33f0/0xaf35132e63033dfa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2113536->35667967] (req 2113536->2121727) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f3a expref: 12 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.297308:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f2300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6461:1866777604937728:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298643.297341:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.297972:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f2300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6461:1866777604937728:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298643.297982:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298643.298004:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f747800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609468288:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 796us (911us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.298010:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15078 00000100:00100000:3.0:1780298643.298205:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.298227:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609468416 00000100:00100000:0.0:1780298643.298231:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609468416 00002000:00100000:0.0:1780298643.298251:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x12:0x0] (2113536->2117631) req@ffff9cf242861180 x1866777609468416/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.298258:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242861180 with x1866777609468416 ext(2113536->2117631) 00002000:00010000:0.0:1780298643.298261:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf242861180 00000100:00100000:0.0:1780298643.298277:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6124 00000100:00100000:0.0:1780298643.298282:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242861180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609468416:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298643.298404:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242861180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609468416:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 121us (203us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298643.298414:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6124 00002000:00100000:0.0:1780298643.298418:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x12:0x0] (2113536->2117631) req@ffff9cf242861180 x1866777609468416/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.298429:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242861180 with x1866777609468416 ext(2113536->2117631) 00002000:00010000:0.0:1780298643.298431:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf242861180 00000100:00100000:0.0:1780298643.299036:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.299086:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609468544 00000100:00100000:3.0:1780298643.299091:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609468544 00002000:00100000:3.0:1780298643.299119:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x12:0x0] (2117632->2121727) req@ffff9cf2425f0380 x1866777609468544/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298643.299143:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2425f0380 with x1866777609468544 ext(2117632->2121727) 00002000:00010000:3.0:1780298643.299148:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2425f0380 00000100:00100000:3.0:1780298643.299189:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6125 00000100:00100000:3.0:1780298643.299200:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609468544:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298643.299342:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609468544:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 143us (303us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298643.299353:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6125 00002000:00100000:2.0:1780298643.299361:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x12:0x0] (2117632->2121727) req@ffff9cf2425f0380 x1866777609468544/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.299371:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2425f0380 with x1866777609468544 ext(2117632->2121727) 00002000:00010000:2.0:1780298643.299375:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2425f0380 00000100:00100000:3.0:1780298643.300257:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.300296:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609468800 00000100:00100000:1.0:1780298643.300300:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609468800 00000100:00100000:1.0:1780298643.300328:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9296 00000100:00100000:1.0:1780298643.300333:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243061500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609468800:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.300340:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.300348:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e63033e01 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2f48 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.300372:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5db100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604937856:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.300411:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348439840 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.300421:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.300443:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609468672 00000100:00100000:1.0:1780298643.300447:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609468672 00000100:00100000:1.0:1780298643.300481:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9297 00000100:00100000:1.0:1780298643.300487:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2397:x1866777609468672:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.300493:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.300502:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e63033e08 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2f41 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.300524:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604937984:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.300547:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837ae40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.301020:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5db100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604937856:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.301037:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e63033e01 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f48 expref: 14 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.301052:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e63033e01 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f48 expref: 14 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.301061:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92a00, rc 0) 00000100:00100000:1.0:1780298643.301091:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243061500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609468800:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 757us (834us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.301096:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9296 00000100:00100000:1.0:1780298643.301338:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604937984:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.301349:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e63033e08 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f41 expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.301357:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e63033e08 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f41 expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.301363:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92600, rc 0) 00000100:00100000:1.0:1780298643.301387:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2397:x1866777609468672:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 901us (969us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.301390:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9297 00000100:00100000:0.0:1780298643.302066:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.302101:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.302114:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609468928 00000100:00100000:2.0:1780298643.302118:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609468928 00000100:00100000:2.0:1780298643.302172:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609469056 00000100:00100000:2.0:1780298643.302175:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609469056 00000100:00100000:1.0:1780298643.302184:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9298 00000100:00100000:0.0:1780298643.302193:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9299 00000100:00100000:1.0:1780298643.302198:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2398:x1866777609468928:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298643.302208:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609469056:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.302211:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.302217:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.302224:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e90122d2/0xaf35132e63033e0f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2f4f expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.302227:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63033e16 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2f56 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.302267:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604938112:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.302275:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604938240:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.302320:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417d80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.302321:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837ae40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.303044:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604938240:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.303062:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e90122d2/0xaf35132e63033e0f lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f4f expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.303074:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604938112:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.303075:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e90122d2/0xaf35132e63033e0f lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f4f expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.303085:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93c00, rc 0) 00010000:00010000:0.0:1780298643.303087:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63033e16 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f56 expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.303099:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63033e16 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f56 expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.303109:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf400, rc 0) 00000100:00100000:2.0:1780298643.303116:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2398:x1866777609468928:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 925us (1052us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.303149:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609469056:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 944us (1046us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.303153:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9298 00000100:00100000:0.0:1780298643.303160:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9299 00000100:00100000:2.0:1780298643.304227:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.304260:0:6476:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609469312 00000100:00100000:1.0:1780298643.304265:0:6476:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609469312 00000100:00100000:1.0:1780298643.304308:0:6476:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61975 00000100:00100000:1.0:1780298643.304317:0:6476:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+134:29360:x1866777609469312:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:1.0:1780298643.304357:0:6476:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298643.304372:0:6476:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3000/0xaf35132e63033dd7 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 71 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f17 expref: 134 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.304383:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298643.304386:0:6476:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298643.304525:0:6476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92c00/0xaf35132e63033e1d lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 70 type: IBT flags: 0x40000000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.304544:0:6476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92c00/0xaf35132e63033e1d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 70 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.304550:0:6476:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92c00/0xaf35132e63033e1d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 70 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1780298643.304586:0:6476:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991044 00000004:00010000:1.0:1780298643.304606:0:6476:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 0000000061e52f61 reply state 000000009b7af421 transno 4294991045 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92c00/0xaf35132e63033e1d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 70 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.304617:0:6476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92c00/0xaf35132e63033e1d lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 70 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.304621:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298643.304623:0:6476:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1780298643.304625:0:6476:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1780298643.304681:0:6476:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+136:29360:x1866777609469312:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 365us (453us total) trans 4294991045 rc 0/0 00000100:00100000:1.0:1780298643.304687:0:6476:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61975 00000020:00080000:1.0:1780298643.304691:0:6476:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1780298643.305350:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.305386:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609469440 00000100:00100000:2.0:1780298643.305392:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609469440 00000100:00100000:2.0:1780298643.305418:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9300 00000100:00100000:2.0:1780298643.305423:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243063100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:29360:x1866777609469440:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.305429:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.305436:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63033e24 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2f5d expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.305455:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e63033d83 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2edf expref: 6 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.305461:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(139 177) 00010000:00010000:2.0:1780298643.305463:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(140 177) 00010000:00010000:2.0:1780298643.305466:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63033e24 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f5d expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.305480:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e63033d83 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2edf expref: 6 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.305488:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e63033d83 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2edf expref: 6 pid: 12101 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.305496:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604938368:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298643.305527:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4900 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.306093:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604938368:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298643.306109:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63033e24 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f5d expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.306121:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2000, rc 0) 00000100:00100000:2.0:1780298643.306179:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243063100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609469440:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 754us (830us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.306184:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9300 00000100:00100000:0.0:1780298643.306373:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.306405:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609469568 00000100:00100000:3.0:1780298643.306409:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609469568 00010000:00010000:3.0:1780298643.306425:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e63033d83 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2edf expref: 7 pid: 12101 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.306444:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15079 00000100:00100000:3.0:1780298643.306450:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609469568:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298643.306456:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298643.306465:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e63033d83 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2edf expref: 7 pid: 12101 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.306472:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e63033d83 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2edf expref: 7 pid: 12101 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.306482:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e63033d83 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2edf expref: 7 pid: 12101 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.306491:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(140 177) 00010000:00010000:3.0:1780298643.306493:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(140 177) 00010000:00010000:3.0:1780298643.306497:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63033e24 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f5d expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.306502:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63033e24 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f5d expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.306517:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63033e24 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f5d expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.306523:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243062d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777604938496:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298643.306552:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856900 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.307106:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243062d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777604938496:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298643.307116:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298643.307159:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609469568:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 710us (787us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298643.307166:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15079 00000100:00100000:3.0:1780298643.307235:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.307260:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609469696 00000100:00100000:1.0:1780298643.307263:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609469696 00000100:00100000:1.0:1780298643.307298:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9301 00000100:00100000:1.0:1780298643.307307:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243063800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:29360:x1866777609469696:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.307315:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.307323:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e63033e2b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2f64 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.307340:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000670f33f0/0xaf35132e63033dfa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2113536->35667967] (req 2113536->2121727) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f3a expref: 14 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.307346:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(104 130) 00010000:00010000:1.0:1780298643.307348:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(105 130) 00010000:00010000:1.0:1780298643.307351:0:20252:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e63033e2b lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f64 expref: 14 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.307363:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000670f33f0/0xaf35132e63033dfa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2113536->35667967] (req 2113536->2121727) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f3a expref: 14 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.307370:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000670f33f0/0xaf35132e63033dfa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2113536->35667967] (req 2113536->2121727) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f3a expref: 14 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.307376:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5db100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604938624:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298643.307403:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.307856:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5db100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604938624:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298643.307867:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e63033e2b lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f64 expref: 14 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.307874:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92a00, rc 0) 00000100:00100000:1.0:1780298643.307894:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243063800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:29360:x1866777609469696:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 590us (660us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.307905:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9301 00000100:00100000:3.0:1780298643.308102:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.308147:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609469824 00000100:00100000:1.0:1780298643.308151:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609469824 00002000:00100000:1.0:1780298643.308169:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (2113536->2121727) req@ffff9cf243063b80 x1866777609469824/t0(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 8680/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.308176:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243063b80 with x1866777609469824 ext(2113536->2121727) 00010000:00010000:1.0:1780298643.308181:0:8362:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000670f33f0/0xaf35132e63033dfa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2113536->35667967] (req 2113536->2121727) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f3a expref: 14 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.308191:0:8362:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000670f33f0/0xaf35132e63033dfa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2113536->35667967] (req 2113536->2121727) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f3a expref: 14 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1780298643.308199:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9cf243063b80 00000100:00100000:1.0:1780298643.308213:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 195 00000100:00100000:1.0:1780298643.308219:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243063b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609469824:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:1.0:1780298643.308690:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243063b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2399:x1866777609469824:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 470us (587us total) trans 4294969224 rc 0/0 00000100:00100000:1.0:1780298643.308698:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 195 00002000:00100000:1.0:1780298643.308705:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (2113536->2121727) req@ffff9cf243063b80 x1866777609469824/t4294969224(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 8680/448 e 0 to 0 dl 1780298654 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.308715:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243063b80 with x1866777609469824 ext(2113536->2121727) 00010000:00010000:1.0:1780298643.308720:0:8362:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000670f33f0/0xaf35132e63033dfa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2113536->35667967] (req 2113536->2121727) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f3a expref: 16 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.308730:0:8362:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000670f33f0/0xaf35132e63033dfa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2113536->35667967] (req 2113536->2121727) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f3a expref: 16 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1780298643.308735:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9cf243063b80 00000100:00100000:1.0:1780298643.309303:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.309343:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609469952 00000100:00100000:2.0:1780298643.309347:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609469952 00010000:00010000:2.0:1780298643.309364:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000670f33f0/0xaf35132e63033dfa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2113536->35667967] (req 2113536->2121727) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f3a expref: 16 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.309389:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15080 00000100:00100000:2.0:1780298643.309396:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5db100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2398:x1866777609469952:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298643.309404:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:2.0:1780298643.309411:0:6462:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 96 -> 112 00010000:00010000:2.0:1780298643.309417:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000670f33f0/0xaf35132e63033dfa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2113536->35667967] (req 2113536->2121727) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f3a expref: 16 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.309428:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000670f33f0/0xaf35132e63033dfa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2113536->35667967] (req 2113536->2121727) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f3a expref: 16 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.309439:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000670f33f0/0xaf35132e63033dfa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2113536->35667967] (req 2113536->2121727) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f3a expref: 16 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.309447:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(105 130) 00010000:00010000:2.0:1780298643.309449:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(105 130) 00010000:00010000:2.0:1780298643.309453:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e63033e2b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f64 expref: 15 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.309458:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e63033e2b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f64 expref: 15 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.309471:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e63033e2b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f64 expref: 15 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.309477:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd4e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777604938752:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298643.309505:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4900 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.310152:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd4e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777604938752:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298643.310163:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298643.310192:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5db100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2398:x1866777609469952:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 795us (887us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.310199:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15080 00000100:00100000:0.0:1780298643.310324:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.310354:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609470080 00000100:00100000:2.0:1780298643.310358:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609470080 00002000:00010000:2.0:1780298643.310372:0:8364:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (225671->18446744073709551615) 00002000:00100000:2.0:1780298643.310375:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf375f56050 with x1866777609470080 ext(225671->18446744073709551615) 00002000:00010000:2.0:1780298643.310380:0:8364:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf375f56050. 00000100:00100000:0.0:1780298643.310386:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.310400:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609470208 00000100:00100000:2.0:1780298643.310403:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609470208 00002000:00010000:2.0:1780298643.310411:0:8364:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00002000:00100000:2.0:1780298643.310414:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2425f1c00 with x1866777609470208 ext(0->18446744073709551615) 00002000:00010000:2.0:1780298643.310417:0:8364:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2425f1c00. 00000100:00100000:2.0:1780298643.310435:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6126 00000100:00100000:1.0:1780298643.310443:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6127 00000100:00100000:2.0:1780298643.310444:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf375f56050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609470080:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:1.0:1780298643.310456:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2397:x1866777609470208:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:1.0:1780298643.312919:0:19928:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4194304 -> 0 00002000:00010000:1.0:1780298643.312923:0:19928:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298643.312925:0:19928:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298643.312927:0:19928:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298643.312929:0:19928:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 112 -> 0 00000100:00100000:1.0:1780298643.312954:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609470208:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 2502us (2568us total) trans 4294969225 rc 0/0 00000100:00100000:1.0:1780298643.312959:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6127 00002000:00010000:1.0:1780298643.312962:0:19928:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00002000:00100000:1.0:1780298643.312964:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2425f1c00 with x1866777609470208 ext(0->18446744073709551615) 00002000:00010000:1.0:1780298643.312968:0:19928:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2425f1c00. 00002000:00010000:2.0:1780298643.313639:0:8364:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4983467 -> 225671 00002000:00010000:2.0:1780298643.313641:0:8364:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:2.0:1780298643.313643:0:8364:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:2.0:1780298643.313645:0:8364:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:2.0:1780298643.313647:0:8364:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 464 -> 296 00000100:00100000:2.0:1780298643.313671:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf375f56050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609470080:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 3230us (3349us total) trans 4294969667 rc 0/0 00000100:00100000:2.0:1780298643.313676:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6126 00002000:00010000:2.0:1780298643.313680:0:8364:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (225671->18446744073709551615) 00002000:00100000:2.0:1780298643.313682:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf375f56050 with x1866777609470080 ext(225671->18446744073709551615) 00002000:00010000:2.0:1780298643.313685:0:8364:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf375f56050. 00000100:00100000:0.0:1780298643.314357:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.314387:0:9976:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609470336 00000100:00100000:2.0:1780298643.314391:0:9976:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609470336 00000100:00100000:2.0:1780298643.314416:0:9976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61976 00000100:00100000:2.0:1780298643.314421:0:9976:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+108:29360:x1866777609470336:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.314432:0:9976:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.314441:0:9976:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2400/0xaf35132e63033e32 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 71 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2f6b expref: -99 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.314498:0:9976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2e00/0xaf35132e63033e39 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 72 type: IBT flags: 0x40000000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.314508:0:9976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2e00/0xaf35132e63033e39 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 72 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.314515:0:9976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2e00/0xaf35132e63033e39 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 72 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298643.314532:0:9976:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 225671 00000004:00200000:2.0:1780298643.314535:0:9976:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298643.314549:0:9976:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2e00/0xaf35132e63033e39 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 72 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.314573:0:9976:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2400/0xaf35132e63033e32 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 72 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f6b expref: 110 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.314588:0:9976:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2e00/0xaf35132e63033e39 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 71 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f6b expref: 109 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.314596:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298643.314599:0:9976:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298643.314602:0:9976:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2e00, rc 0) 00000100:00100000:2.0:1780298643.314624:0:9976:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+109:29360:x1866777609470336:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 203us (268us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.314629:0:9976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61976 00000100:00100000:3.0:1780298643.315404:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.315432:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609470464 00000100:00100000:2.0:1780298643.315435:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609470464 00000100:00100000:2.0:1780298643.315457:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9302 00000100:00100000:2.0:1780298643.315462:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243060a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609470464:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.315468:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.315475:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e63033e40 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2f72 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.315498:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604938880:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.315526:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.315540:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.315582:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609470592 00000100:00100000:3.0:1780298643.315587:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609470592 00000100:00100000:3.0:1780298643.315609:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9303 00000100:00100000:3.0:1780298643.315616:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609470592:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.315622:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.315629:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000fa1945a/0xaf35132e63033e47 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2f79 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.315706:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243061f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604939008:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.315740:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.316299:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604938880:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.316311:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e63033e40 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f72 expref: 7 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.316327:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e63033e40 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f72 expref: 7 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.316334:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3c00, rc 0) 00000100:00100000:3.0:1780298643.316342:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243061f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604939008:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.316359:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000fa1945a/0xaf35132e63033e47 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f79 expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.316359:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243060a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609470464:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 895us (954us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.316375:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9302 00010000:00010000:3.0:1780298643.316382:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000fa1945a/0xaf35132e63033e47 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f79 expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.316389:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3000, rc 0) 00000100:00100000:3.0:1780298643.316416:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609470592:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 802us (891us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.316422:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9303 00000100:00100000:0.0:1780298643.317261:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.317293:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609470720 00000100:00100000:2.0:1780298643.317297:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609470720 00000100:00100000:2.0:1780298643.317318:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9304 00000100:00100000:2.0:1780298643.317323:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609470720:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.317329:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.317335:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63033e4e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2f80 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.317354:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604939136:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.317390:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.317418:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.317462:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609470848 00000100:00100000:0.0:1780298643.317467:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609470848 00000100:00100000:0.0:1780298643.317492:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9305 00000100:00100000:0.0:1780298643.317498:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609470848:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.317504:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.317510:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e63033e55 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2f87 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.317531:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604939264:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.317561:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417d80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.317934:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604939136:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.317945:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63033e4e lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f80 expref: 7 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.317953:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63033e4e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f80 expref: 7 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.317964:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2600, rc 0) 00000100:00100000:2.0:1780298643.317988:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609470720:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 665us (728us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.317993:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9304 00000100:00100000:1.0:1780298643.318054:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604939264:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.318072:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e63033e55 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f87 expref: 12 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.318083:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e63033e55 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f87 expref: 12 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.318091:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf200, rc 0) 00000100:00100000:1.0:1780298643.318113:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609470848:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 617us (696us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.318119:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9305 00000100:00100000:3.0:1780298643.318902:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.318928:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609470976 00000100:00100000:1.0:1780298643.318932:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609470976 00000100:00100000:1.0:1780298643.318954:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9306 00000100:00100000:1.0:1780298643.318959:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243061f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609470976:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.318964:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.318972:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000186136f8/0xaf35132e63033e5c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2f8e expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.318994:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5db480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604939392:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.319040:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.319045:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.319081:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609471104 00000100:00100000:3.0:1780298643.319086:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609471104 00000100:00100000:3.0:1780298643.319110:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9307 00000100:00100000:3.0:1780298643.319116:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609471104:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.319174:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.319181:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d828f4f8/0xaf35132e63033e63 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2f95 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.319201:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243060000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604939520:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.319229:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.319643:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5db480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604939392:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.319658:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000186136f8/0xaf35132e63033e5c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f8e expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.319672:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000186136f8/0xaf35132e63033e5c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f8e expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.319681:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93000, rc 0) 00000100:00100000:1.0:1780298643.319705:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243061f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609470976:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 745us (803us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.319709:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9306 00000100:00100000:3.0:1780298643.319855:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243060000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604939520:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.319870:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d828f4f8/0xaf35132e63033e63 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f95 expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.319890:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d828f4f8/0xaf35132e63033e63 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f95 expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.319900:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2e00, rc 0) 00000100:00100000:3.0:1780298643.319924:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609471104:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 809us (884us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.319929:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9307 00000100:00100000:0.0:1780298643.320704:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.320732:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609471232 00000100:00100000:1.0:1780298643.320735:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609471232 00000100:00100000:1.0:1780298643.320759:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9308 00000100:00100000:1.0:1780298643.320766:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609471232:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.320772:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.320779:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c1a1ebef/0xaf35132e63033e6a lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2f9c expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.320789:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.320806:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5db480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604939648:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.320820:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609471360 00000100:00100000:0.0:1780298643.320824:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609471360 00000100:00100000:1.0:1780298643.320842:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.320859:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9309 00000100:00100000:0.0:1780298643.320865:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243060000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609471360:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.320871:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.320878:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d4d8e3d7/0xaf35132e63033e71 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2fa3 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.320906:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604939776:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.320931:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.321368:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5db480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604939648:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.321378:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c1a1ebef/0xaf35132e63033e6a lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f9c expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.321387:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c1a1ebef/0xaf35132e63033e6a lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f9c expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.321393:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93a00, rc 0) 00000100:00100000:1.0:1780298643.321417:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609471232:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 653us (713us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.321425:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9308 00000100:00100000:0.0:1780298643.321608:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604939776:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.321618:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d4d8e3d7/0xaf35132e63033e71 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2fa3 expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.321626:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d4d8e3d7/0xaf35132e63033e71 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2fa3 expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.321632:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf000, rc 0) 00000100:00100000:0.0:1780298643.321651:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243060000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609471360:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 787us (866us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.321656:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9309 00000100:00100000:2.0:1780298643.322487:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.322531:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609471488 00000100:00100000:0.0:1780298643.322536:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609471488 00000100:00100000:0.0:1780298643.322567:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9310 00000100:00100000:0.0:1780298643.322576:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609471488:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.322584:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.322594:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e63033e78 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2faa expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.322623:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604939904:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.322633:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.322673:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609471616 00000100:00100000:0.0:1780298643.322678:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.322679:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609471616 00000100:00100000:1.0:1780298643.322713:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9311 00000100:00100000:1.0:1780298643.322723:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243063480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609471616:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.322729:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.322735:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b87f9a7e/0xaf35132e63033e7f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2fb1 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.322752:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5db480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604940032:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.322779:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.323272:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604939904:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.323296:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e63033e78 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2faa expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.323311:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e63033e78 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2faa expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.323318:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae600, rc 0) 00000100:00100000:0.0:1780298643.323342:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609471488:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 769us (857us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.323348:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9310 00000100:00100000:1.0:1780298643.323399:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5db480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604940032:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.323414:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b87f9a7e/0xaf35132e63033e7f lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2fb1 expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.323423:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b87f9a7e/0xaf35132e63033e7f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2fb1 expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.323430:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93600, rc 0) 00000100:00100000:1.0:1780298643.323453:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243063480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609471616:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 735us (822us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.323458:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9311 00000100:00100000:2.0:1780298643.324172:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.324196:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609471744 00000100:00100000:0.0:1780298643.324201:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609471744 00000100:00100000:0.0:1780298643.324231:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9312 00000100:00100000:0.0:1780298643.324241:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609471744:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.324250:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.324259:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e63033e86 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2fb8 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.324268:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.324308:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609471872 00000100:00100000:0.0:1780298643.324308:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604940160:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.324314:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609471872 00000100:00100000:1.0:1780298643.324359:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9313 00000100:00100000:0.0:1780298643.324369:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.324375:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609471872:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.324384:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.324393:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000056a806f/0xaf35132e63033e8d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2fbf expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.324418:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5db480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604940288:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.324450:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.324894:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604940160:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.324904:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e63033e86 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2fb8 expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.324914:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e63033e86 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2fb8 expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.324920:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae000, rc 0) 00000100:00100000:0.0:1780298643.324940:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609471744:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 704us (771us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.324945:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9312 00000100:00100000:1.0:1780298643.325043:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5db480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604940288:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.325054:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000056a806f/0xaf35132e63033e8d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2fbf expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.325066:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000056a806f/0xaf35132e63033e8d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2fbf expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.325073:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92800, rc 0) 00000100:00100000:1.0:1780298643.325095:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609471872:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 730us (830us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.325100:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9313 00000100:00100000:2.0:1780298643.325894:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.325930:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609472128 00000100:00100000:1.0:1780298643.325934:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609472128 00000100:00100000:2.0:1780298643.325949:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.325959:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609472000 00000100:00100000:1.0:1780298643.325962:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609472000 00000100:00100000:0.0:1780298643.325990:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9314 00000100:00100000:0.0:1780298643.325998:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609472128:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298643.326004:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9315 00010000:00010000:0.0:1780298643.326006:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298643.326014:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609472000:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.326014:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d4d8e3d7/0xaf35132e63033e94 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2fcd expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.326024:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.326033:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000670f33f0/0xaf35132e63033e9b lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2fc6 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.326041:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604940416:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.326066:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5db480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604940544:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.326077:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.326092:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.326710:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f1500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604940416:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.326726:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d4d8e3d7/0xaf35132e63033e94 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2fcd expref: 12 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.326749:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d4d8e3d7/0xaf35132e63033e94 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2fcd expref: 12 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.326759:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf000, rc 0) 00000100:00100000:0.0:1780298643.326792:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609472128:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 794us (896us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.326799:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5db480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604940544:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.326804:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9314 00010000:00010000:1.0:1780298643.326826:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000670f33f0/0xaf35132e63033e9b lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2fc6 expref: 7 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.326836:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000670f33f0/0xaf35132e63033e9b lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2fc6 expref: 7 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.326843:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92e00, rc 0) 00000100:00100000:1.0:1780298643.326866:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609472000:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 857us (917us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.326872:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9315 00000100:00100000:2.0:1780298643.327739:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.327769:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609472256 00000100:00100000:1.0:1780298643.327773:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609472256 00000100:00100000:1.0:1780298643.327796:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9316 00000100:00100000:1.0:1780298643.327801:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609472256:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.327807:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.327814:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000038a88423/0xaf35132e63033ea2 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2fd4 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.327833:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5db480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604940672:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.327858:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.327870:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.327903:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609472384 00000100:00100000:2.0:1780298643.327906:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609472384 00000100:00100000:2.0:1780298643.327933:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9317 00000100:00100000:2.0:1780298643.327941:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243062300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609472384:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.327948:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.327955:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e63033ea9 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2fdb expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.327979:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604940800:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.328007:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.328391:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5db480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604940672:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.328402:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000038a88423/0xaf35132e63033ea2 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2fd4 expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.328410:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000038a88423/0xaf35132e63033ea2 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2fd4 expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.328416:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93e00, rc 0) 00000100:00100000:1.0:1780298643.328440:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609472256:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 639us (701us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.328444:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9316 00000100:00100000:1.0:1780298643.328570:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604940800:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.328582:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e63033ea9 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2fdb expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.328590:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e63033ea9 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2fdb expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.328596:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2600, rc 0) 00000100:00100000:1.0:1780298643.328615:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243062300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609472384:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 676us (759us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.328620:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9317 00000100:00100000:2.0:1780298643.329302:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.329339:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609472512 00000100:00100000:1.0:1780298643.329343:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609472512 00000100:00100000:1.0:1780298643.329368:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9318 00000100:00100000:1.0:1780298643.329376:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:29360:x1866777609472512:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.329385:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.329393:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e63033eb0 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2fe2 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.329409:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63033e24 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f5d expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.329415:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(140 177) 00010000:00010000:1.0:1780298643.329418:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(141 177) 00010000:00010000:1.0:1780298643.329422:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e63033eb0 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [69632->77823] (req 69632->77823) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2fe2 expref: 7 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.329436:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63033e24 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f5d expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.329447:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63033e24 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f5d expref: 9 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.329458:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5db480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604940928:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298643.329497:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.330040:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5db480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604940928:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298643.330051:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e63033eb0 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [69632->77823] (req 69632->77823) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2fe2 expref: 7 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.330058:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92400, rc 0) 00000100:00100000:1.0:1780298643.330079:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609472512:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 704us (777us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.330086:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9318 00000100:00100000:0.0:1780298643.330299:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.330336:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609472640 00000100:00100000:3.0:1780298643.330340:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609472640 00010000:00010000:3.0:1780298643.330356:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63033e24 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f5d expref: 10 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.330377:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15081 00000100:00100000:3.0:1780298643.330382:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f1500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609472640:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298643.330390:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298643.330398:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63033e24 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f5d expref: 10 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.330406:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63033e24 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f5d expref: 10 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.330421:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63033e24 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f5d expref: 10 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.330429:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(141 177) 00010000:00010000:3.0:1780298643.330431:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(141 177) 00010000:00010000:3.0:1780298643.330437:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e63033eb0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [69632->33624063] (req 69632->77823) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2fe2 expref: 6 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.330442:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e63033eb0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [69632->33624063] (req 69632->77823) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2fe2 expref: 6 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.330457:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e63033eb0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [69632->33624063] (req 69632->77823) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2fe2 expref: 6 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.330464:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243063b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777604941056:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298643.330493:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.331225:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243063b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777604941056:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298643.331239:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298643.331269:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f1500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609472640:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 885us (969us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.331282:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15081 00000100:00100000:0.0:1780298643.331282:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.331344:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609472768 00000100:00100000:1.0:1780298643.331350:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609472768 00000100:00100000:1.0:1780298643.331390:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6128 00000100:00100000:1.0:1780298643.331399:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609472768:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298643.331619:0:19943:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 225671 -> 225671 00002000:00010000:1.0:1780298643.331623:0:19943:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298643.331626:0:19943:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298643.331628:0:19943:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298643.331629:0:19943:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 296 -> 304 00000100:00100000:1.0:1780298643.331660:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609472768:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 263us (382us total) trans 4294969668 rc 0/0 00000100:00100000:1.0:1780298643.331666:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6128 00000100:00100000:0.0:1780298643.332364:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.332396:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609472896 00000100:00100000:1.0:1780298643.332401:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609472896 00000100:00100000:1.0:1780298643.332431:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9319 00000100:00100000:1.0:1780298643.332439:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b00380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609472896:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.332448:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.332457:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000056a806f/0xaf35132e63033eb7 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2fe9 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.332470:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.332487:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5db480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604941184:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.332531:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609473024 00000100:00100000:0.0:1780298643.332536:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609473024 00000100:00100000:1.0:1780298643.332537:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.332569:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9320 00000100:00100000:0.0:1780298643.332579:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243062d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609473024:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.332585:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.332592:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63033ebe lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2ff0 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.332614:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b00e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604941312:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.332642:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.333050:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5db480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604941184:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.333067:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000056a806f/0xaf35132e63033eb7 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2fe9 expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.333082:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000056a806f/0xaf35132e63033eb7 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2fe9 expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.333092:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92800, rc 0) 00000100:00100000:1.0:1780298643.333147:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b00380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609472896:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 685us (759us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.333156:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9319 00000100:00100000:0.0:1780298643.333210:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b00e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604941312:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.333225:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63033ebe lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ff0 expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.333241:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63033ebe lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ff0 expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.333247:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae000, rc 0) 00000100:00100000:0.0:1780298643.333269:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243062d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609473024:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 693us (800us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.333274:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9320 00000100:00100000:3.0:1780298643.333994:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.334027:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609473152 00000100:00100000:1.0:1780298643.334030:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609473152 00000100:00100000:1.0:1780298643.334052:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9321 00000100:00100000:1.0:1780298643.334057:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243063b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609473152:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.334063:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.334070:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e63033ec5 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2ff7 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.334089:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5db480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604941440:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.334122:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.334174:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.334205:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609473280 00000100:00100000:1.0:1780298643.334209:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609473280 00000100:00100000:1.0:1780298643.334239:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9322 00000100:00100000:1.0:1780298643.334248:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609473280:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.334255:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.334263:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c1a1ebef/0xaf35132e63033ecc lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d2ffe expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.334311:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5dad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604941568:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.334345:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a480 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.334636:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5db480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604941440:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.334650:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e63033ec5 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ff7 expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.334669:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e63033ec5 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ff7 expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.334679:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93600, rc 0) 00000100:00100000:1.0:1780298643.334713:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243063b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609473152:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 650us (714us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.334725:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9321 00000100:00100000:1.0:1780298643.334816:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5dad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604941568:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.334827:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c1a1ebef/0xaf35132e63033ecc lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ffe expref: 12 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.334835:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c1a1ebef/0xaf35132e63033ecc lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2ffe expref: 12 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.334840:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93a00, rc 0) 00000100:00100000:1.0:1780298643.334860:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609473280:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 616us (689us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.334865:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9322 00000100:00100000:3.0:1780298643.335566:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.335615:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609473408 00000100:00100000:2.0:1780298643.335620:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609473408 00000100:00100000:2.0:1780298643.335661:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6129 00000100:00100000:2.0:1780298643.335670:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243808700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609473408:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298643.335871:0:19949:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 225671 -> 225671 00002000:00010000:2.0:1780298643.335875:0:19949:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:2.0:1780298643.335878:0:19949:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:2.0:1780298643.335880:0:19949:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:2.0:1780298643.335882:0:19949:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 304 -> 304 00000100:00100000:2.0:1780298643.335924:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243808700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609473408:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 255us (357us total) trans 4294969669 rc 0/0 00000100:00100000:2.0:1780298643.335931:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6129 00000100:00100000:3.0:1780298643.336602:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.336632:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609473536 00000100:00100000:1.0:1780298643.336636:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609473536 00000100:00100000:1.0:1780298643.336663:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61977 00000100:00100000:1.0:1780298643.336669:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243808e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+136:29360:x1866777609473536:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.336682:0:6477:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.336693:0:6477:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93e00/0xaf35132e63033ed3 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 72 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3005 expref: -99 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.336761:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92e00/0xaf35132e63033eda lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 73 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.336771:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92e00/0xaf35132e63033eda lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 73 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.336779:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92e00/0xaf35132e63033eda lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 73 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1780298643.336801:0:6477:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 225671 00000004:00200000:1.0:1780298643.336804:0:6477:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1780298643.336818:0:6477:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92e00/0xaf35132e63033eda lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 73 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.336835:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93e00/0xaf35132e63033ed3 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 73 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3005 expref: 138 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.336842:0:6477:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92e00/0xaf35132e63033eda lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 72 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3005 expref: 137 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.336848:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298643.336850:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1780298643.336852:0:6477:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92e00, rc 0) 00000100:00100000:1.0:1780298643.336873:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243808e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+137:29360:x1866777609473536:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 206us (273us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.336877:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61977 00000100:00100000:0.0:1780298643.337759:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.337788:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609473664 00000100:00100000:1.0:1780298643.337791:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609473664 00000100:00100000:1.0:1780298643.337813:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9323 00000100:00100000:1.0:1780298643.337818:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b00e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609473664:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.337824:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.337831:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000186136f8/0xaf35132e63033ee1 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d300c expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.337856:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5dad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604941696:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.337888:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a480 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.338472:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5dad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604941696:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.338491:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000186136f8/0xaf35132e63033ee1 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d300c expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.338505:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000186136f8/0xaf35132e63033ee1 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d300c expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.338514:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93000, rc 0) 00000100:00100000:1.0:1780298643.338544:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b00e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609473664:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 725us (786us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.338550:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9323 00000100:00100000:3.0:1780298643.339250:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.339277:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609473792 00000100:00100000:0.0:1780298643.339282:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609473792 00000100:00100000:0.0:1780298643.339319:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9324 00000100:00100000:0.0:1780298643.339325:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24380b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609473792:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.339332:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.339338:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e63033ee8 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3013 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.339361:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b01c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604941824:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.339392:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.340006:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b01c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604941824:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.340021:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e63033ee8 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3013 expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.340034:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e63033ee8 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3013 expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.340043:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf000, rc 0) 00000100:00100000:1.0:1780298643.340068:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24380b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609473792:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 744us (819us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.340075:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9324 00000100:00100000:3.0:1780298643.342591:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.342640:0:7411:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609474048 00000100:00100000:2.0:1780298643.342646:0:7411:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609474048 00000100:00100000:2.0:1780298643.342687:0:7411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61978 00000100:00100000:2.0:1780298643.342697:0:7411:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24380ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+137:29360:x1866777609474048:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:2.0:1780298643.342735:0:7411:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298643.342753:0:7411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92e00/0xaf35132e63033eda lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 73 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3005 expref: 137 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.342767:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298643.342772:0:7411:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298643.342913:0:7411:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3a00/0xaf35132e63033eef lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 72 type: IBT flags: 0x40000000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.342927:0:7411:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2e00/0xaf35132e63033e39 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 72 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f6b expref: 108 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.342948:0:7411:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3a00/0xaf35132e63033eef lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 72 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.342971:0:7411:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2e00/0xaf35132e63033e39 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 72 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f6b expref: 108 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.342983:0:7411:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2e00/0xaf35132e63033e39 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 72 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f6b expref: 108 pid: 9976 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.342994:0:7411:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f746680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7411:1866777604941952:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298643.343038:0:7411:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.343656:0:7411:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f746680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7411:1866777604941952:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298643.343668:0:7411:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3a00/0xaf35132e63033eef lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 72 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.343894:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.343940:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609474176 00000100:00100000:1.0:1780298643.343945:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609474176 00010000:00010000:1.0:1780298643.343964:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2e00/0xaf35132e63033e39 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 72 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f6b expref: 109 pid: 9976 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.343987:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15082 00000100:00100000:1.0:1780298643.343999:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243808000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+109:2399:x1866777609474176:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298643.344008:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298643.344011:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2e00/0xaf35132e63033e39 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 73 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f6b expref: 109 pid: 9976 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.344019:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2e00/0xaf35132e63033e39 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 73 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f6b expref: 109 pid: 9976 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.344029:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2e00/0xaf35132e63033e39 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 73 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f6b expref: 109 pid: 9976 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.344040:0:20253:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298643.344042:0:20253:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3a00/0xaf35132e63033eef lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 72 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.344054:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3a00/0xaf35132e63033eef lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 72 type: IBT flags: 0x40290000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.344058:0:20253:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3a00/0xaf35132e63033eef lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 72 type: IBT flags: 0x40290000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.344072:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298643.344089:0:7411:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3a00/0xaf35132e63033eef lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 71 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.344094:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243808000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+108:2399:x1866777609474176:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 101us (201us total) trans 0 rc 0/0 00010000:00010000:2.0:1780298643.344106:0:7411:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3a00/0xaf35132e63033eef lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 71 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.344110:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15082 00000001:00080000:2.0:1780298643.344175:0:7411:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991046 00000004:00010000:2.0:1780298643.344198:0:7411:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000111b859d reply state 00000000436e8471 transno 4294991047 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3a00/0xaf35132e63033eef lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 71 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.344211:0:7411:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3a00/0xaf35132e63033eef lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 71 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.344216:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298643.344218:0:7411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1780298643.344220:0:7411:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1780298643.344277:0:7411:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24380ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+139:29360:x1866777609474048:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1581us (1686us total) trans 4294991047 rc 0/0 00000100:00100000:2.0:1780298643.344294:0:7411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61978 00000020:00080000:2.0:1780298643.344298:0:7411:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1780298643.344949:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.344994:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609474304 00000100:00100000:0.0:1780298643.345000:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609474304 00000100:00100000:0.0:1780298643.345036:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9325 00000100:00100000:0.0:1780298643.345046:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243809180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609474304:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.345056:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.345066:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63033ef6 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d301a expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.345086:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e63033eb0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [69632->33624063] (req 69632->77823) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2fe2 expref: 8 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.345095:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(141 177) 00010000:00010000:0.0:1780298643.345099:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(142 177) 00010000:00010000:0.0:1780298643.345104:0:20252:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63033ef6 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d301a expref: 10 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.345143:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e63033eb0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [69632->33624063] (req 69632->77823) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2fe2 expref: 8 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.345156:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e63033eb0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [69632->33624063] (req 69632->77823) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2fe2 expref: 8 pid: 12101 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.345168:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b03480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604942080:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298643.345219:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417540 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.345774:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b03480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604942080:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298643.345790:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63033ef6 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d301a expref: 10 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.345802:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bafa00, rc 0) 00000100:00100000:0.0:1780298643.345834:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243809180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609474304:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 790us (885us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298643.345845:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9325 00000100:00100000:2.0:1780298643.346029:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.346070:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609474432 00000100:00100000:0.0:1780298643.346075:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609474432 00010000:00010000:0.0:1780298643.346093:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e63033eb0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [69632->33624063] (req 69632->77823) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2fe2 expref: 9 pid: 12101 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.346120:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15083 00000100:00100000:0.0:1780298643.346152:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b5f450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609474432:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298643.346162:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298643.346179:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e63033eb0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [69632->33624063] (req 69632->77823) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2fe2 expref: 9 pid: 12101 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.346189:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e63033eb0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [69632->33624063] (req 69632->77823) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2fe2 expref: 9 pid: 12101 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.346204:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e63033eb0 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [69632->33624063] (req 69632->77823) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2fe2 expref: 9 pid: 12101 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.346214:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(142 177) 00010000:00010000:0.0:1780298643.346217:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(142 177) 00010000:00010000:0.0:1780298643.346222:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63033ef6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d301a expref: 9 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.346231:0:6463:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63033ef6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d301a expref: 9 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.346258:0:6463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63033ef6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d301a expref: 9 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.346269:0:6463:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b03480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6463:1866777604942208:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298643.346321:0:6463:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417540 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.346968:0:6463:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b03480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6463:1866777604942208:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298643.346979:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298643.347009:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b5f450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609474432:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 861us (979us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298643.347017:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15083 00000100:00100000:3.0:1780298643.347210:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.347229:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.347246:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609474688 00000100:00100000:1.0:1780298643.347251:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609474688 00000100:00100000:0.0:1780298643.347267:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609474560 00000100:00100000:0.0:1780298643.347272:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609474560 00002000:00010000:1.0:1780298643.347276:0:8362:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4194304->18446744073709551615) 00002000:00100000:1.0:1780298643.347282:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243809c00 with x1866777609474688 ext(4194304->18446744073709551615) 00002000:00010000:0.0:1780298643.347295:0:16972:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (5112819->18446744073709551615) 00002000:00100000:0.0:1780298643.347300:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243808380 with x1866777609474560 ext(5112819->18446744073709551615) 00002000:00010000:1.0:1780298643.347304:0:8362:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf243809c00. 00002000:00010000:0.0:1780298643.347305:0:16972:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf243808380. 00000100:00100000:1.0:1780298643.347334:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6130 00000100:00100000:0.0:1780298643.347342:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6131 00000100:00100000:1.0:1780298643.347346:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243809c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2398:x1866777609474688:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:0.0:1780298643.347352:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243808380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609474560:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:1.0:1780298643.347476:0:8362:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 0 -> 4194304 00002000:00010000:1.0:1780298643.347481:0:8362:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298643.347483:0:8362:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298643.347486:0:8362:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298643.347489:0:8362:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 0 -> 0 00002000:00010000:0.0:1780298643.347497:0:16972:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 225671 -> 5112819 00002000:00010000:0.0:1780298643.347501:0:16972:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:0.0:1780298643.347504:0:16972:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:0.0:1780298643.347507:0:16972:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:0.0:1780298643.347510:0:16972:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 304 -> 304 00000100:00100000:1.0:1780298643.347523:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243809c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2398:x1866777609474688:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 180us (310us total) trans 4294969226 rc 0/0 00000100:00100000:1.0:1780298643.347533:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6130 00002000:00010000:1.0:1780298643.347537:0:8362:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4194304->18446744073709551615) 00000100:00100000:0.0:1780298643.347541:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243808380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609474560:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 191us (308us total) trans 4294969670 rc 0/0 00002000:00100000:1.0:1780298643.347543:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243809c00 with x1866777609474688 ext(4194304->18446744073709551615) 00002000:00010000:1.0:1780298643.347553:0:8362:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf243809c00. 00000100:00100000:0.0:1780298643.347557:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6131 00002000:00010000:0.0:1780298643.347562:0:16972:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (5112819->18446744073709551615) 00002000:00100000:0.0:1780298643.347565:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243808380 with x1866777609474560 ext(5112819->18446744073709551615) 00002000:00010000:0.0:1780298643.347570:0:16972:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf243808380. 00000100:00100000:2.0:1780298643.348713:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.348747:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609474816 00000100:00100000:0.0:1780298643.348752:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609474816 00002000:00100000:0.0:1780298643.348777:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (5111808->5115903) req@ffff9cf37f746680 x1866777609474816/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.348787:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f746680 with x1866777609474816 ext(5111808->5115903) 00002000:00010000:0.0:1780298643.348791:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f746680 00000100:00100000:0.0:1780298643.348815:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6132 00000100:00100000:0.0:1780298643.348822:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f746680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609474816:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298643.348942:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f746680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609474816:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 121us (230us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298643.348947:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6132 00002000:00100000:0.0:1780298643.348952:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (5111808->5115903) req@ffff9cf37f746680 x1866777609474816/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.348958:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f746680 with x1866777609474816 ext(5111808->5115903) 00002000:00010000:0.0:1780298643.348962:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f746680 00000100:00100000:3.0:1780298643.350275:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.350350:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609474944 00000100:00100000:1.0:1780298643.350355:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609474944 00002000:00100000:1.0:1780298643.350377:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (5107712->5111807) req@ffff9cf24380a680 x1866777609474944/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.350387:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24380a680 with x1866777609474944 ext(5107712->5111807) 00002000:00010000:1.0:1780298643.350395:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24380a680 00000100:00100000:1.0:1780298643.350416:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6133 00000100:00100000:1.0:1780298643.350424:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24380a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609474944:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298643.350526:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24380a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609474944:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 103us (251us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298643.350533:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6133 00002000:00100000:1.0:1780298643.350539:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (5107712->5111807) req@ffff9cf24380a680 x1866777609474944/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.350549:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24380a680 with x1866777609474944 ext(5107712->5111807) 00002000:00010000:1.0:1780298643.350553:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24380a680 00000100:00100000:3.0:1780298643.351537:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.351580:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609475072 00000100:00100000:2.0:1780298643.351585:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609475072 00002000:00100000:2.0:1780298643.351608:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (5107712->5112818) req@ffff9cf243808a80 x1866777609475072/t0(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 5600/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.351615:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243808a80 with x1866777609475072 ext(5107712->5112818) 00002000:00010000:2.0:1780298643.351618:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf243808a80 00000100:00100000:2.0:1780298643.351635:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6134 00000100:00100000:2.0:1780298643.351640:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243808a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609475072:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:2.0:1780298643.352146:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243808a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609475072:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 505us (609us total) trans 4294969671 rc 0/0 00000100:00100000:2.0:1780298643.352154:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6134 00002000:00100000:2.0:1780298643.352159:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (5107712->5112818) req@ffff9cf243808a80 x1866777609475072/t4294969671(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 5600/448 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.352170:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243808a80 with x1866777609475072 ext(5107712->5112818) 00002000:00010000:2.0:1780298643.352173:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf243808a80 00000100:00100000:2.0:1780298643.352935:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.352977:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609475200 00000100:00100000:1.0:1780298643.352982:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609475200 00000100:00100000:1.0:1780298643.353015:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9326 00000100:00100000:1.0:1780298643.353024:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f744000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609475200:12345-192.168.203.25@tcp:16:fsx.0 00000001:00080000:2.0:1780298643.353765:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969671 is committed 00000001:00080000:2.0:1780298643.353775:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969669 is committed 00000100:00100000:1.0:1780298643.353857:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f744000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609475200:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 833us (920us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.353865:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9326 00000100:00100000:3.0:1780298643.354667:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.354711:0:6476:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609475328 00000100:00100000:1.0:1780298643.354715:0:6476:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609475328 00000100:00100000:1.0:1780298643.354745:0:6476:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61979 00000100:00100000:1.0:1780298643.354765:0:6476:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24380a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+139:29360:x1866777609475328:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.354782:0:6476:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.354799:0:6476:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93000/0xaf35132e63033efd lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 72 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3021 expref: -99 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.354872:0:6476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93e00/0xaf35132e63033f04 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 73 type: IBT flags: 0x40000000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.354885:0:6476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93e00/0xaf35132e63033f04 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 73 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.354903:0:6476:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93e00/0xaf35132e63033f04 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 73 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1780298643.354925:0:6476:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 9307123 00000004:00200000:1.0:1780298643.354928:0:6476:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1780298643.354943:0:6476:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93e00/0xaf35132e63033f04 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 73 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.354962:0:6476:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93000/0xaf35132e63033efd lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 73 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3021 expref: 141 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.354973:0:6476:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93e00/0xaf35132e63033f04 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 72 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3021 expref: 140 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.354979:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298643.354981:0:6476:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1780298643.354983:0:6476:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93e00, rc 0) 00000100:00100000:1.0:1780298643.355011:0:6476:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24380a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+140:29360:x1866777609475328:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 251us (344us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.355018:0:6476:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61979 00000100:00100000:1.0:1780298643.356387:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.356423:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609475456 00000100:00100000:3.0:1780298643.356428:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609475456 00002000:00100000:3.0:1780298643.356451:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x2c0000401:0x11:0x0] (4886528->4890623) req@ffff9cf36e5dad80 x1866777609475456/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298643.356457:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e5dad80 with x1866777609475456 ext(4886528->4890623) 00002000:00010000:3.0:1780298643.356461:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf36e5dad80 00000100:00100000:3.0:1780298643.356478:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6135 00000100:00100000:3.0:1780298643.356485:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5dad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609475456:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298643.356603:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5dad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609475456:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 118us (216us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1780298643.356608:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6135 00002000:00100000:3.0:1780298643.356612:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x2c0000401:0x11:0x0] (4886528->4890623) req@ffff9cf36e5dad80 x1866777609475456/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298643.356617:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e5dad80 with x1866777609475456 ext(4886528->4890623) 00002000:00010000:3.0:1780298643.356620:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf36e5dad80 00000100:00100000:2.0:1780298643.357592:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.357628:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609475584 00000100:00100000:0.0:1780298643.357632:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609475584 00002000:00100000:0.0:1780298643.357650:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (4890624->4894719) req@ffff9cf37f745500 x1866777609475584/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.357656:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f745500 with x1866777609475584 ext(4890624->4894719) 00002000:00010000:0.0:1780298643.357659:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f745500 00000100:00100000:0.0:1780298643.357678:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6136 00000100:00100000:0.0:1780298643.357683:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609475584:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298643.357753:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609475584:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 69us (161us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298643.357758:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6136 00002000:00100000:0.0:1780298643.357762:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (4890624->4894719) req@ffff9cf37f745500 x1866777609475584/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.357767:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f745500 with x1866777609475584 ext(4890624->4894719) 00002000:00010000:0.0:1780298643.357770:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f745500 00000100:00100000:0.0:1780298643.358578:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.358618:0:9976:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609475712 00000100:00100000:2.0:1780298643.358622:0:9976:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609475712 00000100:00100000:2.0:1780298643.358645:0:9976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61980 00000100:00100000:2.0:1780298643.358650:0:9976:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b03480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+108:29360:x1866777609475712:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.358659:0:9976:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.358670:0:9976:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2a00/0xaf35132e63033f0b lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 73 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3028 expref: -99 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.358732:0:9976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2400/0xaf35132e63033f12 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 74 type: IBT flags: 0x40000000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.358741:0:9976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2400/0xaf35132e63033f12 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 74 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.358748:0:9976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2400/0xaf35132e63033f12 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 74 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298643.358762:0:9976:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 9307123 00000004:00200000:2.0:1780298643.358765:0:9976:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298643.358774:0:9976:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2400/0xaf35132e63033f12 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 74 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.358792:0:9976:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2a00/0xaf35132e63033f0b lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 74 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3028 expref: 110 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.358804:0:9976:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2400/0xaf35132e63033f12 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 73 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3028 expref: 109 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.358810:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298643.358811:0:9976:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298643.358813:0:9976:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2400, rc 0) 00000100:00100000:2.0:1780298643.358835:0:9976:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b03480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+109:29360:x1866777609475712:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 185us (258us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.358839:0:9976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61980 00000100:00100000:1.0:1780298643.359613:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.359647:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609475840 00000100:00100000:2.0:1780298643.359651:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609475840 00000100:00100000:2.0:1780298643.359680:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9327 00000100:00100000:2.0:1780298643.359688:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609475840:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.359696:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.359707:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e63033f19 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d302f expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.359719:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.359754:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609475968 00000100:00100000:2.0:1780298643.359755:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f746300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604942336:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.359761:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609475968 00000100:00100000:1.0:1780298643.359798:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9328 00000100:00100000:1.0:1780298643.359808:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b00a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609475968:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.359818:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298643.359819:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00010000:00010000:1.0:1780298643.359830:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b87f9a7e/0xaf35132e63033f20 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3036 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.359867:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604942464:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.359906:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a840 going to sleep for 11 seconds 00002000:00010000:2.0:1780298643.360591:0:8358:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks: 304 -> 320 00000100:00100000:2.0:1780298643.360602:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f746300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604942336:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.360619:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e63033f19 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d302f expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.360624:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604942464:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.360641:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e63033f19 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d302f expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.360648:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3c00, rc 0) 00010000:00010000:1.0:1780298643.360650:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b87f9a7e/0xaf35132e63033f20 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3036 expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.360666:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b87f9a7e/0xaf35132e63033f20 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3036 expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.360673:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609475840:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 986us (1061us total) trans 0 rc 301/301 00010000:00010000:1.0:1780298643.360679:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93600, rc 0) 00000100:00100000:2.0:1780298643.360681:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9327 00000100:00100000:1.0:1780298643.360708:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b00a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609475968:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 903us (989us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.360718:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9328 00000100:00100000:2.0:1780298643.361594:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.361639:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609476096 00000100:00100000:0.0:1780298643.361644:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609476096 00000100:00100000:0.0:1780298643.361677:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9329 00000100:00100000:0.0:1780298643.361686:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609476096:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298643.361694:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:0.0:1780298643.361695:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.361705:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63033f27 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d303d expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.361734:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609476224 00000100:00100000:0.0:1780298643.361735:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b03b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604942592:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.361741:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609476224 00000100:00100000:1.0:1780298643.361778:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9330 00000100:00100000:1.0:1780298643.361790:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243809880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609476224:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.361798:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298643.361799:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00010000:00010000:1.0:1780298643.361809:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63033f2e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3044 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.361837:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604942720:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.361873:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.362542:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604942720:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.362562:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63033f2e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3044 expref: 12 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.362576:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63033f2e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3044 expref: 12 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.362586:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93000, rc 0) 00000100:00100000:0.0:1780298643.362594:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b03b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604942592:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.362610:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63033f27 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d303d expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.362619:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243809880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609476224:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 832us (924us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.362631:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9330 00010000:00010000:0.0:1780298643.362635:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63033f27 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d303d expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.362647:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baee00, rc 0) 00000100:00100000:0.0:1780298643.362668:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609476096:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 984us (1076us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.362684:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9329 00000100:00100000:2.0:1780298643.363537:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.363568:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609476352 00000100:00100000:0.0:1780298643.363571:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609476352 00000100:00100000:0.0:1780298643.363594:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9331 00000100:00100000:0.0:1780298643.363601:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f747100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:29360:x1866777609476352:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.363609:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.363616:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63033f35 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d304b expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.363632:0:19992:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63033ef6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d301a expref: 8 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.363638:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(142 177) 00010000:00010000:0.0:1780298643.363641:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(143 177) 00010000:00010000:0.0:1780298643.363645:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63033f35 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1581056->1589247] (req 1581056->1589247) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d304b expref: 7 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.363658:0:19992:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63033ef6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d301a expref: 8 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.363667:0:19992:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63033ef6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d301a expref: 8 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.363675:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b03b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604942848:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298643.363701:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.364259:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b03b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604942848:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298643.364271:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63033f35 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1581056->1589247] (req 1581056->1589247) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d304b expref: 7 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.364279:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf200, rc 0) 00000100:00100000:0.0:1780298643.364309:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f747100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609476352:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 709us (771us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298643.364315:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9331 00000100:00100000:3.0:1780298643.364484:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.364514:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609476480 00000100:00100000:2.0:1780298643.364518:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609476480 00010000:00010000:2.0:1780298643.364533:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63033ef6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d301a expref: 9 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.364551:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15084 00000100:00100000:2.0:1780298643.364556:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243808000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609476480:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298643.364563:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298643.364571:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63033ef6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d301a expref: 9 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.364579:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63033ef6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d301a expref: 9 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.364591:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63033ef6 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d301a expref: 9 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.364599:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(143 177) 00010000:00010000:2.0:1780298643.364601:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(143 177) 00010000:00010000:2.0:1780298643.364606:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63033f35 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1581056->1589247) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d304b expref: 6 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.364611:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63033f35 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1581056->1589247) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d304b expref: 6 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.364626:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63033f35 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1581056->1589247) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d304b expref: 6 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.364632:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f744700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777604942976:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298643.364658:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.365277:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f744700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777604942976:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298643.365296:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298643.365319:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243808000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2400:x1866777609476480:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 763us (837us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298643.365324:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15084 00000100:00100000:3.0:1780298643.365506:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.365537:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609476608 00000100:00100000:0.0:1780298643.365540:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609476608 00002000:00100000:0.0:1780298643.365557:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (1581056->1589247) req@ffff9cf255488000 x1866777609476608/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.365564:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255488000 with x1866777609476608 ext(1581056->1589247) 00002000:00010000:0.0:1780298643.365567:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255488000 00000100:00100000:0.0:1780298643.365583:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6137 00000100:00100000:0.0:1780298643.365590:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255488000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609476608:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298643.365710:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255488000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609476608:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 120us (205us total) trans 0 rc 8192/8192 00000100:00100000:0.0:1780298643.365717:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6137 00002000:00100000:0.0:1780298643.365723:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (1581056->1589247) req@ffff9cf255488000 x1866777609476608/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/8632 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 8192/8192 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.365731:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255488000 with x1866777609476608 ext(1581056->1589247) 00002000:00010000:0.0:1780298643.365734:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255488000 00000100:00100000:3.0:1780298643.366596:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.366632:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609476736 00000100:00100000:1.0:1780298643.366637:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609476736 00000100:00100000:1.0:1780298643.366664:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9332 00000100:00100000:1.0:1780298643.366671:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609476736:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.366680:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.366689:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000056a806f/0xaf35132e63033f3c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3052 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.366711:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604943104:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.366740:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.367407:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604943104:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.367422:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000056a806f/0xaf35132e63033f3c lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3052 expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.367436:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000056a806f/0xaf35132e63033f3c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3052 expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.367446:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92800, rc 0) 00000100:00100000:1.0:1780298643.367472:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609476736:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 801us (913us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.367478:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9332 00000100:00100000:3.0:1780298643.368326:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.368354:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609476864 00000100:00100000:2.0:1780298643.368358:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609476864 00000100:00100000:2.0:1780298643.368379:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9333 00000100:00100000:2.0:1780298643.368385:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609476864:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.368396:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.368402:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63033f43 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3059 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.368422:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f745c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604943232:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.368454:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4900 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.369104:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f745c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604943232:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.369116:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63033f43 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3059 expref: 12 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.369142:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63033f43 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3059 expref: 12 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.369149:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3c00, rc 0) 00000100:00100000:2.0:1780298643.369168:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609476864:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 784us (844us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.369172:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9333 00000100:00100000:1.0:1780298643.370084:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.370155:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609476992 00000100:00100000:0.0:1780298643.370161:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609476992 00000100:00100000:0.0:1780298643.370193:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9334 00000100:00100000:0.0:1780298643.370202:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609476992:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.370210:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.370219:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c37b15a2/0xaf35132e63033f4a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3060 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.370249:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b03b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604943360:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.370303:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417e40 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.370950:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b03b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604943360:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.370967:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c37b15a2/0xaf35132e63033f4a lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3060 expref: 12 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.370987:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c37b15a2/0xaf35132e63033f4a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3060 expref: 12 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.371007:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baee00, rc 0) 00000100:00100000:0.0:1780298643.371035:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609476992:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 835us (952us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.371042:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9334 00000100:00100000:3.0:1780298643.371832:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.371871:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609477120 00000100:00100000:0.0:1780298643.371876:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609477120 00000100:00100000:0.0:1780298643.371903:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9335 00000100:00100000:0.0:1780298643.371911:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609477120:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.371920:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.371929:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63033f51 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3067 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.371959:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b03b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604943488:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.371997:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417e40 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.372670:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b03b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604943488:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.372692:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63033f51 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3067 expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.372703:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63033f51 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3067 expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.372739:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf400, rc 0) 00000100:00100000:0.0:1780298643.372769:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609477120:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 859us (939us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.372776:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9335 00000100:00100000:3.0:1780298643.373669:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.373713:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609477248 00000100:00100000:1.0:1780298643.373718:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609477248 00000100:00100000:1.0:1780298643.373755:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9336 00000100:00100000:1.0:1780298643.373764:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e583450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:29360:x1866777609477248:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.373773:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.373785:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e63033f58 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d306e expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.373806:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63033f35 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1581056->1589247) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d304b expref: 6 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.373815:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(143 177) 00010000:00010000:1.0:1780298643.373819:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(143 177) 00010000:00010000:1.0:1780298643.373825:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e63033f58 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [5013504->5025791] (req 5013504->5025791) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d306e expref: 9 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.373847:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63033f35 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1581056->1589247) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d304b expref: 6 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.373862:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63033f35 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1581056->1589247) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d304b expref: 6 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.373874:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604943616:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298643.373915:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.374530:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604943616:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298643.374546:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e63033f58 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [5013504->5025791] (req 5013504->5025791) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d306e expref: 9 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.374557:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92600, rc 0) 00000100:00100000:1.0:1780298643.374585:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e583450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609477248:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 823us (917us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.374596:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9336 00000100:00100000:2.0:1780298643.374814:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.374851:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609477376 00000100:00100000:0.0:1780298643.374855:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609477376 00010000:00010000:0.0:1780298643.374874:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63033f35 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1581056->1589247) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d304b expref: 7 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.374901:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15085 00000100:00100000:0.0:1780298643.374908:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609477376:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298643.374917:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298643.374930:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63033f35 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1581056->1589247) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d304b expref: 7 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.374941:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63033f35 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1581056->1589247) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d304b expref: 7 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.374957:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63033f35 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1581056->1589247) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d304b expref: 7 pid: 19992 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.374967:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(143 177) 00010000:00010000:0.0:1780298643.374970:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(143 177) 00010000:00010000:0.0:1780298643.374978:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e63033f58 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [5013504->38567935] (req 5013504->5025791) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d306e expref: 8 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.374987:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e63033f58 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [5013504->38567935] (req 5013504->5025791) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d306e expref: 8 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.375009:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e63033f58 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [5013504->38567935] (req 5013504->5025791) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d306e expref: 8 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.375018:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b03b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777604943744:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298643.375059:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417e40 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.375838:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b03b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777604943744:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298643.375847:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298643.375869:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609477376:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 963us (1057us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298643.375875:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15085 00000100:00100000:3.0:1780298643.375977:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.376016:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609477504 00000100:00100000:1.0:1780298643.376020:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609477504 00000100:00100000:1.0:1780298643.376061:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6138 00000100:00100000:1.0:1780298643.376069:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255489c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609477504:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298643.376318:0:19943:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 5112819 -> 5112819 00002000:00010000:1.0:1780298643.376322:0:19943:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298643.376325:0:19943:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298643.376328:0:19943:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298643.376330:0:19943:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 320 -> 320 00000100:00100000:1.0:1780298643.376358:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255489c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609477504:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 290us (381us total) trans 4294969672 rc 0/0 00000100:00100000:1.0:1780298643.376366:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6138 00000100:00100000:2.0:1780298643.377194:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.377269:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609477632 00000100:00100000:3.0:1780298643.377282:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609477632 00000100:00100000:0.0:1780298643.377304:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.377342:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609477760 00000100:00100000:2.0:1780298643.377347:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609477760 00000100:00100000:3.0:1780298643.377363:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9337 00000100:00100000:3.0:1780298643.377373:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609477632:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.377383:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298643.377390:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9338 00010000:00010000:3.0:1780298643.377393:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d828f4f8/0xaf35132e63033f5f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3075 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.377402:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b03b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609477760:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.377413:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.377421:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63033f66 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d307c expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.377437:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604943872:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.377448:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b00000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604944000:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.377487:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417e40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.377488:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.378166:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604943872:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.378185:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d828f4f8/0xaf35132e63033f5f lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3075 expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.378199:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d828f4f8/0xaf35132e63033f5f lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3075 expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.378210:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2e00, rc 0) 00000100:00100000:1.0:1780298643.378257:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b00000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604944000:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.378275:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63033f66 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d307c expref: 12 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.378300:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609477632:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 871us (1049us total) trans 0 rc 301/301 00010000:00010000:1.0:1780298643.378301:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63033f66 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d307c expref: 12 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.378316:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9337 00010000:00010000:1.0:1780298643.378320:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae400, rc 0) 00000100:00100000:1.0:1780298643.378351:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b03b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609477760:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 953us (1048us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.378359:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9338 00000100:00100000:0.0:1780298643.379042:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.379085:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609477888 00000100:00100000:1.0:1780298643.379090:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609477888 00000100:00100000:1.0:1780298643.379149:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9339 00000100:00100000:1.0:1780298643.379159:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b03100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609477888:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.379168:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.379176:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e90122d2/0xaf35132e63033f6d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3083 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.379204:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604944128:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.379266:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a840 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.379353:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.379385:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609478016 00000100:00100000:1.0:1780298643.379389:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609478016 00000100:00100000:1.0:1780298643.379410:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9340 00000100:00100000:1.0:1780298643.379415:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609478016:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.379420:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.379426:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004cdc8249/0xaf35132e63033f74 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d308a expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.379445:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5db480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604944256:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.379470:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.379882:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604944128:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.379897:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e90122d2/0xaf35132e63033f6d lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3083 expref: 7 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.379911:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e90122d2/0xaf35132e63033f6d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3083 expref: 7 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.379926:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93c00, rc 0) 00000100:00100000:1.0:1780298643.379954:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b03100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609477888:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 798us (912us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.379961:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9339 00000100:00100000:1.0:1780298643.380076:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5db480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604944256:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.380091:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004cdc8249/0xaf35132e63033f74 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d308a expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.380101:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004cdc8249/0xaf35132e63033f74 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d308a expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.380108:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92200, rc 0) 00000100:00100000:1.0:1780298643.380148:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609478016:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 733us (796us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.380154:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9340 00000100:00100000:0.0:1780298643.380955:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.380998:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609478144 00000100:00100000:2.0:1780298643.381004:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609478144 00000100:00100000:2.0:1780298643.381057:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9341 00000100:00100000:3.0:1780298643.381062:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.381073:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b03800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609478144:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298643.381079:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609478272 00010000:00010000:2.0:1780298643.381083:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298643.381083:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609478272 00010000:00010000:2.0:1780298643.381093:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63033f7b lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3091 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.381107:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9342 00000100:00100000:0.0:1780298643.381115:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609478272:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.381121:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.381155:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c37b15a2/0xaf35132e63033f82 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3098 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.381156:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f744000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604944384:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.381193:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b01f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604944512:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.381231:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.381232:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4900 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.381976:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f744000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604944384:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.381997:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63033f7b lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3091 expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.382002:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b01f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604944512:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.382022:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63033f7b lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3091 expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.382033:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c37b15a2/0xaf35132e63033f82 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3098 expref: 12 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.382034:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3000, rc 0) 00010000:00010000:3.0:1780298643.382052:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c37b15a2/0xaf35132e63033f82 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3098 expref: 12 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.382062:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baee00, rc 0) 00000100:00100000:2.0:1780298643.382069:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b03800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609478144:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1001us (1113us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.382077:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9341 00000100:00100000:3.0:1780298643.382092:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609478272:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 978us (1032us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.382100:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9342 00000100:00100000:1.0:1780298643.383208:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.383237:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609478400 00000100:00100000:2.0:1780298643.383242:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609478400 00000100:00100000:2.0:1780298643.383265:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9343 00000100:00100000:2.0:1780298643.383270:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:29360:x1866777609478400:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.383277:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.383293:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e63033f89 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d309f expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.383311:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(143 177) 00010000:00010000:2.0:1780298643.383314:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(143 177) 00010000:00010000:2.0:1780298643.383321:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e63033f89 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1597440->5013503] (req 1597440->1601535) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d309f expref: 7 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.383332:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e63033f89 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1597440->5013503] (req 1597440->1601535) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d309f expref: 7 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.383342:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3c00, rc 0) 00000100:00100000:2.0:1780298643.383363:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609478400:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 94us (158us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.383368:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9343 00000100:00100000:0.0:1780298643.384179:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.384232:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609478528 00000100:00100000:2.0:1780298643.384237:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609478528 00002000:00100000:2.0:1780298643.384266:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (1597440->1601535) req@ffff9cf241b01880 x1866777609478528/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.384277:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf241b01880 with x1866777609478528 ext(1597440->1601535) 00002000:00010000:2.0:1780298643.384282:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf241b01880 00000100:00100000:2.0:1780298643.384312:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6139 00000100:00100000:2.0:1780298643.384321:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b01880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609478528:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298643.384502:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b01880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609478528:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 181us (323us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298643.384510:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6139 00002000:00100000:2.0:1780298643.384517:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (1597440->1601535) req@ffff9cf241b01880 x1866777609478528/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.384526:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf241b01880 with x1866777609478528 ext(1597440->1601535) 00002000:00010000:2.0:1780298643.384529:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf241b01880 00000100:00100000:0.0:1780298643.385612:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.385658:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609478656 00000100:00100000:3.0:1780298643.385663:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609478656 00002000:00100000:3.0:1780298643.385687:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x2c0000401:0x11:0x0] (1601536->1605631) req@ffff9cf241b01f80 x1866777609478656/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298643.385697:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf241b01f80 with x1866777609478656 ext(1601536->1605631) 00002000:00010000:3.0:1780298643.385700:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf241b01f80 00000100:00100000:3.0:1780298643.385722:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6140 00000100:00100000:3.0:1780298643.385728:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b01f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609478656:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298643.385804:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b01f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609478656:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 77us (194us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1780298643.385809:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6140 00002000:00100000:3.0:1780298643.385814:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x2c0000401:0x11:0x0] (1601536->1605631) req@ffff9cf241b01f80 x1866777609478656/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298643.385820:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf241b01f80 with x1866777609478656 ext(1601536->1605631) 00002000:00010000:3.0:1780298643.385823:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf241b01f80 00000100:00100000:1.0:1780298643.386771:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.386813:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609478784 00000100:00100000:0.0:1780298643.386818:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609478784 00000100:00100000:0.0:1780298643.386849:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9344 00000100:00100000:0.0:1780298643.386857:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609478784:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.386865:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.386874:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e63033f90 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d30a6 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.386913:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b00000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604944640:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.386972:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417d80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.387622:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b00000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604944640:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.387640:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e63033f90 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d30a6 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.387651:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e63033f90 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d30a6 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.387664:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf600, rc 0) 00000100:00100000:1.0:1780298643.387690:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609478784:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 834us (919us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.387697:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9344 00000100:00100000:3.0:1780298643.388449:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.388485:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609478912 00000100:00100000:2.0:1780298643.388490:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609478912 00000100:00100000:2.0:1780298643.388519:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9345 00000100:00100000:2.0:1780298643.388526:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255489500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609478912:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.388534:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.388541:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e63033f97 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d30ad expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.388565:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f744000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604944768:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.388597:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.389205:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f744000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604944768:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.389220:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e63033f97 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d30ad expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.389233:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e63033f97 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d30ad expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.389242:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2800, rc 0) 00000100:00100000:2.0:1780298643.389268:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255489500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609478912:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 743us (820us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.389275:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9345 00000100:00100000:0.0:1780298643.390199:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.390225:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.390244:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609479040 00000100:00100000:3.0:1780298643.390248:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609479040 00000100:00100000:1.0:1780298643.390260:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609479168 00000100:00100000:1.0:1780298643.390264:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609479168 00000100:00100000:3.0:1780298643.390275:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9346 00000100:00100000:3.0:1780298643.390294:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b00380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609479040:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.390302:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298643.390308:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9347 00010000:00010000:3.0:1780298643.390309:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d828f4f8/0xaf35132e63033f9e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d30b4 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.390320:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b00e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609479168:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.390328:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298643.390336:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255489180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604944896:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.390340:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004cdc8249/0xaf35132e63033fa5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d30bb expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.390370:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604945024:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.390372:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.390396:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.390999:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255489180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604944896:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.391015:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d828f4f8/0xaf35132e63033f9e lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d30b4 expref: 8 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.391026:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d828f4f8/0xaf35132e63033f9e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d30b4 expref: 8 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.391034:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2e00, rc 0) 00000100:00100000:0.0:1780298643.391063:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b00380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609479040:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 781us (864us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.391071:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9346 00000100:00100000:2.0:1780298643.391116:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604945024:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.391151:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004cdc8249/0xaf35132e63033fa5 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d30bb expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.391164:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004cdc8249/0xaf35132e63033fa5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d30bb expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.391178:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92200, rc 0) 00000100:00100000:2.0:1780298643.391203:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b00e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609479168:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 887us (978us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.391208:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9347 00000100:00100000:1.0:1780298643.392048:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.392079:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609479296 00000100:00100000:0.0:1780298643.392084:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609479296 00000100:00100000:0.0:1780298643.392114:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9348 00000100:00100000:0.0:1780298643.392142:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609479296:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.392151:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.392159:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63033fac lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d30c2 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.392175:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e63033f58 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [5013504->38567935] (req 5013504->5025791) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d306e expref: 9 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.392184:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(143 177) 00010000:00010000:0.0:1780298643.392188:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(144 177) 00010000:00010000:0.0:1780298643.392193:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63033fac lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [5062656->5066751] (req 5062656->5066751) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d30c2 expref: 8 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.392220:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e63033f58 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [5013504->38567935] (req 5013504->5025791) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d306e expref: 9 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.392230:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e63033f58 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [5013504->38567935] (req 5013504->5025791) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d306e expref: 9 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.392238:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b00a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604945152:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298643.392276:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417d80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.392877:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b00a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604945152:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298643.392895:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63033fac lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [5062656->5066751] (req 5062656->5066751) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d30c2 expref: 8 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.392906:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf200, rc 0) 00000100:00100000:0.0:1780298643.392928:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:29360:x1866777609479296:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 808us (881us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298643.392933:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9348 00000100:00100000:3.0:1780298643.393118:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.393164:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609479424 00000100:00100000:1.0:1780298643.393168:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609479424 00010000:00010000:1.0:1780298643.393185:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e63033f58 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [5013504->38567935] (req 5013504->5025791) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d306e expref: 10 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.393208:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15086 00000100:00100000:1.0:1780298643.393214:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609479424:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298643.393221:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298643.393229:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e63033f58 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [5013504->38567935] (req 5013504->5025791) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d306e expref: 10 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.393238:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e63033f58 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [5013504->38567935] (req 5013504->5025791) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d306e expref: 10 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.393249:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e63033f58 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [5013504->38567935] (req 5013504->5025791) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d306e expref: 10 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.393256:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(144 177) 00010000:00010000:1.0:1780298643.393259:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(144 177) 00010000:00010000:1.0:1780298643.393264:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63033fac lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [5062656->38617087] (req 5062656->5066751) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d30c2 expref: 7 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.393270:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63033fac lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [5062656->38617087] (req 5062656->5066751) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d30c2 expref: 7 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.393301:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63033fac lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [5062656->38617087] (req 5062656->5066751) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d30c2 expref: 7 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.393311:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d8a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777604945280:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298643.393350:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a840 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.393993:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d8a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777604945280:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298643.394002:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298643.394021:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609479424:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 807us (905us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.394027:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15086 00000100:00100000:2.0:1780298643.394273:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.394317:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609479552 00000100:00100000:0.0:1780298643.394321:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609479552 00002000:00100000:0.0:1780298643.394340:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (5062656->5066751) req@ffff9cf37f744000 x1866777609479552/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.394347:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f744000 with x1866777609479552 ext(5062656->5066751) 00002000:00010000:0.0:1780298643.394350:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f744000 00000100:00100000:0.0:1780298643.394367:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6141 00000100:00100000:0.0:1780298643.394372:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f744000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609479552:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298643.394491:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f744000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609479552:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 118us (219us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298643.394496:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6141 00002000:00100000:0.0:1780298643.394500:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (5062656->5066751) req@ffff9cf37f744000 x1866777609479552/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.394506:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f744000 with x1866777609479552 ext(5062656->5066751) 00002000:00010000:0.0:1780298643.394508:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f744000 00000100:00100000:3.0:1780298643.395724:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.395773:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609479680 00000100:00100000:0.0:1780298643.395778:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609479680 00002000:00100000:0.0:1780298643.395799:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (5066752->5070847) req@ffff9cf255488380 x1866777609479680/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.395808:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255488380 with x1866777609479680 ext(5066752->5070847) 00002000:00010000:0.0:1780298643.395812:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255488380 00000100:00100000:0.0:1780298643.395839:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6142 00000100:00100000:0.0:1780298643.395847:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255488380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609479680:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298643.395932:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255488380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609479680:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 86us (241us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298643.395940:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6142 00002000:00100000:0.0:1780298643.395947:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (5066752->5070847) req@ffff9cf255488380 x1866777609479680/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.395955:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255488380 with x1866777609479680 ext(5066752->5070847) 00002000:00010000:0.0:1780298643.395959:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255488380 00000100:00100000:1.0:1780298643.397107:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.397180:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609479808 00000100:00100000:0.0:1780298643.397185:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609479808 00002000:00100000:0.0:1780298643.397205:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (5062656->5070847) req@ffff9cf36e5db100 x1866777609479808/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 8680/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.397214:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e5db100 with x1866777609479808 ext(5062656->5070847) 00002000:00010000:0.0:1780298643.397218:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf36e5db100 00000100:00100000:0.0:1780298643.397237:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6143 00000100:00100000:0.0:1780298643.397243:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5db100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609479808:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:0.0:1780298643.397769:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5db100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609479808:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 526us (661us total) trans 4294969673 rc 0/0 00000100:00100000:0.0:1780298643.397777:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6143 00002000:00100000:0.0:1780298643.397783:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (5062656->5070847) req@ffff9cf36e5db100 x1866777609479808/t4294969673(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 8680/448 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.397789:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e5db100 with x1866777609479808 ext(5062656->5070847) 00002000:00010000:0.0:1780298643.397792:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf36e5db100 00000100:00100000:2.0:1780298643.398434:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.398468:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609479936 00000100:00100000:3.0:1780298643.398472:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609479936 00000100:00100000:3.0:1780298643.398496:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9349 00000100:00100000:3.0:1780298643.398502:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609479936:12345-192.168.203.25@tcp:16:fsx.0 00000001:00080000:2.0:1780298643.399176:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969673 is committed 00000001:00080000:2.0:1780298643.399184:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969672 is committed 00000100:00100000:3.0:1780298643.399258:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609479936:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 755us (823us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298643.399272:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9349 00000100:00100000:1.0:1780298643.400223:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.400280:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609480064 00000100:00100000:0.0:1780298643.400295:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609480064 00000100:00100000:0.0:1780298643.400334:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9350 00000100:00100000:0.0:1780298643.400343:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609480064:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.400356:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.400373:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e63033fb3 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d30c9 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.400408:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.400430:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b00a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604945408:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.400452:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609480192 00000100:00100000:1.0:1780298643.400457:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609480192 00000100:00100000:1.0:1780298643.400494:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9351 00000100:00100000:1.0:1780298643.400505:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f747100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609480192:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298643.400511:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417e40 going to sleep for 11 seconds 00010000:00010000:1.0:1780298643.400516:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.400529:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb0d13ab/0xaf35132e63033fba lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d30d0 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.400561:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604945536:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.400597:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837af00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.401266:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604945536:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.401302:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d13ab/0xaf35132e63033fba lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d30d0 expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1780298643.401307:0:20252:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks: 320 -> 336 00000100:00100000:1.0:1780298643.401320:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b00a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604945408:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.401327:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb0d13ab/0xaf35132e63033fba lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d30d0 expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.401335:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e63033fb3 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d30c9 expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.401338:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92400, rc 0) 00010000:00010000:1.0:1780298643.401349:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e63033fb3 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d30c9 expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.401359:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf000, rc 0) 00000100:00100000:3.0:1780298643.401370:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f747100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609480192:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 867us (963us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.401380:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609480064:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1038us (1157us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.401382:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9351 00000100:00100000:1.0:1780298643.401391:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9350 00000100:00100000:3.0:1780298643.402426:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.402496:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609480320 00000100:00100000:0.0:1780298643.402502:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609480320 00000100:00100000:3.0:1780298643.402516:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.402559:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609480448 00000100:00100000:0.0:1780298643.402563:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609480448 00000100:00100000:2.0:1780298643.402567:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9352 00000100:00100000:2.0:1780298643.402581:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255489f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609480320:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298643.402582:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9353 00010000:00010000:2.0:1780298643.402593:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298643.402594:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609480448:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.402601:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.402603:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e63033fc1 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d30d7 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.402609:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000670f33f0/0xaf35132e63033fc8 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d30de expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.402636:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f744700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604945664:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.402639:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604945792:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.402686:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a540 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.402687:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.403387:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f744700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604945664:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.403395:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604945792:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.403405:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e63033fc1 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d30d7 expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.403413:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000670f33f0/0xaf35132e63033fc8 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d30de expref: 12 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.403420:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e63033fc1 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d30d7 expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.403429:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000670f33f0/0xaf35132e63033fc8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d30de expref: 12 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.403437:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92e00, rc 0) 00010000:00010000:1.0:1780298643.403447:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2800, rc 0) 00000100:00100000:3.0:1780298643.403467:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609480448:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 876us (952us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.403472:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255489f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609480320:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 896us (1047us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.403478:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9353 00000100:00100000:1.0:1780298643.403484:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9352 00000100:00100000:0.0:1780298643.404733:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.404753:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.404771:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609480576 00000100:00100000:3.0:1780298643.404777:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609480576 00000100:00100000:1.0:1780298643.404782:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609480704 00000100:00100000:1.0:1780298643.404786:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609480704 00000100:00100000:3.0:1780298643.404811:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9354 00000100:00100000:3.0:1780298643.404820:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b01c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609480576:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.404827:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298643.404830:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9355 00010000:00010000:3.0:1780298643.404837:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002c458c77/0xaf35132e63033fcf lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d30e5 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.404840:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b00a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609480704:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.404853:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.404861:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b35ce980/0xaf35132e63033fd6 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d30ec expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.404873:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255488a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604945920:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.404892:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b00000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604946048:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.404920:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856900 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.404923:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.405721:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b00000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604946048:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.405730:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255488a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604945920:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.405736:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b35ce980/0xaf35132e63033fd6 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d30ec expref: 12 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.405748:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b35ce980/0xaf35132e63033fd6 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d30ec expref: 12 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.405755:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bafa00, rc 0) 00010000:00010000:3.0:1780298643.405756:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002c458c77/0xaf35132e63033fcf lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d30e5 expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.405773:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002c458c77/0xaf35132e63033fcf lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d30e5 expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.405782:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b00a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609480704:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 944us (1026us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.405788:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9355 00010000:00010000:3.0:1780298643.405789:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2a00, rc 0) 00000100:00100000:3.0:1780298643.405818:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b01c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609480576:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 999us (1084us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.405825:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9354 00000100:00100000:2.0:1780298643.406933:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.406961:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609480832 00000100:00100000:3.0:1780298643.406966:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609480832 00000100:00100000:3.0:1780298643.406990:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9356 00000100:00100000:3.0:1780298643.406995:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:29360:x1866777609480832:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.407003:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.407011:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003540747e/0xaf35132e63033fdd lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d30f3 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.407031:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e63033f89 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1597440->5013503] (req 1597440->1601535) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d309f expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.407046:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(144 177) 00010000:00010000:3.0:1780298643.407050:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(145 177) 00010000:00010000:3.0:1780298643.407054:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003540747e/0xaf35132e63033fdd lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1605632->1613823] (req 1605632->1613823) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d30f3 expref: 9 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.407070:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e63033f89 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1597440->5013503] (req 1597440->1601535) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d309f expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.407080:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e63033f89 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1597440->5013503] (req 1597440->1601535) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d309f expref: 9 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.407088:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255488a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604946176:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298643.407120:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856900 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.407836:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255488a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604946176:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298643.407853:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003540747e/0xaf35132e63033fdd lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1605632->1613823] (req 1605632->1613823) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d30f3 expref: 9 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.407866:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3c00, rc 0) 00000100:00100000:3.0:1780298643.407894:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:29360:x1866777609480832:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 898us (962us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298643.407901:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9356 00000100:00100000:0.0:1780298643.408143:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.408180:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609480960 00000100:00100000:3.0:1780298643.408185:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609480960 00010000:00010000:3.0:1780298643.408204:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e63033f89 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1597440->5013503] (req 1597440->1601535) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d309f expref: 9 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.408231:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15087 00000100:00100000:3.0:1780298643.408238:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b00000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609480960:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298643.408247:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298643.408259:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e63033f89 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1597440->5013503] (req 1597440->1601535) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d309f expref: 9 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.408270:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e63033f89 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1597440->5013503] (req 1597440->1601535) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d309f expref: 9 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.408299:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e63033f89 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1597440->5013503] (req 1597440->1601535) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d309f expref: 9 pid: 8356 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.408309:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(145 177) 00010000:00010000:3.0:1780298643.408313:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(145 177) 00010000:00010000:3.0:1780298643.408321:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000003540747e/0xaf35132e63033fdd lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1605632->5062655] (req 1605632->1613823) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d30f3 expref: 8 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.408330:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003540747e/0xaf35132e63033fdd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1605632->5062655] (req 1605632->1613823) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d30f3 expref: 8 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.408350:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000003540747e/0xaf35132e63033fdd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1605632->5062655] (req 1605632->1613823) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d30f3 expref: 8 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.408358:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255488a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777604946304:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298643.408388:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856900 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.409059:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255488a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777604946304:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298643.409068:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298643.409091:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b00000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609480960:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 855us (968us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298643.409096:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15087 00000100:00100000:0.0:1780298643.409213:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.409249:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609481088 00000100:00100000:2.0:1780298643.409254:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609481088 00000100:00100000:2.0:1780298643.409306:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6144 00000100:00100000:2.0:1780298643.409314:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609481088:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298643.409526:0:19949:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 5112819 -> 5112819 00002000:00010000:2.0:1780298643.409529:0:19949:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:2.0:1780298643.409532:0:19949:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:2.0:1780298643.409534:0:19949:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:2.0:1780298643.409536:0:19949:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 336 -> 352 00000100:00100000:2.0:1780298643.409569:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609481088:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 256us (355us total) trans 4294969674 rc 0/0 00000100:00100000:2.0:1780298643.409575:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6144 00000100:00100000:2.0:1780298643.410705:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.410735:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609481216 00000100:00100000:3.0:1780298643.410741:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609481216 00000100:00100000:3.0:1780298643.410780:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9357 00000100:00100000:3.0:1780298643.410789:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f744700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609481216:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.410801:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.410813:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000aa41763f/0xaf35132e63033fe4 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d30fa expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.410850:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255488a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604946432:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.410895:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856900 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.411647:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255488a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604946432:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.411666:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000aa41763f/0xaf35132e63033fe4 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d30fa expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.411678:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000aa41763f/0xaf35132e63033fe4 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d30fa expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.411686:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3600, rc 0) 00000100:00100000:3.0:1780298643.411710:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f744700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609481216:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 924us (1006us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.411715:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9357 00000100:00100000:0.0:1780298643.412633:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.412671:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609481344 00000100:00100000:2.0:1780298643.412675:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609481344 00000100:00100000:2.0:1780298643.412702:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9358 00000100:00100000:2.0:1780298643.412707:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2400:x1866777609481344:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.412716:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.412724:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e63033feb lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3101 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.412750:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255493800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604946560:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.412785:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.413483:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255493800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604946560:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.413499:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e63033feb lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3101 expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.413510:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e63033feb lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3101 expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.413518:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2a00, rc 0) 00000100:00100000:0.0:1780298643.413546:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2400:x1866777609481344:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 838us (913us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.413553:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9358 00000100:00100000:1.0:1780298643.414931:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.414970:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609481472 00000100:00100000:3.0:1780298643.414978:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609481472 00002000:00100000:3.0:1780298643.415024:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (2957312->2961407) req@ffff9cf36e5da680 x1866777609481472/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298643.415033:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e5da680 with x1866777609481472 ext(2957312->2961407) 00002000:00010000:3.0:1780298643.415038:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf36e5da680 00000100:00100000:3.0:1780298643.415056:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6145 00000100:00100000:3.0:1780298643.415062:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609481472:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298643.415340:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609481472:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 278us (411us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298643.415347:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6145 00002000:00100000:2.0:1780298643.415352:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (2957312->2961407) req@ffff9cf36e5da680 x1866777609481472/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.415359:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e5da680 with x1866777609481472 ext(2957312->2961407) 00002000:00010000:2.0:1780298643.415362:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf36e5da680 00000100:00100000:3.0:1780298643.416618:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.416689:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.416691:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609481600 00000100:00100000:1.0:1780298643.416696:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609481600 00000100:00100000:1.0:1780298643.416730:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609481728 00000100:00100000:1.0:1780298643.416732:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609481728 00000100:00100000:1.0:1780298643.416756:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9359 00000100:00100000:2.0:1780298643.416766:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9360 00000100:00100000:1.0:1780298643.416766:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255488a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609481600:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.416779:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298643.416784:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255489180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609481728:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.416794:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.416794:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e63033ff2 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3108 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.416808:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e63033ff9 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d310f expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.416838:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604946688:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.416848:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255491500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604946816:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.416902:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.416903:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.417835:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604946688:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.417837:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255491500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604946816:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.417866:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e63033ff9 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d310f expref: 12 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.417866:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e63033ff2 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3108 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.417897:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e63033ff9 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d310f expref: 12 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.417897:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e63033ff2 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3108 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.417921:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92600, rc 0) 00010000:00010000:2.0:1780298643.417922:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3600, rc 0) 00000100:00100000:2.0:1780298643.417956:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255489180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609481728:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1177us (1266us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.417956:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255488a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609481600:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1191us (1337us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.417968:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9360 00000100:00100000:1.0:1780298643.417974:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9359 00000100:00100000:0.0:1780298643.419307:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.419342:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.419371:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609481856 00000100:00100000:3.0:1780298643.419383:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609481856 00000100:00100000:1.0:1780298643.419386:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609481984 00000100:00100000:1.0:1780298643.419395:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609481984 00000100:00100000:3.0:1780298643.419450:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9361 00000100:00100000:1.0:1780298643.419463:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9362 00000100:00100000:3.0:1780298643.419471:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609481856:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298643.419481:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609481984:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.419499:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.419499:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.419515:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004cdc8249/0xaf35132e63034000 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d311d expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.419517:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000aa41763f/0xaf35132e63034007 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3116 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.419584:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604946944:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.419592:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604947072:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.419672:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.419673:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856900 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.420632:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604946944:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.420652:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604947072:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.420661:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000aa41763f/0xaf35132e63034007 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3116 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.420671:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004cdc8249/0xaf35132e63034000 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d311d expref: 12 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.420683:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000aa41763f/0xaf35132e63034007 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3116 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.420684:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004cdc8249/0xaf35132e63034000 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d311d expref: 12 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.420706:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92200, rc 0) 00010000:00010000:1.0:1780298643.420708:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3600, rc 0) 00000100:00100000:0.0:1780298643.420739:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609481984:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1269us (1398us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.420748:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9362 00000100:00100000:1.0:1780298643.420749:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609481856:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1286us (1443us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.420759:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9361 00000100:00100000:0.0:1780298643.421958:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.422005:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609482112 00000100:00100000:3.0:1780298643.422012:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609482112 00000100:00100000:3.0:1780298643.422058:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9363 00000100:00100000:3.0:1780298643.422067:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2400:x1866777609482112:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.422081:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.422091:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002c458c77/0xaf35132e6303400e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3124 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.422158:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604947200:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.422249:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.423060:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604947200:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.423081:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002c458c77/0xaf35132e6303400e lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3124 expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.423095:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002c458c77/0xaf35132e6303400e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3124 expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.423104:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2a00, rc 0) 00000100:00100000:0.0:1780298643.423148:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2400:x1866777609482112:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1083us (1191us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.423155:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9363 00000100:00100000:2.0:1780298643.423988:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.424038:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609482240 00000100:00100000:1.0:1780298643.424045:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609482240 00000100:00100000:1.0:1780298643.424085:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9364 00000100:00100000:1.0:1780298643.424095:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255491500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2398:x1866777609482240:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.424109:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.424121:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e63034015 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d312b expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.424191:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604947328:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.424234:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.424950:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604947328:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.424968:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e63034015 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d312b expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.424983:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e63034015 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d312b expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.424993:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92600, rc 0) 00000100:00100000:1.0:1780298643.425022:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255491500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2398:x1866777609482240:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 929us (1034us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.425028:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9364 00000100:00100000:2.0:1780298643.425964:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.426006:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609482368 00000100:00100000:0.0:1780298643.426013:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609482368 00000100:00100000:0.0:1780298643.426052:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9365 00000100:00100000:0.0:1780298643.426060:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255490380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:29360:x1866777609482368:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.426069:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.426077:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e6303401c lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3132 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.426093:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(145 177) 00010000:00010000:0.0:1780298643.426096:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(145 177) 00010000:00010000:0.0:1780298643.426103:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e6303401c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1011712->1605631] (req 1011712->1023999) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3132 expref: 9 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.426115:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e6303401c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1011712->1605631] (req 1011712->1023999) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3132 expref: 9 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.426121:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baee00, rc 0) 00000100:00100000:0.0:1780298643.426158:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255490380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609482368:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 100us (196us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298643.426162:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9365 00000100:00100000:2.0:1780298643.426909:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.426951:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609482496 00000100:00100000:0.0:1780298643.426955:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609482496 00000100:00100000:0.0:1780298643.426993:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6146 00000100:00100000:0.0:1780298643.427001:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255492a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609482496:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298643.427309:0:16972:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 5112819 -> 5112819 00002000:00010000:0.0:1780298643.427314:0:16972:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:0.0:1780298643.427319:0:16972:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:0.0:1780298643.427322:0:16972:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:0.0:1780298643.427324:0:16972:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 352 -> 376 00000100:00100000:0.0:1780298643.427352:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255492a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609482496:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 352us (442us total) trans 4294969675 rc 0/0 00000100:00100000:0.0:1780298643.427360:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6146 00000100:00100000:2.0:1780298643.428558:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.428601:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.428629:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609482624 00000100:00100000:1.0:1780298643.428636:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609482624 00000100:00100000:3.0:1780298643.428673:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609482752 00000100:00100000:3.0:1780298643.428685:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609482752 00000100:00100000:1.0:1780298643.428685:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9366 00000100:00100000:1.0:1780298643.428700:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255492300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609482624:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.428713:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.428728:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c1a1ebef/0xaf35132e63034023 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3139 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.428753:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9367 00000100:00100000:3.0:1780298643.428767:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255493480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609482752:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298643.428783:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604947456:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.428785:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.428808:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e6303402a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3140 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.428860:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604947584:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.428870:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.428929:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.429862:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604947456:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.429891:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c1a1ebef/0xaf35132e63034023 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3139 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.429912:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604947584:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.429915:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c1a1ebef/0xaf35132e63034023 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3139 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.429935:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93a00, rc 0) 00010000:00010000:3.0:1780298643.429953:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e6303402a lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3140 expref: 12 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.429976:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e6303402a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3140 expref: 12 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.429986:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255492300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609482624:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1284us (1423us total) trans 0 rc 301/301 00010000:00010000:3.0:1780298643.429999:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3200, rc 0) 00000100:00100000:1.0:1780298643.430009:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9366 00000100:00100000:3.0:1780298643.430044:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255493480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609482752:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1279us (1440us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.430055:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9367 00000100:00100000:2.0:1780298643.431339:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.431378:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609482880 00000100:00100000:1.0:1780298643.431384:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609482880 00000100:00100000:2.0:1780298643.431385:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.431421:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609483008 00000100:00100000:1.0:1780298643.431426:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609483008 00000100:00100000:0.0:1780298643.431427:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9368 00000100:00100000:0.0:1780298643.431439:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255491f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609482880:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.431448:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.431458:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63034031 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3147 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.431470:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9369 00000100:00100000:1.0:1780298643.431480:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255493100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609483008:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.431490:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298643.431490:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f2300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604947712:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.431503:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63034038 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d314e expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.431539:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604947840:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.431548:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.431573:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a6c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.432367:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f2300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604947712:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.432389:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63034031 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3147 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.432393:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604947840:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.432411:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63034031 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3147 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.432419:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63034038 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d314e expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.432420:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bafa00, rc 0) 00010000:00010000:1.0:1780298643.432442:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63034038 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d314e expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.432453:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255491f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609482880:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1015us (1114us total) trans 0 rc 301/301 00010000:00010000:1.0:1780298643.432454:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93000, rc 0) 00000100:00100000:0.0:1780298643.432463:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9368 00000100:00100000:1.0:1780298643.432483:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255493100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609483008:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1004us (1097us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.432490:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9369 00000100:00100000:2.0:1780298643.433651:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.433688:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609483136 00000100:00100000:0.0:1780298643.433693:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609483136 00000100:00100000:0.0:1780298643.433729:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6147 00000100:00100000:0.0:1780298643.433735:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255490e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609483136:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298643.433879:0:8364:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 5112819 -> 5112819 00002000:00010000:0.0:1780298643.433881:0:8364:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:0.0:1780298643.433883:0:8364:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:0.0:1780298643.433885:0:8364:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:0.0:1780298643.433886:0:8364:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 376 -> 376 00000100:00100000:0.0:1780298643.433912:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255490e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609483136:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 176us (261us total) trans 4294969676 rc 0/0 00000100:00100000:0.0:1780298643.433918:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6147 00000100:00100000:2.0:1780298643.435097:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.435227:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609483264 00000100:00100000:3.0:1780298643.435239:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609483264 00000100:00100000:2.0:1780298643.435248:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.435279:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609483392 00000100:00100000:1.0:1780298643.435294:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609483392 00000100:00100000:2.0:1780298643.435343:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9370 00000100:00100000:1.0:1780298643.435355:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9371 00000100:00100000:2.0:1780298643.435356:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255493800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609483264:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.435370:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298643.435371:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609483392:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.435381:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.435388:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e6303403f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3155 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.435397:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b87f9a7e/0xaf35132e63034046 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d315c expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.435444:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604948096:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.435444:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604947968:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.435510:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.435510:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.436452:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604948096:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.436480:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604947968:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.436489:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e6303403f lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3155 expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.436506:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b87f9a7e/0xaf35132e63034046 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d315c expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.436511:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e6303403f lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3155 expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.436540:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2800, rc 0) 00010000:00010000:1.0:1780298643.436540:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b87f9a7e/0xaf35132e63034046 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d315c expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.436559:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93600, rc 0) 00000100:00100000:3.0:1780298643.436589:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255493800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609483264:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1237us (1491us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.436593:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609483392:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1224us (1341us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.436609:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9370 00000100:00100000:1.0:1780298643.436623:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9371 00000100:00100000:0.0:1780298643.437734:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.437790:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609483520 00000100:00100000:2.0:1780298643.437797:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609483520 00000100:00100000:2.0:1780298643.437840:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9372 00000100:00100000:2.0:1780298643.437851:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609483520:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.437866:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298643.437867:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:2.0:1780298643.437880:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e6303404d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3163 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.437905:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609483648 00000100:00100000:1.0:1780298643.437911:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609483648 00000100:00100000:2.0:1780298643.437930:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7be680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604948224:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.437968:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9373 00000100:00100000:1.0:1780298643.437985:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609483648:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.437997:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.438010:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e90122d2/0xaf35132e63034054 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d316a expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.438015:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.438057:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604948352:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.438103:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a6c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.438794:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7be680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604948224:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.438817:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e6303404d lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3163 expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.438834:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e6303404d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3163 expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.438847:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2000, rc 0) 00000100:00100000:2.0:1780298643.438895:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609483520:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1045us (1159us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.438904:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9372 00000100:00100000:1.0:1780298643.438967:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604948352:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.438988:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e90122d2/0xaf35132e63034054 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d316a expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.439007:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e90122d2/0xaf35132e63034054 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d316a expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.439018:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93c00, rc 0) 00000100:00100000:1.0:1780298643.439052:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609483648:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1073us (1188us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.439063:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9373 00000100:00100000:0.0:1780298643.440044:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.440099:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609483776 00000100:00100000:2.0:1780298643.440107:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609483776 00000100:00100000:3.0:1780298643.440166:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.440216:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609483904 00000100:00100000:2.0:1780298643.440221:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609483904 00000100:00100000:1.0:1780298643.440237:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9374 00000100:00100000:1.0:1780298643.440248:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609483776:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.440257:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298643.440268:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9375 00010000:00010000:1.0:1780298643.440271:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000056a806f/0xaf35132e6303405b lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3171 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.440293:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609483904:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.440305:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.440320:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e63034062 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3178 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.440321:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604948480:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.440375:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7be680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604948608:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.440377:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a6c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.440423:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.441219:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604948480:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.441223:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7be680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604948608:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.441245:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000056a806f/0xaf35132e6303405b lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3171 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.441256:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e63034062 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3178 expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.441259:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000056a806f/0xaf35132e6303405b lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3171 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.441299:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92800, rc 0) 00010000:00010000:2.0:1780298643.441311:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e63034062 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3178 expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.441326:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2600, rc 0) 00000100:00100000:1.0:1780298643.441332:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609483776:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1086us (1289us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.441338:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9374 00000100:00100000:2.0:1780298643.441366:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609483904:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1086us (1201us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.441373:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9375 00000100:00100000:3.0:1780298643.442382:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.442426:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609484032 00000100:00100000:2.0:1780298643.442430:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609484032 00000100:00100000:2.0:1780298643.442475:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9376 00000100:00100000:2.0:1780298643.442482:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609484032:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.442492:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.442503:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63034069 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d317f expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.442536:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7be680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604948736:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.442591:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.442657:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.442694:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609484160 00000100:00100000:1.0:1780298643.442700:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609484160 00000100:00100000:1.0:1780298643.442733:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9377 00000100:00100000:1.0:1780298643.442738:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609484160:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.442747:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.442757:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c1a1ebef/0xaf35132e63034070 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3186 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.442785:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604948864:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.442821:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a6c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.443350:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7be680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604948736:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.443369:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63034069 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d317f expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.443383:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63034069 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d317f expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.443392:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3000, rc 0) 00000100:00100000:2.0:1780298643.443425:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609484032:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 944us (1048us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.443433:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9376 00000100:00100000:1.0:1780298643.443572:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604948864:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.443591:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c1a1ebef/0xaf35132e63034070 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3186 expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.443607:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c1a1ebef/0xaf35132e63034070 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3186 expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.443626:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93a00, rc 0) 00000100:00100000:1.0:1780298643.443657:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609484160:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 919us (1001us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.443663:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9377 00000100:00100000:3.0:1780298643.444837:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.444884:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609484288 00000100:00100000:1.0:1780298643.444891:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609484288 00000100:00100000:1.0:1780298643.444946:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9378 00000100:00100000:1.0:1780298643.444956:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609484288:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.444967:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.444978:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e63034077 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d318d expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.445033:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604948992:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.445094:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.445952:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604948992:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.445967:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e63034077 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d318d expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.445976:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e63034077 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d318d expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.445984:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92600, rc 0) 00000100:00100000:1.0:1780298643.446011:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609484288:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1059us (1179us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.446016:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9378 00000100:00100000:0.0:1780298643.447041:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.447083:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609484416 00000100:00100000:2.0:1780298643.447090:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609484416 00000100:00100000:2.0:1780298643.447181:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9379 00000100:00100000:2.0:1780298643.447193:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609484416:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.447208:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.447222:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e6303407e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3194 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.447277:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7be680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604949120:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.447352:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.448081:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7be680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604949120:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.448102:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e6303407e lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3194 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.448117:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e6303407e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3194 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.448144:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2600, rc 0) 00000100:00100000:2.0:1780298643.448180:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609484416:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 991us (1143us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.448187:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9379 00000100:00100000:3.0:1780298643.449110:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.449163:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609484544 00000100:00100000:0.0:1780298643.449169:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609484544 00000100:00100000:0.0:1780298643.449202:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9380 00000100:00100000:0.0:1780298643.449208:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:29360:x1866777609484544:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.449217:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.449225:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000058e2d0/0xaf35132e63034085 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d319b expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.449243:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(145 177) 00010000:00010000:0.0:1780298643.449246:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(145 177) 00010000:00010000:0.0:1780298643.449254:0:20252:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000058e2d0/0xaf35132e63034085 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [790528->1011711] (req 790528->794623) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d319b expref: 11 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.449264:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000058e2d0/0xaf35132e63034085 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [790528->1011711] (req 790528->794623) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d319b expref: 11 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.449270:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baea00, rc 0) 00000100:00100000:0.0:1780298643.449302:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:29360:x1866777609484544:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 96us (194us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298643.449307:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9380 00000100:00100000:0.0:1780298643.450004:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.450049:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609484672 00000100:00100000:1.0:1780298643.450057:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609484672 00002000:00100000:1.0:1780298643.450091:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (790528->794623) req@ffff9cf2425f0a80 x1866777609484672/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.450099:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2425f0a80 with x1866777609484672 ext(790528->794623) 00002000:00010000:1.0:1780298643.450103:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2425f0a80 00000100:00100000:1.0:1780298643.450152:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6148 00000100:00100000:1.0:1780298643.450164:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609484672:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298643.450341:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609484672:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 181us (337us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298643.450348:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6148 00002000:00100000:1.0:1780298643.450354:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (790528->794623) req@ffff9cf2425f0a80 x1866777609484672/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.450360:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2425f0a80 with x1866777609484672 ext(790528->794623) 00002000:00010000:1.0:1780298643.450362:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2425f0a80 00000100:00100000:3.0:1780298643.451327:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.451401:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609484800 00000100:00100000:1.0:1780298643.451406:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609484800 00000100:00100000:1.0:1780298643.451441:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9381 00000100:00100000:1.0:1780298643.451446:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548bad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609484800:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.451453:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.451463:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e6303408c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d31a2 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.451498:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604949248:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.451548:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a900 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.452374:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604949248:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.452393:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e6303408c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31a2 expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.452405:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e6303408c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31a2 expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.452413:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92600, rc 0) 00000100:00100000:1.0:1780298643.452443:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548bad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609484800:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 996us (1119us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.452450:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9381 00000100:00100000:0.0:1780298643.453370:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.453410:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609484928 00000100:00100000:1.0:1780298643.453415:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609484928 00000100:00100000:1.0:1780298643.453449:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9382 00000100:00100000:1.0:1780298643.453459:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609484928:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.453467:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.453475:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c1a1ebef/0xaf35132e63034093 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d31a9 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.453504:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604949376:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.453548:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a900 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.454335:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604949376:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.454352:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c1a1ebef/0xaf35132e63034093 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31a9 expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.454365:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c1a1ebef/0xaf35132e63034093 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31a9 expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.454375:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93a00, rc 0) 00000100:00100000:1.0:1780298643.454406:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609484928:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 949us (1037us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.454413:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9382 00000100:00100000:3.0:1780298643.455544:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.455568:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.455621:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609485056 00000100:00100000:0.0:1780298643.455625:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609485056 00000100:00100000:0.0:1780298643.455652:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609485184 00000100:00100000:0.0:1780298643.455659:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609485184 00000100:00100000:2.0:1780298643.455683:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9383 00000100:00100000:0.0:1780298643.455692:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9384 00000100:00100000:2.0:1780298643.455701:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e581450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609485056:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298643.455706:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548bb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609485184:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.455718:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.455721:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.455729:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b35ce980/0xaf35132e6303409a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d31b7 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.455731:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e630340a1 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d31b0 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.455768:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604949504:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.455782:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7be680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604949632:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.455813:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.455814:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.456753:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604949504:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.456777:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b35ce980/0xaf35132e6303409a lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31b7 expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.456794:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b35ce980/0xaf35132e6303409a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31b7 expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.456800:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7be680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604949632:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.456818:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bafa00, rc 0) 00010000:00010000:3.0:1780298643.456851:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e630340a1 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31b0 expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.456864:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548bb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609485184:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1158us (1288us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.456891:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9384 00010000:00010000:3.0:1780298643.456906:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e630340a1 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31b0 expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.456922:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2600, rc 0) 00000100:00100000:3.0:1780298643.456963:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e581450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609485056:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1271us (1422us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.456973:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9383 00000100:00100000:3.0:1780298643.458282:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.458335:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609485312 00000100:00100000:1.0:1780298643.458342:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609485312 00000100:00100000:1.0:1780298643.458383:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9385 00000100:00100000:1.0:1780298643.458392:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:29360:x1866777609485312:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.458405:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.458416:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000056a806f/0xaf35132e630340a8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d31be expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.458438:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e63033e2b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f64 expref: 16 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.458449:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(105 130) 00010000:00010000:1.0:1780298643.458453:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(106 130) 00010000:00010000:1.0:1780298643.458459:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000056a806f/0xaf35132e630340a8 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3174400->3178495] (req 3174400->3178495) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31be expref: 12 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.458489:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e63033e2b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f64 expref: 16 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.458503:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e63033e2b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f64 expref: 16 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.458516:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604949760:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298643.458565:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a900 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.459379:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604949760:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298643.459396:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000056a806f/0xaf35132e630340a8 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3174400->3178495] (req 3174400->3178495) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31be expref: 12 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.459416:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92800, rc 0) 00000100:00100000:1.0:1780298643.459459:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:29360:x1866777609485312:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1069us (1178us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.459465:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9385 00000100:00100000:0.0:1780298643.459733:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.459772:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609485440 00000100:00100000:1.0:1780298643.459776:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609485440 00010000:00010000:1.0:1780298643.459794:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e63033e2b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f64 expref: 17 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.459818:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15088 00000100:00100000:1.0:1780298643.459824:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2399:x1866777609485440:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298643.459832:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298643.459844:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e63033e2b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f64 expref: 17 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.459852:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e63033e2b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f64 expref: 17 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.459866:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e63033e2b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d2f64 expref: 17 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.459875:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(106 130) 00010000:00010000:1.0:1780298643.459877:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(106 130) 00010000:00010000:1.0:1780298643.459882:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000056a806f/0xaf35132e630340a8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3174400->36728831] (req 3174400->3178495) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31be expref: 11 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.459889:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000056a806f/0xaf35132e630340a8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3174400->36728831] (req 3174400->3178495) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31be expref: 11 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.459905:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000056a806f/0xaf35132e630340a8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3174400->36728831] (req 3174400->3178495) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31be expref: 11 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.459913:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:20253:1866777604949888:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298643.459945:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a900 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.460746:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:20253:1866777604949888:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298643.460754:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298643.460783:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2399:x1866777609485440:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 959us (1051us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.460792:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15088 00000100:00100000:0.0:1780298643.461117:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.461188:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609485568 00000100:00100000:2.0:1780298643.461194:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609485568 00002000:00100000:2.0:1780298643.461229:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x12:0x0] (3174400->3178495) req@ffff9cf2425f1500 x1866777609485568/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.461260:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2425f1500 with x1866777609485568 ext(3174400->3178495) 00002000:00010000:2.0:1780298643.461264:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2425f1500 00000100:00100000:2.0:1780298643.461303:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6149 00000100:00100000:2.0:1780298643.461315:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609485568:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298643.461487:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609485568:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 176us (371us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298643.461496:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6149 00002000:00100000:2.0:1780298643.461511:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x12:0x0] (3174400->3178495) req@ffff9cf2425f1500 x1866777609485568/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.461521:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2425f1500 with x1866777609485568 ext(3174400->3178495) 00002000:00010000:2.0:1780298643.461524:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2425f1500 00000100:00100000:0.0:1780298643.462846:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.462889:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609485696 00000100:00100000:2.0:1780298643.462895:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609485696 00002000:00100000:2.0:1780298643.462919:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (3174400->3178495) req@ffff9cf2425f1180 x1866777609485696/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 4584/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.462929:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2425f1180 with x1866777609485696 ext(3174400->3178495) 00002000:00010000:2.0:1780298643.462933:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2425f1180 00000100:00100000:2.0:1780298643.462959:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6150 00000100:00100000:2.0:1780298643.462968:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609485696:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:2.0:1780298643.463551:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2397:x1866777609485696:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 584us (705us total) trans 4294969227 rc 0/0 00000100:00100000:2.0:1780298643.463559:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6150 00002000:00100000:2.0:1780298643.463568:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (3174400->3178495) req@ffff9cf2425f1180 x1866777609485696/t4294969227(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 4584/448 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.463577:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2425f1180 with x1866777609485696 ext(3174400->3178495) 00002000:00010000:2.0:1780298643.463581:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2425f1180 00000100:00100000:3.0:1780298643.464475:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.464518:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609485824 00000100:00100000:1.0:1780298643.464528:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609485824 00000100:00100000:1.0:1780298643.464560:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9386 00000100:00100000:1.0:1780298643.464565:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609485824:12345-192.168.203.25@tcp:16:fsx.0 00000001:00080000:2.0:1780298643.465430:0:8351:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0000: transno 4294969227 is committed 00000001:00080000:2.0:1780298643.465440:0:8351:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0000: transno 4294969226 is committed 00000100:00100000:1.0:1780298643.465537:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609485824:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 971us (1065us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.465543:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9386 00000100:00100000:0.0:1780298643.466658:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.466697:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609485952 00000100:00100000:3.0:1780298643.466706:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609485952 00000100:00100000:0.0:1780298643.466718:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.466765:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609486080 00000100:00100000:2.0:1780298643.466772:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609486080 00000100:00100000:1.0:1780298643.466785:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9387 00000100:00100000:1.0:1780298643.466799:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609485952:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298643.466801:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9388 00000100:00100000:3.0:1780298643.466815:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609486080:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.466817:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.466823:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.466834:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f6fb1b9/0xaf35132e630340b6 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d31cc expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.466834:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e630340af lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d31c5 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.466897:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604950016:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.466906:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548bad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604950144:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.466965:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a900 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.466966:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.467951:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604950016:192.168.203.25@tcp:106: 00002000:00010000:3.0:1780298643.467957:0:8356:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks: 0 -> 8 00000100:00100000:3.0:1780298643.467976:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548bad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604950144:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.467990:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e630340af lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31c5 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.468016:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e630340af lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31c5 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.468020:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002f6fb1b9/0xaf35132e630340b6 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31cc expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.468064:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92600, rc 0) 00010000:00010000:3.0:1780298643.468074:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f6fb1b9/0xaf35132e630340b6 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31cc expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.468087:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2400, rc 0) 00000100:00100000:1.0:1780298643.468111:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609485952:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1314us (1453us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.468116:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609486080:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1303us (1397us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.468167:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9388 00000100:00100000:1.0:1780298643.468185:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9387 00000100:00100000:0.0:1780298643.469470:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.469519:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609486208 00000100:00100000:1.0:1780298643.469526:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609486208 00000100:00100000:0.0:1780298643.469532:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.469560:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609486336 00000100:00100000:1.0:1780298643.469563:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609486336 00000100:00100000:1.0:1780298643.469587:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9389 00000100:00100000:1.0:1780298643.469595:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609486208:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.469604:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298643.469611:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9390 00010000:00010000:1.0:1780298643.469614:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c1a1ebef/0xaf35132e630340bd lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d31d3 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.469630:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609486336:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.469644:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298643.469655:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604950272:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.469660:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e630340c4 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d31da expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.469700:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a900 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.469717:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bf100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604950400:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.469766:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.470385:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604950272:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.470404:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c1a1ebef/0xaf35132e630340bd lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31d3 expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.470419:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c1a1ebef/0xaf35132e630340bd lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31d3 expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.470428:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93a00, rc 0) 00000100:00100000:1.0:1780298643.470471:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609486208:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 876us (1002us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.470478:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9389 00000100:00100000:1.0:1780298643.470588:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bf100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604950400:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.470607:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e630340c4 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31da expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.470621:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e630340c4 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31da expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.470629:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3000, rc 0) 00000100:00100000:1.0:1780298643.470661:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609486336:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1042us (1134us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.470667:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9390 00000100:00100000:0.0:1780298643.471488:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.471532:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609486592 00000100:00100000:1.0:1780298643.471538:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609486592 00000100:00100000:1.0:1780298643.471579:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61981 00000100:00100000:1.0:1780298643.471591:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+140:29360:x1866777609486592:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:1.0:1780298643.471635:0:6477:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298643.471653:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93e00/0xaf35132e63033f04 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 74 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3021 expref: 140 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.471667:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298643.471671:0:6477:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298643.471830:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93c00/0xaf35132e630340cb lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 73 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.471846:0:6477:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2400/0xaf35132e63033f12 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 73 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3028 expref: 108 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.471866:0:6477:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93c00/0xaf35132e630340cb lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 73 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.471892:0:6477:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2400/0xaf35132e63033f12 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 73 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3028 expref: 108 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.471908:0:6477:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2400/0xaf35132e63033f12 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 73 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3028 expref: 108 pid: 9976 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.471920:0:6477:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6477:1866777604950528:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298643.471967:0:6477:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.472646:0:6477:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6477:1866777604950528:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298643.472657:0:6477:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93c00/0xaf35132e630340cb lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 73 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.472853:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.472907:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609486720 00000100:00100000:2.0:1780298643.472913:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609486720 00010000:00010000:2.0:1780298643.472939:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2400/0xaf35132e63033f12 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 73 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3028 expref: 109 pid: 9976 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.472965:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15089 00000100:00100000:2.0:1780298643.472971:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f0700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+109:2399:x1866777609486720:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298643.472985:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298643.472990:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2400/0xaf35132e63033f12 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 74 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3028 expref: 109 pid: 9976 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.473006:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2400/0xaf35132e63033f12 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 74 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3028 expref: 109 pid: 9976 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.473026:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2400/0xaf35132e63033f12 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 74 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3028 expref: 109 pid: 9976 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.473042:0:6463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298643.473047:0:6463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93c00/0xaf35132e630340cb lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 73 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.473066:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93c00/0xaf35132e630340cb lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 73 type: IBT flags: 0x40290000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.473074:0:6463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93c00/0xaf35132e630340cb lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 73 type: IBT flags: 0x40290000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.473101:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298643.473121:0:6477:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93c00/0xaf35132e630340cb lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 72 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.473167:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93c00/0xaf35132e630340cb lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 72 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.473181:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f0700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+108:2399:x1866777609486720:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 208us (327us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.473190:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15089 00000001:00080000:1.0:1780298643.473222:0:6477:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991048 00000004:00010000:1.0:1780298643.473246:0:6477:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000880c645a reply state 00000000c8d360d0 transno 4294991049 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93c00/0xaf35132e630340cb lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 72 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.473258:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93c00/0xaf35132e630340cb lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 72 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.473264:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298643.473266:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1780298643.473268:0:6477:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1780298643.473340:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+142:29360:x1866777609486592:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1751us (1851us total) trans 4294991049 rc 0/0 00000100:00100000:1.0:1780298643.473347:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61981 00000020:00080000:1.0:1780298643.473353:0:6477:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1780298643.473963:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.474006:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609486848 00000100:00100000:2.0:1780298643.474011:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609486848 00000100:00100000:2.0:1780298643.474040:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9391 00000100:00100000:2.0:1780298643.474045:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609486848:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.474053:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.474062:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e630340d2 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d31e1 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.474081:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e6303401c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [1011712->1605631] (req 1011712->1023999) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3132 expref: 11 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.474088:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000003540747e/0xaf35132e63033fdd lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [1605632->5062655] (req 1605632->1613823) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d30f3 expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.474093:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63033fac lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [5062656->38617087] (req 5062656->5066751) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d30c2 expref: 10 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.474097:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000000058e2d0/0xaf35132e63034085 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [790528->1011711] (req 790528->794623) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d319b expref: 10 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.474102:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 4 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(145 177) 00010000:00010000:2.0:1780298643.474105:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(149 177) 00010000:00010000:2.0:1780298643.474108:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e630340d2 lrc: 7/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31e1 expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.474122:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000000058e2d0/0xaf35132e63034085 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [790528->1011711] (req 790528->794623) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d319b expref: 10 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.474152:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000000058e2d0/0xaf35132e63034085 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [790528->1011711] (req 790528->794623) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d319b expref: 10 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.474160:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bf800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604950656:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298643.474199:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63033fac lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [5062656->38617087] (req 5062656->5066751) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d30c2 expref: 10 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.474206:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63033fac lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [5062656->38617087] (req 5062656->5066751) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d30c2 expref: 10 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.474216:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604950784:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298643.474249:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000003540747e/0xaf35132e63033fdd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [1605632->5062655] (req 1605632->1613823) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d30f3 expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.474258:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000003540747e/0xaf35132e63033fdd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [1605632->5062655] (req 1605632->1613823) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d30f3 expref: 10 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.474267:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7be300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604950912:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298643.474315:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e6303401c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [1011712->1605631] (req 1011712->1023999) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3132 expref: 11 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.474325:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e6303401c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [1011712->1605631] (req 1011712->1023999) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3132 expref: 11 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.474334:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604951040:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298643.474358:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b40c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.474738:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bf800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604950656:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298643.475037:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604950784:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298643.475046:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7be300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604950912:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298643.475053:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604951040:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298643.475071:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e630340d2 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31e1 expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.475079:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2a00, rc 0) 00000100:00100000:2.0:1780298643.475101:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:29360:x1866777609486848:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1056us (1140us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.475106:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9391 00000100:00100000:0.0:1780298643.475363:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.475395:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609487232 00000100:00100000:2.0:1780298643.475399:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609487232 00010000:00010000:2.0:1780298643.475413:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000003540747e/0xaf35132e63033fdd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [1605632->5062655] (req 1605632->1613823) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d30f3 expref: 11 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.475431:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15090 00000100:00100000:2.0:1780298643.475437:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b5f850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609487232:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298643.475443:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298643.475455:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000003540747e/0xaf35132e63033fdd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [1605632->5062655] (req 1605632->1613823) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d30f3 expref: 11 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.475462:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000003540747e/0xaf35132e63033fdd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [1605632->5062655] (req 1605632->1613823) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d30f3 expref: 11 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.475475:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003540747e/0xaf35132e63033fdd lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [1605632->5062655] (req 1605632->1613823) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d30f3 expref: 11 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.475487:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298643.475514:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b5f850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609487232:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 73us (147us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.475519:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15090 00000100:00100000:3.0:1780298643.475559:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.475596:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609486976 00000100:00100000:1.0:1780298643.475601:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609486976 00010000:00010000:1.0:1780298643.475619:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000000058e2d0/0xaf35132e63034085 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [790528->1011711] (req 790528->794623) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d319b expref: 10 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.475656:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15091 00000100:00100000:1.0:1780298643.475664:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548bad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609486976:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298643.475673:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298643.475683:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000000058e2d0/0xaf35132e63034085 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [790528->1011711] (req 790528->794623) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d319b expref: 10 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.475693:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000000058e2d0/0xaf35132e63034085 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [790528->1011711] (req 790528->794623) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d319b expref: 10 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.475704:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000058e2d0/0xaf35132e63034085 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [790528->1011711] (req 790528->794623) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d319b expref: 10 pid: 20252 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.475714:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298643.475742:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548bad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609486976:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 78us (183us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298643.475746:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.475749:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15091 00000100:00100000:1.0:1780298643.475763:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609487360 00000100:00100000:1.0:1780298643.475767:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609487360 00010000:00010000:1.0:1780298643.475781:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e6303401c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1011712->1605631] (req 1011712->1023999) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3132 expref: 11 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.475802:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15092 00000100:00100000:1.0:1780298643.475809:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548ba300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609487360:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298643.475818:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298643.475827:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e6303401c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1011712->1605631] (req 1011712->1023999) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3132 expref: 11 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.475839:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:1.0:1780298643.475844:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e6303401c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1011712->1605631] (req 1011712->1023999) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3132 expref: 11 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.475859:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e6303401c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1011712->1605631] (req 1011712->1023999) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3132 expref: 11 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.475870:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298643.475885:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609487104 00000100:00100000:3.0:1780298643.475889:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609487104 00000100:00100000:1.0:1780298643.475894:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548ba300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609487360:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 86us (241us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.475902:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15092 00010000:00010000:3.0:1780298643.475906:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63033fac lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [5062656->38617087] (req 5062656->5066751) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d30c2 expref: 9 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.475929:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15093 00000100:00100000:3.0:1780298643.475938:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f0380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609487104:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298643.475945:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298643.475952:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63033fac lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [5062656->38617087] (req 5062656->5066751) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d30c2 expref: 9 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.475960:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63033fac lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [5062656->38617087] (req 5062656->5066751) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d30c2 expref: 9 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.475968:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63033fac lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [5062656->38617087] (req 5062656->5066751) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d30c2 expref: 9 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.475977:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(149 177) 00010000:00010000:3.0:1780298643.475979:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(149 177) 00010000:00010000:3.0:1780298643.476015:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e630340d2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31e1 expref: 9 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.476021:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e630340d2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31e1 expref: 9 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.476076:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e630340d2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31e1 expref: 9 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.476084:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548b9880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777604951168:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298643.476114:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.476754:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548b9880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777604951168:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298643.476763:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298643.476781:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f0380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609487104:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 847us (948us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298643.476787:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15093 00000100:00100000:0.0:1780298643.476941:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.476980:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609487488 00000100:00100000:1.0:1780298643.476985:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609487488 00000100:00100000:1.0:1780298643.477011:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9392 00000100:00100000:1.0:1780298643.477017:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609487488:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.477027:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.477037:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f2c2b189/0xaf35132e630340d9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d31e8 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.477059:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000056a806f/0xaf35132e630340a8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3174400->36728831] (req 3174400->3178495) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31be expref: 6 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.477067:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(106 130) 00010000:00010000:1.0:1780298643.477071:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(107 130) 00010000:00010000:1.0:1780298643.477075:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f2c2b189/0xaf35132e630340d9 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31e8 expref: 10 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.477093:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000056a806f/0xaf35132e630340a8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3174400->36728831] (req 3174400->3178495) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31be expref: 6 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.477105:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000056a806f/0xaf35132e630340a8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3174400->36728831] (req 3174400->3178495) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31be expref: 6 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.477116:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604951296:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298643.477173:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a900 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.477682:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604951296:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298643.477693:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f2c2b189/0xaf35132e630340d9 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31e8 expref: 10 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.477701:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92600, rc 0) 00000100:00100000:1.0:1780298643.477723:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609487488:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 706us (782us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.477731:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9392 00000100:00100000:0.0:1780298643.477967:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.477998:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609487616 00000100:00100000:2.0:1780298643.478002:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609487616 00010000:00010000:2.0:1780298643.478016:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000056a806f/0xaf35132e630340a8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3174400->36728831] (req 3174400->3178495) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31be expref: 7 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.478038:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15094 00000100:00100000:2.0:1780298643.478043:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609487616:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298643.478053:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298643.478063:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000056a806f/0xaf35132e630340a8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3174400->36728831] (req 3174400->3178495) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31be expref: 7 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.478072:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000056a806f/0xaf35132e630340a8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3174400->36728831] (req 3174400->3178495) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31be expref: 7 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.478087:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000056a806f/0xaf35132e630340a8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3174400->36728831] (req 3174400->3178495) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31be expref: 7 pid: 8358 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.478097:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(107 130) 00010000:00010000:2.0:1780298643.478101:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(107 130) 00010000:00010000:2.0:1780298643.478107:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000f2c2b189/0xaf35132e630340d9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31e8 expref: 9 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.478117:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f2c2b189/0xaf35132e630340d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31e8 expref: 9 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.478151:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000f2c2b189/0xaf35132e630340d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31e8 expref: 9 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.478162:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bdf80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:20253:1866777604951424:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298643.478199:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4900 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.478817:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bdf80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:20253:1866777604951424:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298643.478825:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298643.478844:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609487616:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 802us (879us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.478851:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15094 00000100:00100000:0.0:1780298643.478906:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.478946:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609487744 00000100:00100000:1.0:1780298643.478952:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609487744 00002000:00010000:1.0:1780298643.478973:0:16972:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (5069434->18446744073709551615) 00002000:00100000:1.0:1780298643.478977:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2425f0700 with x1866777609487744 ext(5069434->18446744073709551615) 00002000:00010000:1.0:1780298643.478984:0:16972:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2425f0700. 00000100:00100000:1.0:1780298643.478999:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6151 00000100:00100000:1.0:1780298643.479005:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609487744:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:0.0:1780298643.479067:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.479103:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609487872 00000100:00100000:2.0:1780298643.479108:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609487872 00002000:00010000:2.0:1780298643.479157:0:8364:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4194304->18446744073709551615) 00002000:00100000:2.0:1780298643.479162:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2556a9500 with x1866777609487872 ext(4194304->18446744073709551615) 00002000:00010000:2.0:1780298643.479168:0:8364:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2556a9500. 00000100:00100000:2.0:1780298643.479192:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6152 00000100:00100000:2.0:1780298643.479202:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609487872:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:2.0:1780298643.479306:0:8364:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:2.0:1780298643.479310:0:8364:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00002000:00010000:2.0:1780298643.479313:0:8364:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:2.0:1780298643.479315:0:8364:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:2.0:1780298643.479318:0:8364:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 8 -> 8 00000100:00100000:2.0:1780298643.479350:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609487872:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 150us (282us total) trans 4294969228 rc 0/0 00002000:00010000:1.0:1780298643.479354:0:16972:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 5112819 -> 5069434 00002000:00010000:1.0:1780298643.479359:0:16972:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298643 -> 1780298643 00000100:00100000:2.0:1780298643.479361:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6152 00002000:00010000:1.0:1780298643.479366:0:16972:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298643 -> 1780298643 00002000:00010000:2.0:1780298643.479367:0:8364:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4194304->18446744073709551615) 00002000:00100000:2.0:1780298643.479374:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2556a9500 with x1866777609487872 ext(4194304->18446744073709551615) 00002000:00010000:1.0:1780298643.479374:0:16972:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298643 -> 1780298643 00002000:00010000:1.0:1780298643.479380:0:16972:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 376 -> 360 00002000:00010000:2.0:1780298643.479382:0:8364:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2556a9500. 00000100:00100000:1.0:1780298643.479410:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609487744:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 403us (503us total) trans 4294969677 rc 0/0 00000100:00100000:1.0:1780298643.479419:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6151 00002000:00010000:1.0:1780298643.479422:0:16972:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (5069434->18446744073709551615) 00002000:00100000:1.0:1780298643.479424:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2425f0700 with x1866777609487744 ext(5069434->18446744073709551615) 00002000:00010000:1.0:1780298643.479428:0:16972:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2425f0700. 00000100:00100000:0.0:1780298643.480385:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.480414:0:7411:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609488000 00000100:00100000:2.0:1780298643.480418:0:7411:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609488000 00000100:00100000:2.0:1780298643.480443:0:7411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61982 00000100:00100000:2.0:1780298643.480448:0:7411:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+108:29360:x1866777609488000:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.480457:0:7411:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.480468:0:7411:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3c00/0xaf35132e630340e0 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 73 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d31ef expref: -99 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.480537:0:7411:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3e00/0xaf35132e630340e7 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 74 type: IBT flags: 0x40000000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.480548:0:7411:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3e00/0xaf35132e630340e7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 74 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.480555:0:7411:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3e00/0xaf35132e630340e7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 74 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298643.480571:0:7411:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 9263738 00000004:00200000:2.0:1780298643.480573:0:7411:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298643.480586:0:7411:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3e00/0xaf35132e630340e7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 74 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.480611:0:7411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3c00/0xaf35132e630340e0 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 74 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31ef expref: 110 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.480626:0:7411:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3e00/0xaf35132e630340e7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 73 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31ef expref: 109 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.480633:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298643.480635:0:7411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298643.480637:0:7411:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3e00, rc 0) 00000100:00100000:2.0:1780298643.480659:0:7411:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+109:29360:x1866777609488000:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 212us (276us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.480664:0:7411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61982 00000100:00100000:0.0:1780298643.481686:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.481725:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609488128 00000100:00100000:0.0:1780298643.481728:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.481731:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609488128 00000100:00100000:2.0:1780298643.481766:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609488256 00000100:00100000:2.0:1780298643.481769:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609488256 00000100:00100000:1.0:1780298643.481773:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9393 00000100:00100000:1.0:1780298643.481782:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609488128:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.481788:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.481795:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e630340ee lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d31f6 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.481802:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9394 00000100:00100000:2.0:1780298643.481815:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609488256:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.481823:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298643.481832:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604951552:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.481833:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e630340f5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d31fd expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.481868:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604951680:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.481876:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a900 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.481904:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4900 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.482512:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604951552:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.482531:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e630340ee lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31f6 expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.482546:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e630340ee lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31f6 expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.482557:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92a00, rc 0) 00000100:00100000:1.0:1780298643.482588:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609488128:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 808us (902us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.482596:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9393 00000100:00100000:2.0:1780298643.482761:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604951680:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.482775:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e630340f5 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31fd expref: 7 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.482788:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e630340f5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31fd expref: 7 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.482796:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3600, rc 0) 00000100:00100000:2.0:1780298643.482821:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609488256:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1010us (1095us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.482827:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9394 00000100:00100000:3.0:1780298643.483960:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.483995:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.484013:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609488384 00000100:00100000:2.0:1780298643.484018:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609488384 00000100:00100000:2.0:1780298643.484048:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609488512 00000100:00100000:2.0:1780298643.484052:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609488512 00000100:00100000:1.0:1780298643.484056:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9395 00000100:00100000:3.0:1780298643.484068:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9396 00000100:00100000:1.0:1780298643.484070:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609488384:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.484080:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298643.484084:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609488512:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.484090:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e630340fc lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3204 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.484094:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.484106:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f6fb1b9/0xaf35132e63034103 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d320b expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.484122:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604951808:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.484191:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604951936:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.484203:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a900 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.484219:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4900 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.484826:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604951808:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.484847:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e630340fc lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3204 expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.484864:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e630340fc lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3204 expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.484874:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93600, rc 0) 00000100:00100000:1.0:1780298643.484908:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609488384:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 842us (948us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.484916:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9395 00000100:00100000:2.0:1780298643.484933:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604951936:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.484947:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002f6fb1b9/0xaf35132e63034103 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d320b expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.484964:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f6fb1b9/0xaf35132e63034103 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d320b expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.484971:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2400, rc 0) 00000100:00100000:2.0:1780298643.484993:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609488512:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 918us (999us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.484998:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9396 00000100:00100000:0.0:1780298643.486257:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.486296:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609488640 00000100:00100000:3.0:1780298643.486302:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609488640 00000100:00100000:3.0:1780298643.486336:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9397 00000100:00100000:3.0:1780298643.486344:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556abb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:29360:x1866777609488640:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.486353:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.486363:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000075e99a4b/0xaf35132e6303410a lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3212 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.486383:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e630340d2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31e1 expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.486392:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(149 177) 00010000:00010000:3.0:1780298643.486396:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(150 177) 00010000:00010000:3.0:1780298643.486401:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000075e99a4b/0xaf35132e6303410a lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2170880->2174975] (req 2170880->2174975) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3212 expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.486423:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e630340d2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31e1 expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.486437:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e630340d2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31e1 expref: 10 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.486448:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548b9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604952064:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298643.486491:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.487085:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548b9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604952064:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298643.487101:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000075e99a4b/0xaf35132e6303410a lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2170880->2174975] (req 2170880->2174975) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3212 expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.487113:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3800, rc 0) 00000100:00100000:3.0:1780298643.487163:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556abb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:29360:x1866777609488640:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 820us (907us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298643.487171:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9397 00000100:00100000:0.0:1780298643.487368:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.487399:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609488768 00000100:00100000:2.0:1780298643.487402:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609488768 00010000:00010000:2.0:1780298643.487423:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e630340d2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31e1 expref: 11 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.487442:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15095 00000100:00100000:2.0:1780298643.487448:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556ab800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609488768:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298643.487455:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298643.487463:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e630340d2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31e1 expref: 11 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.487471:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e630340d2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31e1 expref: 11 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.487483:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e630340d2 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31e1 expref: 11 pid: 9128 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.487491:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(150 177) 00010000:00010000:2.0:1780298643.487493:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(150 177) 00010000:00010000:2.0:1780298643.487499:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000075e99a4b/0xaf35132e6303410a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2170880->35725311] (req 2170880->2174975) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3212 expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.487505:0:6463:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000075e99a4b/0xaf35132e6303410a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2170880->35725311] (req 2170880->2174975) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3212 expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.487526:0:6463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000075e99a4b/0xaf35132e6303410a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2170880->35725311] (req 2170880->2174975) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3212 expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.487533:0:6463:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bdf80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6463:1866777604952192:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298643.487560:0:6463:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4900 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.488299:0:6463:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bdf80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6463:1866777604952192:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298643.488307:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298643.488328:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556ab800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609488768:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 881us (961us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.488335:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15095 00000100:00100000:0.0:1780298643.488628:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.488665:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609488896 00000100:00100000:1.0:1780298643.488668:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609488896 00002000:00100000:1.0:1780298643.488690:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (2170880->2174975) req@ffff9cf2556a9c00 x1866777609488896/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.488697:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2556a9c00 with x1866777609488896 ext(2170880->2174975) 00002000:00010000:1.0:1780298643.488700:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2556a9c00 00000100:00100000:1.0:1780298643.488716:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6153 00000100:00100000:1.0:1780298643.488722:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609488896:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298643.488844:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609488896:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 122us (217us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298643.488850:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6153 00002000:00100000:1.0:1780298643.488854:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (2170880->2174975) req@ffff9cf2556a9c00 x1866777609488896/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.488860:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2556a9c00 with x1866777609488896 ext(2170880->2174975) 00002000:00010000:1.0:1780298643.488863:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2556a9c00 00000100:00100000:0.0:1780298643.490068:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.490085:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.490102:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609489152 00000100:00100000:2.0:1780298643.490105:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609489152 00000100:00100000:3.0:1780298643.490146:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609489024 00000100:00100000:3.0:1780298643.490153:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609489024 00000100:00100000:2.0:1780298643.490153:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9398 00000100:00100000:2.0:1780298643.490162:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556aa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609489152:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.490170:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.490179:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e63034111 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3220 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.490196:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9399 00000100:00100000:0.0:1780298643.490205:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609489024:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298643.490210:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604952320:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.490221:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.490234:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e63034118 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3219 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.490252:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4900 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.490267:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2556a8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604952448:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.490305:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417e40 going to sleep for 11 seconds 00002000:00010000:3.0:1780298643.490950:0:20252:0:(ofd_lvb.c:265:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime: 1780298643 -> 1780298644 00002000:00010000:3.0:1780298643.490955:0:20252:0:(ofd_lvb.c:277:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime: 1780298643 -> 1780298644 00000100:00100000:2.0:1780298643.490963:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604952320:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.490967:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2556a8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604952448:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.490985:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e63034111 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3220 expref: 7 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.490993:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e63034118 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3219 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.491005:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e63034111 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3220 expref: 7 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.491013:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e63034118 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3219 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.491030:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae000, rc 0) 00010000:00010000:2.0:1780298643.491042:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2400, rc 0) 00000100:00100000:3.0:1780298643.491071:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609489024:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 866us (981us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.491071:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556aa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609489152:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 907us (1000us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.491086:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9398 00000100:00100000:3.0:1780298643.491093:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9399 00000100:00100000:3.0:1780298643.492054:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.492089:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609489280 00000100:00100000:2.0:1780298643.492095:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609489280 00000100:00100000:3.0:1780298643.492115:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.492186:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609489408 00000100:00100000:2.0:1780298643.492190:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609489408 00000100:00100000:1.0:1780298643.492201:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9400 00000100:00100000:1.0:1780298643.492212:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609489280:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298643.492218:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9401 00010000:00010000:1.0:1780298643.492219:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.492229:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e6303411f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3227 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.492234:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609489408:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.492246:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.492256:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63034126 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d322e expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.492261:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604952576:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.492304:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2556a8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604952704:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.492314:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.492334:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.493039:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604952576:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.493048:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2556a8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604952704:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.493056:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e6303411f lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3227 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.493069:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63034126 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d322e expref: 7 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.493069:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e6303411f lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3227 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.493092:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93600, rc 0) 00010000:00010000:2.0:1780298643.493098:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63034126 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d322e expref: 7 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.493106:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf400, rc 0) 00000100:00100000:1.0:1780298643.493116:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609489280:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 907us (1062us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.493122:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9400 00000100:00100000:2.0:1780298643.493155:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609489408:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 926us (1039us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.493161:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9401 00000100:00100000:2.0:1780298643.494040:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.494077:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609489536 00000100:00100000:1.0:1780298643.494082:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609489536 00000100:00100000:1.0:1780298643.494113:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9402 00000100:00100000:1.0:1780298643.494141:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609489536:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.494149:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.494156:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e6303412d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3235 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.494178:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604952832:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.494209:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.494786:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604952832:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.494799:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e6303412d lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3235 expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.494808:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e6303412d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3235 expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.494815:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92a00, rc 0) 00000100:00100000:1.0:1780298643.494836:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609489536:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 717us (797us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.494841:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9402 00000100:00100000:0.0:1780298643.495783:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.495814:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609489664 00000100:00100000:2.0:1780298643.495818:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609489664 00002000:00100000:2.0:1780298643.495838:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x12:0x0] (2314240->2318335) req@ffff9cf2556ab480 x1866777609489664/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.495849:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2556ab480 with x1866777609489664 ext(2314240->2318335) 00002000:00010000:2.0:1780298643.495853:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2556ab480 00000100:00100000:2.0:1780298643.495869:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6154 00000100:00100000:2.0:1780298643.495874:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556ab480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609489664:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298643.496000:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556ab480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609489664:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 124us (216us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298643.496007:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6154 00002000:00100000:2.0:1780298643.496021:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x12:0x0] (2314240->2318335) req@ffff9cf2556ab480 x1866777609489664/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.496029:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2556ab480 with x1866777609489664 ext(2314240->2318335) 00002000:00010000:2.0:1780298643.496031:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2556ab480 00000100:00100000:2.0:1780298643.496938:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.496971:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609489792 00000100:00100000:1.0:1780298643.496975:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609489792 00002000:00100000:1.0:1780298643.496991:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (2318336->2322431) req@ffff9cf37f7be300 x1866777609489792/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.496997:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f7be300 with x1866777609489792 ext(2318336->2322431) 00002000:00010000:1.0:1780298643.497000:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f7be300 00000100:00100000:1.0:1780298643.497014:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6155 00000100:00100000:1.0:1780298643.497019:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7be300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609489792:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298643.497091:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7be300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609489792:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 71us (154us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298643.497099:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6155 00002000:00100000:1.0:1780298643.497106:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (2318336->2322431) req@ffff9cf37f7be300 x1866777609489792/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.497115:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f7be300 with x1866777609489792 ext(2318336->2322431) 00002000:00010000:1.0:1780298643.497118:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f7be300 00000100:00100000:3.0:1780298643.498171:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.498207:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609489920 00000100:00100000:0.0:1780298643.498211:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609489920 00002000:00100000:0.0:1780298643.498228:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x12:0x0] (2293760->2359295) req@ffff9cf2548b8700 x1866777609489920/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 504/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.498235:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2548b8700 with x1866777609489920 ext(2293760->2359295) 00002000:00010000:0.0:1780298643.498237:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2548b8700 00000100:00100000:0.0:1780298643.498267:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6156 00000100:00100000:0.0:1780298643.498276:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609489920:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298643.499710:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609489920:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 1433us (1537us total) trans 0 rc 57344/57344 00000100:00100000:0.0:1780298643.499718:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6156 00002000:00100000:0.0:1780298643.499724:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x12:0x0] (2293760->2359295) req@ffff9cf2548b8700 x1866777609489920/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 504/440 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 57344/57344 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.499730:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2548b8700 with x1866777609489920 ext(2293760->2359295) 00002000:00010000:0.0:1780298643.499733:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2548b8700 00000100:00100000:0.0:1780298643.500601:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.500663:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609490048 00000100:00100000:0.0:1780298643.500684:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.500703:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609490048 00000100:00100000:2.0:1780298643.500727:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609490176 00000100:00100000:2.0:1780298643.500733:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609490176 00000100:00100000:3.0:1780298643.500749:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9403 00000100:00100000:3.0:1780298643.500760:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609490048:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298643.500761:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9404 00010000:00010000:3.0:1780298643.500773:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298643.500773:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556aad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609490176:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.500781:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.500791:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e6303413b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3243 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.500792:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000024bbcf41/0xaf35132e63034134 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d323c expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.500846:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2556aa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604952960:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.500848:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548b8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604953088:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.500901:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417d80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.500902:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856600 going to sleep for 11 seconds 00002000:00010000:1.0:1780298643.501639:0:9128:0:(ofd_lvb.c:265:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime: 1780298643 -> 1780298644 00002000:00010000:1.0:1780298643.501643:0:9128:0:(ofd_lvb.c:271:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime: 1780298643 -> 1780298644 00002000:00010000:1.0:1780298643.501645:0:9128:0:(ofd_lvb.c:277:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime: 1780298643 -> 1780298644 00000100:00100000:3.0:1780298643.501649:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548b8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604953088:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.501661:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2556aa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604952960:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.501677:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000024bbcf41/0xaf35132e63034134 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d323c expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.501681:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e6303413b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3243 expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.501707:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e6303413b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3243 expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.501711:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000024bbcf41/0xaf35132e63034134 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d323c expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.501716:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baea00, rc 0) 00010000:00010000:3.0:1780298643.501728:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2c00, rc 0) 00000100:00100000:1.0:1780298643.501744:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556aad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609490176:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 974us (1059us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.501756:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9404 00000100:00100000:3.0:1780298643.501757:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609490048:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 995us (1152us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.501766:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9403 00000100:00100000:2.0:1780298643.502797:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.502822:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.502837:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609490304 00000100:00100000:0.0:1780298643.502842:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609490304 00000100:00100000:3.0:1780298643.502857:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609490432 00000100:00100000:3.0:1780298643.502863:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609490432 00000100:00100000:0.0:1780298643.502871:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9405 00000100:00100000:0.0:1780298643.502880:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609490304:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.502890:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.502898:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e63034142 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d324a expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.502904:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9406 00000100:00100000:1.0:1780298643.502920:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609490432:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.502928:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298643.502929:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2556aaa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604953216:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.502936:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e63034149 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3251 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.502963:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604953344:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.502970:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417d80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.502989:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a540 going to sleep for 11 seconds 00002000:00010000:2.0:1780298643.503682:0:8356:0:(ofd_lvb.c:271:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime: 1780298643 -> 1780298644 00000100:00100000:2.0:1780298643.503693:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2556aaa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604953216:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.503705:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604953344:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.503710:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e63034142 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d324a expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.503721:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e63034149 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3251 expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.503734:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e63034149 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3251 expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.503738:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e63034142 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d324a expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.503774:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92a00, rc 0) 00010000:00010000:2.0:1780298643.503775:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf000, rc 0) 00000100:00100000:1.0:1780298643.503811:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609490432:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 896us (987us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.503819:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609490304:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 939us (1021us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.503819:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9406 00000100:00100000:2.0:1780298643.503831:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9405 00000100:00100000:3.0:1780298643.504786:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.504813:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.504856:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609490560 00000100:00100000:0.0:1780298643.504860:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609490688 00000100:00100000:1.0:1780298643.504861:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609490560 00000100:00100000:0.0:1780298643.504867:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609490688 00000100:00100000:1.0:1780298643.504903:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9407 00000100:00100000:3.0:1780298643.504912:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9408 00000100:00100000:1.0:1780298643.504914:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609490560:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.504925:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298643.504927:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b00000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609490688:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.504937:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.504937:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e63034150 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3258 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.504947:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e63034157 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d325f expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.504975:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604953472:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.504982:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b03b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604953600:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.505017:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a540 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.505019:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.505886:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b03b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604953600:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.505912:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e63034157 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d325f expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.505921:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604953472:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.505931:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e63034157 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d325f expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.505942:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3200, rc 0) 00010000:00010000:1.0:1780298643.505942:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e63034150 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3258 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.505960:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e63034150 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3258 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.505972:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b00000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609490688:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1050us (1157us total) trans 0 rc 301/301 00010000:00010000:1.0:1780298643.505972:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93600, rc 0) 00000100:00100000:3.0:1780298643.505979:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9408 00000100:00100000:1.0:1780298643.506003:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609490560:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1092us (1217us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.506011:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9407 00000100:00100000:0.0:1780298643.507051:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.507100:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609490816 00000100:00100000:3.0:1780298643.507104:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609490816 00000100:00100000:3.0:1780298643.507169:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9409 00000100:00100000:3.0:1780298643.507177:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556ab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609490816:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.507185:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.507194:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000024bbcf41/0xaf35132e6303415e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3266 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.507205:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.507231:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b03b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604953728:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.507249:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609490944 00000100:00100000:2.0:1780298643.507255:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609490944 00000100:00100000:2.0:1780298643.507294:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9410 00000100:00100000:2.0:1780298643.507302:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556aaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609490944:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.507309:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.507317:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e63034165 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d326d expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.507333:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856600 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.507356:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604953856:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.507382:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.508014:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b03b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604953728:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.508032:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000024bbcf41/0xaf35132e6303415e lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3266 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.508044:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000024bbcf41/0xaf35132e6303415e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3266 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.508051:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2c00, rc 0) 00000100:00100000:3.0:1780298643.508079:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556ab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609490816:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 904us (1030us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.508109:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604953856:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.508144:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9409 00010000:00010000:2.0:1780298643.508146:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e63034165 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d326d expref: 7 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.508161:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e63034165 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d326d expref: 7 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.508173:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2400, rc 0) 00000100:00100000:2.0:1780298643.508202:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556aaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609490944:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 900us (1001us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.508210:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9410 00000100:00100000:0.0:1780298643.509100:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.509141:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.509163:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609491072 00000100:00100000:3.0:1780298643.509167:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609491072 00000100:00100000:2.0:1780298643.509170:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609491200 00000100:00100000:2.0:1780298643.509175:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609491200 00000100:00100000:3.0:1780298643.509193:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9411 00000100:00100000:3.0:1780298643.509200:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556aa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609491072:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.509207:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298643.509213:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9412 00010000:00010000:3.0:1780298643.509214:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000045ae2ef5/0xaf35132e6303416c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3274 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.509226:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609491200:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.509235:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298643.509239:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b03b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604953984:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.509245:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e63034173 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d327b expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.509279:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604954112:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.509292:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856600 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.509318:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.509947:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b03b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604953984:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.509962:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000045ae2ef5/0xaf35132e6303416c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3274 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.509972:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000045ae2ef5/0xaf35132e6303416c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3274 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.509979:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3200, rc 0) 00000100:00100000:3.0:1780298643.510007:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556aa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609491072:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 808us (908us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.510018:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9411 00000100:00100000:2.0:1780298643.510036:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604954112:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.510051:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e63034173 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d327b expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.510063:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e63034173 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d327b expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.510071:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3000, rc 0) 00000100:00100000:2.0:1780298643.510099:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609491200:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 878us (975us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.510106:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9412 00000100:00100000:1.0:1780298643.510840:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.510884:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609491328 00000100:00100000:0.0:1780298643.510889:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609491328 00000100:00100000:0.0:1780298643.510918:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9413 00000100:00100000:0.0:1780298643.510926:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5db800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:29360:x1866777609491328:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.510935:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.510944:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e6303417a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3282 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.510963:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000f2c2b189/0xaf35132e630340d9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31e8 expref: 10 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.510972:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(107 130) 00010000:00010000:0.0:1780298643.510976:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(108 130) 00010000:00010000:0.0:1780298643.510981:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e6303417a lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1720320->1728511] (req 1720320->1728511) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3282 expref: 7 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.511001:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000f2c2b189/0xaf35132e630340d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31e8 expref: 10 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.511024:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000f2c2b189/0xaf35132e630340d9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31e8 expref: 10 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.511039:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2556a8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604954240:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298643.511081:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.511700:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2556a8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604954240:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298643.511714:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e6303417a lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1720320->1728511] (req 1720320->1728511) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3282 expref: 7 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.511726:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baec00, rc 0) 00000100:00100000:1.0:1780298643.511756:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5db800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609491328:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 832us (917us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.511763:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9413 00000100:00100000:2.0:1780298643.511977:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.512021:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609491456 00000100:00100000:0.0:1780298643.512026:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609491456 00010000:00010000:0.0:1780298643.512049:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000f2c2b189/0xaf35132e630340d9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31e8 expref: 11 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.512076:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15096 00000100:00100000:0.0:1780298643.512085:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609491456:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298643.512096:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298643.512108:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000f2c2b189/0xaf35132e630340d9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31e8 expref: 11 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.512118:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000f2c2b189/0xaf35132e630340d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31e8 expref: 11 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.512156:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f2c2b189/0xaf35132e630340d9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31e8 expref: 11 pid: 8359 timeout: 932 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.512164:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(108 130) 00010000:00010000:0.0:1780298643.512166:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(108 130) 00010000:00010000:0.0:1780298643.512172:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e6303417a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1720320->35274751] (req 1720320->1728511) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3282 expref: 6 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.512181:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e6303417a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1720320->35274751] (req 1720320->1728511) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3282 expref: 6 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.512198:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e6303417a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1720320->35274751] (req 1720320->1728511) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3282 expref: 6 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.512204:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2556a8e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6461:1866777604954368:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298643.512232:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417e40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.512906:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2556a8e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6461:1866777604954368:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298643.512919:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298643.512948:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609491456:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 866us (973us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.512955:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15096 00000100:00100000:0.0:1780298643.512991:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.513030:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609491584 00000100:00100000:2.0:1780298643.513034:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609491584 00000100:00100000:2.0:1780298643.513069:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6157 00000100:00100000:2.0:1780298643.513077:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609491584:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298643.513247:0:16972:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:2.0:1780298643.513250:0:16972:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298644 -> 1780298643 00002000:00010000:2.0:1780298643.513251:0:16972:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298644 -> 1780298643 00002000:00010000:2.0:1780298643.513253:0:16972:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298644 -> 1780298643 00002000:00010000:2.0:1780298643.513254:0:16972:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 8 -> 24 00000100:00100000:2.0:1780298643.513276:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609491584:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 201us (287us total) trans 4294969229 rc 0/0 00000100:00100000:2.0:1780298643.513281:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6157 00000100:00100000:0.0:1780298643.514023:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.514062:0:6476:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609491712 00000100:00100000:1.0:1780298643.514066:0:6476:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609491712 00000100:00100000:1.0:1780298643.514094:0:6476:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61983 00000100:00100000:1.0:1780298643.514100:0:6476:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348005880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+142:29360:x1866777609491712:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.514112:0:6476:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.514141:0:6476:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93600/0xaf35132e63034181 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 74 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3289 expref: -99 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.514221:0:6476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93000/0xaf35132e63034188 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 75 type: IBT flags: 0x40000000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.514232:0:6476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93000/0xaf35132e63034188 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 75 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.514240:0:6476:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93000/0xaf35132e63034188 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 75 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1780298643.514259:0:6476:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 9263738 00000004:00200000:1.0:1780298643.514262:0:6476:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1780298643.514274:0:6476:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93000/0xaf35132e63034188 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 75 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.514301:0:6476:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93600/0xaf35132e63034181 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 75 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3289 expref: 144 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.514309:0:6476:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93000/0xaf35132e63034188 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 74 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3289 expref: 143 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.514315:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298643.514317:0:6476:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1780298643.514319:0:6476:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93000, rc 0) 00000100:00100000:1.0:1780298643.514341:0:6476:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348005880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+143:29360:x1866777609491712:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 241us (320us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.514346:0:6476:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61983 00000100:00100000:0.0:1780298643.515157:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.515176:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.515198:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609491840 00000100:00100000:2.0:1780298643.515203:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609491840 00000100:00100000:1.0:1780298643.515203:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609491968 00000100:00100000:1.0:1780298643.515209:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609491968 00000100:00100000:2.0:1780298643.515240:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9414 00000100:00100000:1.0:1780298643.515246:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9415 00000100:00100000:1.0:1780298643.515258:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348007b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609491968:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298643.515260:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348004380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609491840:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.515266:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.515269:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.515274:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e6303418f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3297 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.515279:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63034196 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3290 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.515323:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604954496:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.515325:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bf100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604954624:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.515378:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a540 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.515379:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.516148:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bf100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604954624:192.168.203.25@tcp:106: 00002000:00010000:1.0:1780298643.516153:0:12101:0:(ofd_lvb.c:265:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime: 1780298643 -> 1780298644 00002000:00010000:1.0:1780298643.516156:0:12101:0:(ofd_lvb.c:271:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime: 1780298643 -> 1780298644 00002000:00010000:1.0:1780298643.516158:0:12101:0:(ofd_lvb.c:277:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime: 1780298643 -> 1780298644 00010000:00010000:2.0:1780298643.516167:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63034196 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3290 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.516167:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604954496:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.516182:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63034196 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3290 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.516189:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3000, rc 0) 00010000:00010000:1.0:1780298643.516196:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e6303418f lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3297 expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.516208:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e6303418f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3297 expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.516214:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348004380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609491840:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 966us (1057us total) trans 0 rc 301/301 00010000:00010000:1.0:1780298643.516218:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92a00, rc 0) 00000100:00100000:2.0:1780298643.516221:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9414 00000100:00100000:1.0:1780298643.516251:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348007b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609491968:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 991us (1066us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.516262:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9415 00000100:00100000:0.0:1780298643.517108:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.517155:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609492096 00000100:00100000:2.0:1780298643.517160:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609492096 00000100:00100000:0.0:1780298643.517168:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.517192:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609492224 00000100:00100000:2.0:1780298643.517196:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609492224 00000100:00100000:3.0:1780298643.517205:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9416 00000100:00100000:3.0:1780298643.517214:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348005c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609492096:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298643.517218:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9417 00010000:00010000:3.0:1780298643.517224:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298643.517232:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348007100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609492224:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.517234:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000045ae2ef5/0xaf35132e6303419d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d329e expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.517242:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.517252:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c1a1ebef/0xaf35132e630341a4 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d32a5 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.517295:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604954752:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.517308:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bf100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604954880:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.517342:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.517342:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a540 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.518168:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604954752:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.518184:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bf100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604954880:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.518190:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c1a1ebef/0xaf35132e630341a4 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d32a5 expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.518202:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000045ae2ef5/0xaf35132e6303419d lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d329e expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.518207:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c1a1ebef/0xaf35132e630341a4 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d32a5 expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.518223:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93a00, rc 0) 00010000:00010000:2.0:1780298643.518224:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000045ae2ef5/0xaf35132e6303419d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d329e expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.518237:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3200, rc 0) 00000100:00100000:1.0:1780298643.518258:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348007100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609492224:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1030us (1090us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.518260:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348005c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609492096:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1047us (1153us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.518272:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9417 00000100:00100000:2.0:1780298643.518290:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9416 00000100:00100000:2.0:1780298643.519222:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.519263:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609492352 00000100:00100000:0.0:1780298643.519268:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609492352 00000100:00100000:0.0:1780298643.519317:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9418 00000100:00100000:0.0:1780298643.519327:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609492352:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.519337:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298643.519341:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:0.0:1780298643.519349:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000441cddf1/0xaf35132e630341ab lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d32ac expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.519376:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609492480 00000100:00100000:1.0:1780298643.519381:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609492480 00000100:00100000:0.0:1780298643.519382:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf348006680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604955008:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.519414:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9419 00000100:00100000:1.0:1780298643.519423:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b03b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609492480:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.519432:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298643.519436:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00010000:00010000:1.0:1780298643.519442:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000038a88423/0xaf35132e630341b2 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d32b3 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.519471:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604955136:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.519508:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a540 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.520080:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604955136:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.520100:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000038a88423/0xaf35132e630341b2 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d32b3 expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.520108:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348006680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604955008:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.520117:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038a88423/0xaf35132e630341b2 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d32b3 expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.520147:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93e00, rc 0) 00010000:00010000:0.0:1780298643.520150:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000441cddf1/0xaf35132e630341ab lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d32ac expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.520165:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000441cddf1/0xaf35132e630341ab lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d32ac expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.520175:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf400, rc 0) 00000100:00100000:1.0:1780298643.520180:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b03b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609492480:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 757us (840us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.520187:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9419 00000100:00100000:0.0:1780298643.520204:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609492352:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 878us (980us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.520213:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9418 00000100:00100000:2.0:1780298643.520978:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.520998:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.521010:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609492608 00000100:00100000:0.0:1780298643.521014:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609492608 00000100:00100000:1.0:1780298643.521030:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609492736 00000100:00100000:1.0:1780298643.521035:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609492736 00000100:00100000:0.0:1780298643.521041:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9420 00000100:00100000:0.0:1780298643.521050:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609492608:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.521057:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.521064:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630341b9 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d32ba expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.521071:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9421 00000100:00100000:1.0:1780298643.521082:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7be680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609492736:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298643.521087:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf348006680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604955264:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.521090:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.521101:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000056a806f/0xaf35132e630341c0 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d32c1 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.521122:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.521153:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604955392:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.521190:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a540 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.521765:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348006680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604955264:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.521778:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630341b9 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d32ba expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.521792:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630341b9 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d32ba expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.521799:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf000, rc 0) 00000100:00100000:0.0:1780298643.521826:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609492608:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 779us (848us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.521834:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9420 00000100:00100000:1.0:1780298643.521958:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604955392:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.521976:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000056a806f/0xaf35132e630341c0 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d32c1 expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.521990:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000056a806f/0xaf35132e630341c0 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d32c1 expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.522001:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92800, rc 0) 00000100:00100000:1.0:1780298643.522029:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7be680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609492736:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 951us (1030us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.522039:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9421 00000100:00100000:2.0:1780298643.522815:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.522845:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609492864 00000100:00100000:1.0:1780298643.522848:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609492864 00000100:00100000:1.0:1780298643.522871:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9422 00000100:00100000:1.0:1780298643.522876:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:29360:x1866777609492864:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.522882:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.522889:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004cdc8249/0xaf35132e630341c7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d32c8 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.522902:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(108 130) 00010000:00010000:1.0:1780298643.522905:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(108 130) 00010000:00010000:1.0:1780298643.522911:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004cdc8249/0xaf35132e630341c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [876544->1720319] (req 876544->884735) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d32c8 expref: 9 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.522920:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004cdc8249/0xaf35132e630341c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [876544->1720319] (req 876544->884735) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d32c8 expref: 9 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.522925:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92200, rc 0) 00000100:00100000:1.0:1780298643.522945:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:29360:x1866777609492864:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 70us (133us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.522949:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9422 00000100:00100000:3.0:1780298643.523561:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.523622:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609492992 00000100:00100000:1.0:1780298643.523626:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609492992 00000100:00100000:1.0:1780298643.523659:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6158 00000100:00100000:1.0:1780298643.523665:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b03100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609492992:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298643.523824:0:19943:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:1.0:1780298643.523827:0:19943:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298644 -> 1780298643 00002000:00010000:1.0:1780298643.523829:0:19943:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298644 -> 1780298643 00002000:00010000:1.0:1780298643.523830:0:19943:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298644 -> 1780298643 00002000:00010000:1.0:1780298643.523832:0:19943:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 24 -> 40 00000100:00100000:1.0:1780298643.523855:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b03100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609492992:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 191us (295us total) trans 4294969230 rc 0/0 00000100:00100000:1.0:1780298643.523860:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6158 00000100:00100000:3.0:1780298643.524616:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.524632:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.524648:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609493120 00000100:00100000:0.0:1780298643.524653:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609493120 00000100:00100000:2.0:1780298643.524666:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609493248 00000100:00100000:2.0:1780298643.524670:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609493248 00000100:00100000:0.0:1780298643.524687:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9423 00000100:00100000:0.0:1780298643.524694:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b03800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609493120:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298643.524697:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9424 00010000:00010000:0.0:1780298643.524703:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298643.524704:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b00380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609493248:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.524710:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.524711:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e630341ce lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d32cf expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.524716:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e630341d5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d32d6 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.524741:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255091f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604955520:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.524741:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf348006680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604955648:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.524781:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.524781:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00002000:00010000:0.0:1780298643.525518:0:19992:0:(ofd_lvb.c:265:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime: 1780298643 -> 1780298644 00002000:00010000:0.0:1780298643.525522:0:19992:0:(ofd_lvb.c:271:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime: 1780298643 -> 1780298644 00002000:00010000:0.0:1780298643.525524:0:19992:0:(ofd_lvb.c:277:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime: 1780298643 -> 1780298644 00000100:00100000:0.0:1780298643.525532:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255091f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604955520:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.525548:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348006680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604955648:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.525549:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e630341d5 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d32d6 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.525568:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e630341ce lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d32cf expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.525570:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e630341d5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d32d6 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.525589:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e630341ce lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d32cf expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.525599:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2800, rc 0) 00010000:00010000:3.0:1780298643.525600:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf600, rc 0) 00000100:00100000:3.0:1780298643.525626:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b03800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609493120:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 934us (1011us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.525632:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b00380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609493248:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 927us (996us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.525637:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9423 00000100:00100000:0.0:1780298643.525643:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9424 00000100:00100000:3.0:1780298643.526517:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.526551:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609493376 00000100:00100000:1.0:1780298643.526557:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609493376 00000100:00100000:1.0:1780298643.526585:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9425 00000100:00100000:1.0:1780298643.526595:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b00e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609493376:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.526604:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.526613:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e630341dc lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d32dd expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.526643:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604955776:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.526662:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.526691:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a540 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.526694:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609493504 00000100:00100000:3.0:1780298643.526698:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609493504 00000100:00100000:1.0:1780298643.526739:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9426 00000100:00100000:1.0:1780298643.526744:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255092300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609493504:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.526750:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.526755:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000670f33f0/0xaf35132e630341e3 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d32e4 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.526774:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604955904:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.526797:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837af00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.527254:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604955776:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.527272:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e630341dc lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d32dd expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.527437:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e630341dc lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d32dd expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.527449:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93600, rc 0) 00000100:00100000:3.0:1780298643.527453:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604955904:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.527480:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b00e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609493376:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 887us (962us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.527487:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9425 00010000:00010000:3.0:1780298643.527512:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000670f33f0/0xaf35132e630341e3 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d32e4 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.527521:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000670f33f0/0xaf35132e630341e3 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d32e4 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.527527:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92e00, rc 0) 00000100:00100000:3.0:1780298643.527544:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255092300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609493504:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 801us (884us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.527549:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9426 00000100:00100000:0.0:1780298643.528466:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.528502:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609493632 00000100:00100000:2.0:1780298643.528507:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609493632 00000100:00100000:2.0:1780298643.528540:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6159 00000100:00100000:2.0:1780298643.528548:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348007480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609493632:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298643.528654:0:19949:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 5069434 -> 5069434 00002000:00010000:2.0:1780298643.528657:0:19949:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298644 -> 1780298643 00002000:00010000:2.0:1780298643.528660:0:19949:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298644 -> 1780298643 00002000:00010000:2.0:1780298643.528662:0:19949:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298644 -> 1780298643 00002000:00010000:2.0:1780298643.528664:0:19949:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 360 -> 360 00000100:00100000:2.0:1780298643.528692:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348007480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609493632:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 146us (227us total) trans 4294969678 rc 0/0 00000100:00100000:2.0:1780298643.528699:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6159 00000100:00100000:0.0:1780298643.529478:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.529514:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609493760 00000100:00100000:1.0:1780298643.529520:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609493760 00000100:00100000:1.0:1780298643.529548:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9427 00000100:00100000:1.0:1780298643.529557:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348004e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609493760:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.529566:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.529574:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000038a88423/0xaf35132e630341ea lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d32eb expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.529605:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604956032:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.529647:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a540 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.529796:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.529826:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609493888 00000100:00100000:0.0:1780298643.529830:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609493888 00000100:00100000:0.0:1780298643.529857:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9428 00000100:00100000:0.0:1780298643.529863:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609493888:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.529871:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.529878:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e630341f1 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d32f2 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.529903:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf348006680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604956160:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.529934:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00002000:00010000:1.0:1780298643.530256:0:8358:0:(ofd_lvb.c:265:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime: 1780298643 -> 1780298644 00002000:00010000:1.0:1780298643.530259:0:8358:0:(ofd_lvb.c:271:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime: 1780298643 -> 1780298644 00002000:00010000:1.0:1780298643.530262:0:8358:0:(ofd_lvb.c:277:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime: 1780298643 -> 1780298644 00000100:00100000:1.0:1780298643.530269:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604956032:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.530294:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000038a88423/0xaf35132e630341ea lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d32eb expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.530308:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000038a88423/0xaf35132e630341ea lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d32eb expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.530316:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93e00, rc 0) 00000100:00100000:1.0:1780298643.530342:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348004e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609493760:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 788us (866us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.530348:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9427 00000100:00100000:0.0:1780298643.530560:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348006680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604956160:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.530571:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e630341f1 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d32f2 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.530584:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e630341f1 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d32f2 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.530590:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baea00, rc 0) 00000100:00100000:0.0:1780298643.530612:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609493888:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 750us (818us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.530616:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9428 00000100:00100000:2.0:1780298643.531453:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.531475:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609494016 00000100:00100000:3.0:1780298643.531479:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609494016 00000100:00100000:3.0:1780298643.531503:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9429 00000100:00100000:3.0:1780298643.531509:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255090000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609494016:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.531516:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.531523:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000024bbcf41/0xaf35132e630341f8 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d32f9 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.531545:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b00a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604956288:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.531549:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.531587:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609494144 00000100:00100000:1.0:1780298643.531592:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609494144 00000100:00100000:1.0:1780298643.531628:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9430 00000100:00100000:3.0:1780298643.531634:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.531640:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e581c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609494144:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.531648:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.531659:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb0d13ab/0xaf35132e630341ff lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3300 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.531685:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604956416:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.531732:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a540 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.532387:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b00a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604956288:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.532401:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000024bbcf41/0xaf35132e630341f8 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d32f9 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.532413:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000024bbcf41/0xaf35132e630341f8 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d32f9 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.532421:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2c00, rc 0) 00000100:00100000:1.0:1780298643.532445:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604956416:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.532447:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255090000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609494016:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 938us (998us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.532456:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9429 00010000:00010000:1.0:1780298643.532472:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d13ab/0xaf35132e630341ff lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3300 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.532493:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb0d13ab/0xaf35132e630341ff lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3300 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.532505:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92400, rc 0) 00000100:00100000:1.0:1780298643.532536:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e581c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609494144:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 900us (993us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.532543:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9430 00000100:00100000:2.0:1780298643.533353:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.533394:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609494272 00000100:00100000:1.0:1780298643.533398:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609494272 00000100:00100000:1.0:1780298643.533421:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9431 00000100:00100000:1.0:1780298643.533427:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255093800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609494272:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.533434:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.533441:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c1a1ebef/0xaf35132e63034206 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3307 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.533453:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(150 177) 00010000:00010000:1.0:1780298643.533456:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(150 177) 00010000:00010000:1.0:1780298643.533463:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c1a1ebef/0xaf35132e63034206 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1896448->2170879] (req 1896448->1908735) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3307 expref: 11 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.533473:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c1a1ebef/0xaf35132e63034206 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1896448->2170879] (req 1896448->1908735) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3307 expref: 11 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.533478:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93a00, rc 0) 00000100:00100000:1.0:1780298643.533499:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255093800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:29360:x1866777609494272:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 73us (151us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.533504:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9431 00000100:00100000:0.0:1780298643.534271:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.534315:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609494400 00000100:00100000:1.0:1780298643.534319:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609494400 00002000:00100000:1.0:1780298643.534345:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x2c0000401:0x11:0x0] (1896448->1900543) req@ffff9cf348006680 x1866777609494400/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.534352:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf348006680 with x1866777609494400 ext(1896448->1900543) 00002000:00010000:1.0:1780298643.534355:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf348006680 00000100:00100000:1.0:1780298643.534370:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6160 00000100:00100000:1.0:1780298643.534375:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348006680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609494400:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298643.534510:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348006680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609494400:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 129us (234us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298643.534516:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6160 00002000:00100000:1.0:1780298643.534520:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x2c0000401:0x11:0x0] (1896448->1900543) req@ffff9cf348006680 x1866777609494400/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.534526:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf348006680 with x1866777609494400 ext(1896448->1900543) 00002000:00010000:1.0:1780298643.534529:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf348006680 00000100:00100000:0.0:1780298643.535310:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.535344:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609494528 00000100:00100000:2.0:1780298643.535348:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609494528 00002000:00100000:2.0:1780298643.535365:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (1904640->1908735) req@ffff9cf348004380 x1866777609494528/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.535371:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf348004380 with x1866777609494528 ext(1904640->1908735) 00002000:00010000:2.0:1780298643.535374:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf348004380 00000100:00100000:2.0:1780298643.535389:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6161 00000100:00100000:2.0:1780298643.535395:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348004380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609494528:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298643.535465:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348004380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609494528:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 70us (155us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298643.535470:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6161 00002000:00100000:2.0:1780298643.535474:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (1904640->1908735) req@ffff9cf348004380 x1866777609494528/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.535480:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf348004380 with x1866777609494528 ext(1904640->1908735) 00002000:00010000:2.0:1780298643.535482:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf348004380 00000100:00100000:2.0:1780298643.536365:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.536410:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609494656 00000100:00100000:0.0:1780298643.536415:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609494656 00000100:00100000:3.0:1780298643.536437:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.536476:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609494784 00000100:00100000:0.0:1780298643.536481:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609494784 00000100:00100000:2.0:1780298643.536482:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9432 00000100:00100000:2.0:1780298643.536491:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255091180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609494656:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.536500:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.536508:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e6303420d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d330e expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.536516:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9433 00000100:00100000:0.0:1780298643.536528:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b00a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609494784:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.536536:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298643.536539:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255093480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604956544:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.536546:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e63034214 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3315 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.536580:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf348007b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604956672:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.536581:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.536621:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.537315:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255093480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604956544:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.537334:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e6303420d lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d330e expref: 12 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.537350:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e6303420d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d330e expref: 12 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.537362:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2400, rc 0) 00000100:00100000:2.0:1780298643.537405:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348007b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604956672:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.537421:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255091180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609494656:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 930us (1055us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.537428:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9432 00010000:00010000:2.0:1780298643.537428:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e63034214 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3315 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.537447:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e63034214 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3315 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.537458:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf200, rc 0) 00000100:00100000:2.0:1780298643.537489:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b00a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609494784:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 965us (1052us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.537496:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9433 00000100:00100000:0.0:1780298643.538271:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.538312:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609494912 00000100:00100000:1.0:1780298643.538318:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609494912 00000100:00100000:1.0:1780298643.538350:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9434 00000100:00100000:1.0:1780298643.538361:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348007100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609494912:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.538369:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.538380:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e6303421b lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d331c expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.538418:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604956800:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.538443:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.538461:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a840 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.538488:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609495040 00000100:00100000:0.0:1780298643.538493:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609495040 00000100:00100000:0.0:1780298643.538523:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9435 00000100:00100000:0.0:1780298643.538530:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b01c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609495040:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.538538:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.538545:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b35ce980/0xaf35132e63034222 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3323 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.538569:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf348005c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604956928:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.538602:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417e40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.539029:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604956800:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.539042:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e6303421b lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d331c expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.539052:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e6303421b lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d331c expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.539059:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92600, rc 0) 00000100:00100000:1.0:1780298643.539084:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348007100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609494912:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 727us (813us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.539091:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9434 00000100:00100000:0.0:1780298643.539170:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348005c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604956928:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.539190:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b35ce980/0xaf35132e63034222 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3323 expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.539205:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b35ce980/0xaf35132e63034222 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3323 expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.539215:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bafa00, rc 0) 00000100:00100000:0.0:1780298643.539239:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b01c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609495040:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 710us (851us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.539243:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9435 00000100:00100000:3.0:1780298643.539991:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.540019:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609495168 00000100:00100000:1.0:1780298643.540023:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609495168 00000100:00100000:1.0:1780298643.540045:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9436 00000100:00100000:1.0:1780298643.540050:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b00000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609495168:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.540056:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.540063:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e63034229 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d332a expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.540083:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604957056:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.540116:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a840 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.540152:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.540180:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609495296 00000100:00100000:0.0:1780298643.540184:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609495296 00000100:00100000:0.0:1780298643.540208:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9437 00000100:00100000:0.0:1780298643.540214:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255093100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609495296:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.540220:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.540226:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e63034230 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3331 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.540253:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf348005c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604957184:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.540296:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417e40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.540678:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5db800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604957056:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.540688:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e63034229 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d332a expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.540696:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e63034229 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d332a expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.540702:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92a00, rc 0) 00000100:00100000:1.0:1780298643.540723:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b00000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609495168:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 674us (733us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.540728:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9436 00000100:00100000:0.0:1780298643.540918:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348005c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604957184:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.540933:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e63034230 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3331 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.540947:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e63034230 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3331 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.540956:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baea00, rc 0) 00000100:00100000:0.0:1780298643.540983:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255093100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609495296:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 769us (861us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.540990:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9437 00000100:00100000:0.0:1780298643.541932:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.541965:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609495424 00000100:00100000:2.0:1780298643.541970:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609495424 00002000:00100000:2.0:1780298643.541995:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x12:0x0] (3235840->3239935) req@ffff9cf348005c00 x1866777609495424/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.542005:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf348005c00 with x1866777609495424 ext(3235840->3239935) 00002000:00010000:2.0:1780298643.542010:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf348005c00 00000100:00100000:2.0:1780298643.542029:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6162 00000100:00100000:2.0:1780298643.542034:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348005c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609495424:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298643.542150:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348005c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609495424:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 114us (218us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298643.542155:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6162 00002000:00100000:2.0:1780298643.542160:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x12:0x0] (3235840->3239935) req@ffff9cf348005c00 x1866777609495424/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.542166:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf348005c00 with x1866777609495424 ext(3235840->3239935) 00002000:00010000:2.0:1780298643.542169:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf348005c00 00000100:00100000:2.0:1780298643.543246:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.543294:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609495552 00000100:00100000:1.0:1780298643.543299:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609495552 00002000:00100000:1.0:1780298643.543320:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x12:0x0] (3239936->3244031) req@ffff9cf255090a80 x1866777609495552/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.543330:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255090a80 with x1866777609495552 ext(3239936->3244031) 00002000:00010000:1.0:1780298643.543334:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf255090a80 00000100:00100000:1.0:1780298643.543355:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6163 00000100:00100000:1.0:1780298643.543365:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255090a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609495552:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298643.543451:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255090a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609495552:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 88us (205us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298643.543458:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6163 00002000:00100000:1.0:1780298643.543465:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x12:0x0] (3239936->3244031) req@ffff9cf255090a80 x1866777609495552/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.543481:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255090a80 with x1866777609495552 ext(3239936->3244031) 00002000:00010000:1.0:1780298643.543484:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf255090a80 00000100:00100000:2.0:1780298643.544508:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.544534:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609495680 00000100:00100000:1.0:1780298643.544538:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609495680 00002000:00100000:1.0:1780298643.544554:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x12:0x0] (3235840->3244031) req@ffff9cf255093b80 x1866777609495680/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 8680/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.544560:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255093b80 with x1866777609495680 ext(3235840->3244031) 00002000:00010000:1.0:1780298643.544562:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf255093b80 00000100:00100000:1.0:1780298643.544576:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6164 00000100:00100000:1.0:1780298643.544581:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255093b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609495680:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:1.0:1780298643.544910:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255093b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609495680:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 327us (403us total) trans 4294969231 rc 0/0 00000100:00100000:1.0:1780298643.544917:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6164 00002000:00100000:1.0:1780298643.544922:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x12:0x0] (3235840->3244031) req@ffff9cf255093b80 x1866777609495680/t4294969231(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 8680/448 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.544928:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255093b80 with x1866777609495680 ext(3235840->3244031) 00002000:00010000:1.0:1780298643.544931:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf255093b80 00000100:00100000:0.0:1780298643.545571:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.545655:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609495808 00000100:00100000:1.0:1780298643.545661:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609495808 00000100:00100000:1.0:1780298643.545708:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9438 00000100:00100000:1.0:1780298643.545717:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348004e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609495808:12345-192.168.203.25@tcp:16:fsx.0 00000001:00080000:0.0:1780298643.546717:0:8351:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0000: transno 4294969231 is committed 00000001:00080000:0.0:1780298643.546726:0:8351:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0000: transno 4294969228 is committed 00000100:00100000:1.0:1780298643.546796:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348004e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609495808:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 1081us (1226us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.546805:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9438 00000100:00100000:2.0:1780298643.547800:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.547836:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609495936 00000100:00100000:2.0:1780298643.547840:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.547841:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609495936 00000100:00100000:3.0:1780298643.547864:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609496064 00000100:00100000:3.0:1780298643.547869:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609496064 00000100:00100000:0.0:1780298643.547876:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9439 00000100:00100000:0.0:1780298643.547883:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255090700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609495936:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.547893:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.547907:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000058e2d0/0xaf35132e63034237 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3338 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.547928:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9440 00000100:00100000:2.0:1780298643.547936:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255091500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609496064:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.547942:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.547952:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c47b04a/0xaf35132e6303423e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d333f expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.547961:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf348007b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604957312:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.547978:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604957440:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.548027:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4900 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.548027:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417e40 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.548707:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348007b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604957312:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.548727:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000058e2d0/0xaf35132e63034237 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3338 expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.548742:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000058e2d0/0xaf35132e63034237 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3338 expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.548750:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baea00, rc 0) 00000100:00100000:0.0:1780298643.548776:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255090700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609495936:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 895us (977us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.548782:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9439 00002000:00010000:3.0:1780298643.548876:0:20252:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks: 40 -> 56 00000100:00100000:3.0:1780298643.548885:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604957440:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.548898:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005c47b04a/0xaf35132e6303423e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d333f expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.548908:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c47b04a/0xaf35132e6303423e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d333f expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.548915:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3400, rc 0) 00000100:00100000:3.0:1780298643.548937:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255091500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609496064:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1004us (1097us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.548942:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9440 00000100:00100000:1.0:1780298643.549892:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.549927:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609496192 00000100:00100000:0.0:1780298643.549932:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609496192 00000100:00100000:0.0:1780298643.549962:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9441 00000100:00100000:0.0:1780298643.549968:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5db800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609496192:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.549975:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.549982:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63034245 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3346 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.549992:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.550006:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf348007b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604957568:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.550013:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609496320 00000100:00100000:2.0:1780298643.550020:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609496320 00000100:00100000:0.0:1780298643.550048:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417e40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.550054:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9442 00000100:00100000:2.0:1780298643.550063:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609496320:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.550074:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.550083:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e6303424c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d334d expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.550113:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604957696:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.550187:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.550712:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348007b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604957568:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.550723:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63034245 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3346 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.550732:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63034245 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3346 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.550738:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bafa00, rc 0) 00000100:00100000:0.0:1780298643.550760:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5db800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609496192:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 792us (869us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.550764:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9441 00000100:00100000:2.0:1780298643.550893:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604957696:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.550908:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e6303424c lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d334d expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.550923:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e6303424c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d334d expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.550933:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2200, rc 0) 00000100:00100000:2.0:1780298643.550960:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609496320:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 900us (970us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.550966:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9442 00000100:00100000:0.0:1780298643.551859:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.551899:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609496448 00000100:00100000:1.0:1780298643.551904:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609496448 00000100:00100000:1.0:1780298643.551932:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9443 00000100:00100000:1.0:1780298643.551939:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348007b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609496448:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298643.551944:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:1.0:1780298643.551949:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.551958:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e63034253 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3354 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.551975:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609496576 00000100:00100000:2.0:1780298643.551979:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609496576 00000100:00100000:1.0:1780298643.551989:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604957824:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.551999:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9444 00000100:00100000:2.0:1780298643.552006:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b03b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609496576:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.552013:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.552019:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e6303425a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d335b expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.552036:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.552044:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604957952:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.552067:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.552678:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604957824:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.552694:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e63034253 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3354 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.552708:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e63034253 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3354 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.552718:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92400, rc 0) 00000100:00100000:1.0:1780298643.552745:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348007b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609496448:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 806us (888us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.552752:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9443 00000100:00100000:2.0:1780298643.552794:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604957952:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.552810:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e6303425a lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d335b expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.552823:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e6303425a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d335b expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.552831:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2400, rc 0) 00000100:00100000:2.0:1780298643.552853:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b03b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609496576:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 849us (911us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.552858:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9444 00000100:00100000:3.0:1780298643.553882:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.553938:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609496704 00000100:00100000:1.0:1780298643.553943:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609496704 00002000:00100000:1.0:1780298643.553965:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x2c0000401:0x11:0x0] (3678208->3682303) req@ffff9cf241b03800 x1866777609496704/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.553972:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf241b03800 with x1866777609496704 ext(3678208->3682303) 00002000:00010000:1.0:1780298643.553976:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf241b03800 00000100:00100000:1.0:1780298643.553993:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6165 00000100:00100000:1.0:1780298643.553999:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b03800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609496704:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298643.554116:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b03800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609496704:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 117us (234us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298643.554121:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6165 00002000:00100000:1.0:1780298643.554144:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x2c0000401:0x11:0x0] (3678208->3682303) req@ffff9cf241b03800 x1866777609496704/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.554150:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf241b03800 with x1866777609496704 ext(3678208->3682303) 00002000:00010000:1.0:1780298643.554153:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf241b03800 00000100:00100000:0.0:1780298643.555087:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.555117:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609496832 00000100:00100000:2.0:1780298643.555120:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609496832 00002000:00100000:2.0:1780298643.555148:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (3682304->3686399) req@ffff9cf37f754000 x1866777609496832/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.555155:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f754000 with x1866777609496832 ext(3682304->3686399) 00002000:00010000:2.0:1780298643.555158:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f754000 00000100:00100000:2.0:1780298643.555175:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6166 00000100:00100000:2.0:1780298643.555180:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f754000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609496832:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298643.555244:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f754000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609496832:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 63us (159us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298643.555250:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6166 00002000:00100000:2.0:1780298643.555254:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (3682304->3686399) req@ffff9cf37f754000 x1866777609496832/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.555260:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f754000 with x1866777609496832 ext(3682304->3686399) 00002000:00010000:2.0:1780298643.555262:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f754000 00000100:00100000:3.0:1780298643.556200:0:3623:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3623:1866777604958080:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1780298643.556209:0:3625:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f755f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3625:1866777604958336:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1780298643.556220:0:3624:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3624:1866777604958208:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1780298643.556239:0:3623:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1780298643.556254:0:3625:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1780298643.556296:0:10402:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777604958080 00000100:00100000:0.0:1780298643.556301:0:10402:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777604958080 00000100:00100000:2.0:1780298643.556316:0:6481:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777604958336 00000100:00100000:2.0:1780298643.556321:0:6481:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777604958336 00000100:00100000:3.0:1780298643.556337:0:3624:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1780298643.556342:0:6473:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1780298643.556356:0:10402:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777604958208 00000100:00100000:0.0:1780298643.556362:0:10402:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777604958208 00000100:00100000:2.0:1780298643.556367:0:6481:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 647 00000100:00100000:2.0:1780298643.556377:0:6481:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255093480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3625:x1866777604958336:12345-0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1780298643.556384:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.556388:0:8360:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 587 00000100:00100000:3.0:1780298643.556397:0:8360:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b00380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3623:x1866777604958080:12345-0@lo:13:osp-pre-1-0.0 00000100:00080000:1.0:1780298643.556401:0:6473:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000004:00000024:2.0:1780298643.556403:0:6481:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1446043648 avail 1299804160 00000100:00100000:0.0:1780298643.556410:0:10402:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 588 00002000:00000024:3.0:1780298643.556411:0:8360:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 110592 granted 969636672 pending 0 free 3917504512 avail 3685355520 00000100:00100000:1.0:1780298643.556413:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609496960 00000100:00100000:1.0:1780298643.556417:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609496960 00000100:00100000:0.0:1780298643.556429:0:10402:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b00e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3624:x1866777604958208:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1780298643.556442:0:10402:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 106496 granted 969661248 pending 0 free 3923132416 avail 3691192320 00000100:00100000:1.0:1780298643.556457:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9445 00000100:00100000:3.0:1780298643.556461:0:8360:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b00380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3623:x1866777604958080:12345-0@lo:13:osp-pre-1-0.0 Request processed in 64us (220us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.556461:0:6481:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255093480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3625:x1866777604958336:12345-0@lo:41:osp-pre-0-1.0 Request processed in 85us (206us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.556470:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5dbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609496960:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298643.556476:0:8360:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 587 00000100:00100000:2.0:1780298643.556481:0:6481:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 647 00010000:00010000:1.0:1780298643.556482:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.556492:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000056a806f/0xaf35132e63034261 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3362 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.556492:0:10402:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b00e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3624:x1866777604958208:12345-0@lo:13:osp-pre-0-0.0 Request processed in 72us (155us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298643.556506:0:10402:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 588 00000100:00100000:3.0:1780298643.556508:0:3624:0:(client.c:3101:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294969077 gen 2 00000004:00080000:3.0:1780298643.556513:0:3624:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9cf3481fa680, transno 4294969077 00000100:00100000:3.0:1780298643.556530:0:3624:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3624:1866777604958208:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1780298643.556534:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604958464:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.556541:0:3625:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f755f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3625:1866777604958336:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1780298643.556583:0:3623:0:(client.c:3101:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294969474 gen 2 00000100:00100000:1.0:1780298643.556586:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a480 going to sleep for 11 seconds 00000004:00080000:2.0:1780298643.556588:0:3623:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff9cf254919500, transno 4294969474 00000100:00100000:2.0:1780298643.556607:0:3623:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3623:1866777604958080:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1780298643.556615:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.556643:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609497088 00000100:00100000:2.0:1780298643.556648:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609497088 00000100:00100000:2.0:1780298643.556679:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9446 00000100:00100000:2.0:1780298643.556696:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f757100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609497088:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.556704:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.556712:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e63034268 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3369 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.556739:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604958592:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.556768:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.557307:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604958464:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.557328:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000056a806f/0xaf35132e63034261 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3362 expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.557338:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604958592:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.557346:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000056a806f/0xaf35132e63034261 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3362 expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.557357:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e63034268 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3369 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.557357:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92800, rc 0) 00010000:00010000:2.0:1780298643.557374:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077813dbd/0xaf35132e63034268 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3369 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.557386:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2200, rc 0) 00000100:00100000:1.0:1780298643.557392:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5dbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609496960:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 925us (1007us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.557399:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9445 00000100:00100000:2.0:1780298643.557411:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f757100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609497088:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 723us (797us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.557419:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9446 00000100:00100000:3.0:1780298643.558244:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.558303:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609497216 00000100:00100000:1.0:1780298643.558308:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609497216 00000100:00100000:1.0:1780298643.558334:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9447 00000100:00100000:3.0:1780298643.558339:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.558343:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609497216:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.558349:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.558356:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e6303426f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3370 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.558377:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604958720:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.558378:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609497344 00000100:00100000:0.0:1780298643.558383:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609497344 00000100:00100000:1.0:1780298643.558412:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a480 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.558425:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9448 00000100:00100000:0.0:1780298643.558436:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609497344:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.558444:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.558451:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b35ce980/0xaf35132e63034276 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3377 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.558474:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f754380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604958848:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.558502:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417e40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.558958:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604958720:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.558970:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e6303426f lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3370 expref: 12 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.558981:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e6303426f lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3370 expref: 12 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.558988:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92400, rc 0) 00000100:00100000:1.0:1780298643.559013:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609497216:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 673us (770us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.559018:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9447 00000100:00100000:1.0:1780298643.559057:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f754380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604958848:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.559071:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b35ce980/0xaf35132e63034276 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3377 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.559081:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b35ce980/0xaf35132e63034276 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3377 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.559088:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bafa00, rc 0) 00000100:00100000:1.0:1780298643.559107:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609497344:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 676us (770us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.559111:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9448 00000100:00100000:2.0:1780298643.559958:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.560003:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609497472 00000100:00100000:0.0:1780298643.560009:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609497472 00000100:00100000:0.0:1780298643.560037:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9449 00000100:00100000:3.0:1780298643.560038:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.560045:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255092680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609497472:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298643.560053:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609497600 00010000:00010000:0.0:1780298643.560054:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298643.560058:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609497600 00010000:00010000:0.0:1780298643.560064:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e6303427d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d337e expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.560084:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9450 00000100:00100000:1.0:1780298643.560092:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609497600:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298643.560097:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f754a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604958976:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.560099:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.560107:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000670f33f0/0xaf35132e63034284 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3385 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.560147:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604959104:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.560162:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.560178:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a480 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.560670:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f754a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604958976:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.560682:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e6303427d lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d337e expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.560694:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e6303427d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d337e expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.560705:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae600, rc 0) 00000100:00100000:0.0:1780298643.560742:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255092680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609497472:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 697us (782us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.560750:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9449 00000100:00100000:1.0:1780298643.560777:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604959104:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.560788:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000670f33f0/0xaf35132e63034284 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3385 expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.560796:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000670f33f0/0xaf35132e63034284 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3385 expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.560804:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92e00, rc 0) 00000100:00100000:1.0:1780298643.560827:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609497600:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 738us (791us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.560832:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9450 00000100:00100000:2.0:1780298643.561641:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.561698:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609497728 00000100:00100000:1.0:1780298643.561701:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609497728 00000100:00100000:1.0:1780298643.561726:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6167 00000100:00100000:1.0:1780298643.561732:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255092300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609497728:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298643.561893:0:16972:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:1.0:1780298643.561895:0:16972:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298643.561897:0:16972:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298643.561899:0:16972:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298643.561900:0:16972:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 56 -> 88 00000100:00100000:1.0:1780298643.561921:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255092300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609497728:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 190us (280us total) trans 4294969232 rc 0/0 00000100:00100000:1.0:1780298643.561927:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6167 00000100:00100000:2.0:1780298643.562715:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.562751:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609497856 00000100:00100000:1.0:1780298643.562755:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609497856 00000100:00100000:2.0:1780298643.562780:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.562784:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9451 00000100:00100000:1.0:1780298643.562792:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255090000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609497856:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298643.562794:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609497984 00000100:00100000:3.0:1780298643.562798:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609497984 00010000:00010000:1.0:1780298643.562799:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.562808:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000038a88423/0xaf35132e6303428b lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d338c expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.562834:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604959232:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.562860:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9452 00000100:00100000:3.0:1780298643.562868:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255091f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609497984:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.562875:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298643.562875:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a480 going to sleep for 11 seconds 00010000:00010000:3.0:1780298643.562882:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e63034292 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3393 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.562910:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f746300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604959360:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.562996:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.563486:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f746300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604959360:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.563486:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604959232:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.563501:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e63034292 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3393 expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.563501:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000038a88423/0xaf35132e6303428b lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d338c expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.563514:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000038a88423/0xaf35132e6303428b lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d338c expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.563518:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e63034292 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3393 expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.563523:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93e00, rc 0) 00010000:00010000:3.0:1780298643.563525:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3200, rc 0) 00000100:00100000:3.0:1780298643.563548:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255091f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609497984:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 681us (770us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.563553:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255090000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609497856:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 764us (840us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.563554:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9452 00000100:00100000:1.0:1780298643.563562:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9451 00000100:00100000:1.0:1780298643.564392:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.564416:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.564429:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609498112 00000100:00100000:2.0:1780298643.564433:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609498112 00000100:00100000:0.0:1780298643.564442:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609498240 00000100:00100000:0.0:1780298643.564447:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609498240 00000100:00100000:2.0:1780298643.564457:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9453 00000100:00100000:2.0:1780298643.564464:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5da300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609498112:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.564471:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.564478:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000077813dbd/0xaf35132e63034299 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d339a expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.564486:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9454 00000100:00100000:1.0:1780298643.564496:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609498240:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298643.564504:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242860000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604959488:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.564505:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.564516:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000056a806f/0xaf35132e630342a0 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d33a1 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.564538:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.564545:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604959616:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.564572:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a480 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.565081:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242860000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604959488:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.565098:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000077813dbd/0xaf35132e63034299 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d339a expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.565114:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000077813dbd/0xaf35132e63034299 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d339a expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.565152:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2200, rc 0) 00000100:00100000:2.0:1780298643.565188:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5da300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609498112:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 723us (795us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.565196:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9453 00000100:00100000:0.0:1780298643.565268:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604959616:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.565280:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000056a806f/0xaf35132e630342a0 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d33a1 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.565302:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000056a806f/0xaf35132e630342a0 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d33a1 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.565309:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92800, rc 0) 00000100:00100000:0.0:1780298643.565331:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609498240:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 838us (916us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.565336:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9454 00000100:00100000:0.0:1780298643.566018:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.566055:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609498368 00000100:00100000:2.0:1780298643.566060:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609498368 00000100:00100000:2.0:1780298643.566091:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9455 00000100:00100000:2.0:1780298643.566101:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f754a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:29360:x1866777609498368:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.566110:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.566119:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e630342a7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d33a8 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.566162:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(108 130) 00010000:00010000:2.0:1780298643.566167:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(108 130) 00010000:00010000:2.0:1780298643.566178:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e630342a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [618496->876543] (req 618496->630783) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d33a8 expref: 11 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.566195:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e630342a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [618496->876543] (req 618496->630783) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d33a8 expref: 11 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.566204:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2e00, rc 0) 00000100:00100000:2.0:1780298643.566233:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f754a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:29360:x1866777609498368:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 134us (215us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.566240:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9455 00000100:00100000:2.0:1780298643.566878:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.566912:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609498496 00000100:00100000:1.0:1780298643.566916:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609498496 00000100:00100000:1.0:1780298643.566945:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6168 00000100:00100000:1.0:1780298643.566951:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242860000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609498496:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298643.567048:0:19943:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:1.0:1780298643.567051:0:19943:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298643.567052:0:19943:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298643.567054:0:19943:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298643.567055:0:19943:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 88 -> 88 00000100:00100000:1.0:1780298643.567076:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242860000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609498496:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 126us (200us total) trans 4294969233 rc 0/0 00000100:00100000:1.0:1780298643.567080:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6168 00000100:00100000:1.0:1780298643.567756:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.567790:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609498624 00000100:00100000:2.0:1780298643.567794:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609498624 00000100:00100000:2.0:1780298643.567824:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9456 00000100:00100000:2.0:1780298643.567833:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609498624:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298643.567842:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:2.0:1780298643.567842:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.567853:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e630342ae lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d33af expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.567863:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609498752 00000100:00100000:0.0:1780298643.567867:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609498752 00000100:00100000:2.0:1780298643.567886:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242862d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604959744:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.567901:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9457 00000100:00100000:0.0:1780298643.567910:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609498752:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.567917:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.567925:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e630342b5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d33b6 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.567932:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.567947:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f755180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604959872:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.567976:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.568543:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f755180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604959872:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.568553:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242862d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604959744:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.568558:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e630342b5 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d33b6 expref: 13 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.568569:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e630342b5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d33b6 expref: 13 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.568575:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e630342ae lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d33af expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.568577:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf200, rc 0) 00010000:00010000:2.0:1780298643.568595:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e630342ae lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d33af expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.568603:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609498752:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 694us (760us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.568617:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9457 00010000:00010000:2.0:1780298643.568625:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2800, rc 0) 00000100:00100000:2.0:1780298643.568651:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609498624:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 820us (895us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.568659:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9456 00000100:00100000:3.0:1780298643.569453:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.569475:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609499008 00000100:00100000:0.0:1780298643.569479:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609499008 00000100:00100000:0.0:1780298643.569499:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9458 00000100:00100000:0.0:1780298643.569503:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f747100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609499008:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.569508:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.569513:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e630342bc lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d33c4 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.569531:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f755180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604960000:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.569562:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.569611:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.569665:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609498880 00000100:00100000:3.0:1780298643.569669:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609498880 00000100:00100000:0.0:1780298643.569708:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9459 00000100:00100000:0.0:1780298643.569714:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609498880:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.569721:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.569726:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e630342c3 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d33bd expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.569743:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f755c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604960128:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.569763:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417d80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.570172:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f755180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604960000:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.570182:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e630342bc lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d33c4 expref: 13 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.570191:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e630342bc lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d33c4 expref: 13 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.570197:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae000, rc 0) 00000100:00100000:0.0:1780298643.570217:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f747100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609499008:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 714us (765us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.570222:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9458 00000100:00100000:0.0:1780298643.570363:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f755c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604960128:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.570372:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e630342c3 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d33bd expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.570381:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e630342c3 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d33bd expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.570391:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae600, rc 0) 00000100:00100000:0.0:1780298643.570410:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609498880:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 698us (801us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.570414:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9459 00000100:00100000:1.0:1780298643.571182:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.571215:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609499136 00000100:00100000:2.0:1780298643.571220:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609499136 00000100:00100000:2.0:1780298643.571247:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9460 00000100:00100000:1.0:1780298643.571253:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.571260:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609499136:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.571269:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.571279:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e630342ca lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d33cb expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.571313:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242862d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604960256:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.571318:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609499264 00000100:00100000:3.0:1780298643.571322:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609499264 00000100:00100000:3.0:1780298643.571345:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9461 00000100:00100000:3.0:1780298643.571352:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609499264:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.571359:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298643.571359:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00010000:00010000:3.0:1780298643.571366:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e630342d1 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d33d2 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.571388:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f747800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604960384:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.571408:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.571988:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242862d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604960256:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.572006:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e630342ca lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d33cb expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.572021:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e630342ca lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d33cb expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.572032:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3000, rc 0) 00000100:00100000:1.0:1780298643.572062:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609499136:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 805us (882us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.572070:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9460 00000100:00100000:2.0:1780298643.572172:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f747800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604960384:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.572187:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e630342d1 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d33d2 expref: 13 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.572197:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e630342d1 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d33d2 expref: 13 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.572208:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3200, rc 0) 00000100:00100000:2.0:1780298643.572236:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609499264:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 885us (986us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.572243:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9461 00000100:00100000:0.0:1780298643.572964:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.572982:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.572996:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609499392 00000100:00100000:3.0:1780298643.573001:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609499392 00000100:00100000:1.0:1780298643.573018:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609499520 00000100:00100000:1.0:1780298643.573022:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609499520 00000100:00100000:3.0:1780298643.573035:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9462 00000100:00100000:0.0:1780298643.573047:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9463 00000100:00100000:3.0:1780298643.573049:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f755c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609499392:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298643.573058:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f755180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609499520:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.573059:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.573066:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.573069:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000024bbcf41/0xaf35132e630342d8 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d33d9 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.573073:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e630342df lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d33e0 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.573106:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f756300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604960512:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.573115:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f744700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604960640:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.573165:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417d80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.573206:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.573846:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f744700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604960640:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.573862:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000024bbcf41/0xaf35132e630342d8 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d33d9 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.573868:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f756300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604960512:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.573879:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000024bbcf41/0xaf35132e630342d8 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d33d9 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.573889:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e630342df lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d33e0 expref: 13 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.573896:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2c00, rc 0) 00010000:00010000:2.0:1780298643.573906:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e630342df lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d33e0 expref: 13 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.573916:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf200, rc 0) 00000100:00100000:1.0:1780298643.573930:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f755c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609499392:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 886us (964us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.573939:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9462 00000100:00100000:2.0:1780298643.573943:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f755180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609499520:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 886us (957us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.573952:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9463 00000100:00100000:2.0:1780298643.574964:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.574992:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609499648 00000100:00100000:2.0:1780298643.574993:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.574998:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609499648 00000100:00100000:1.0:1780298643.575035:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609499776 00000100:00100000:1.0:1780298643.575038:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609499776 00000100:00100000:3.0:1780298643.575043:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9464 00000100:00100000:3.0:1780298643.575055:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242863b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609499648:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.575065:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298643.575070:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9465 00010000:00010000:3.0:1780298643.575075:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000052b89172/0xaf35132e630342e6 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d33e7 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.575080:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242861c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609499776:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.575091:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.575098:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a3f03a/0xaf35132e630342ed lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d33ee expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.575118:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f744700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604960768:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.575139:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f755880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604960896:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.575195:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.575205:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.575900:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f744700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604960768:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.575913:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f755880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604960896:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.575920:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000052b89172/0xaf35132e630342e6 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d33e7 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.575929:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000001a3f03a/0xaf35132e630342ed lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d33ee expref: 13 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.575933:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000052b89172/0xaf35132e630342e6 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d33e7 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.575948:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2800, rc 0) 00010000:00010000:0.0:1780298643.575949:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a3f03a/0xaf35132e630342ed lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d33ee expref: 13 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.575958:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae600, rc 0) 00000100:00100000:1.0:1780298643.575977:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242863b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609499648:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 926us (1013us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.575978:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242861c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609499776:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 900us (985us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.575990:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9464 00000100:00100000:0.0:1780298643.575997:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9465 00000100:00100000:1.0:1780298643.576991:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.577029:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609499904 00000100:00100000:2.0:1780298643.577035:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609499904 00002000:00100000:2.0:1780298643.577062:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x12:0x0] (3383296->3387391) req@ffff9cf36e5d9500 x1866777609499904/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.577073:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e5d9500 with x1866777609499904 ext(3383296->3387391) 00002000:00010000:2.0:1780298643.577078:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf36e5d9500 00000100:00100000:2.0:1780298643.577100:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6169 00000100:00100000:2.0:1780298643.577109:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609499904:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298643.577296:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609499904:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 178us (293us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298643.577305:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6169 00002000:00100000:2.0:1780298643.577313:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x12:0x0] (3383296->3387391) req@ffff9cf36e5d9500 x1866777609499904/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.577322:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e5d9500 with x1866777609499904 ext(3383296->3387391) 00002000:00010000:2.0:1780298643.577326:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf36e5d9500 00000100:00100000:2.0:1780298643.578368:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.578400:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609500032 00000100:00100000:1.0:1780298643.578405:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609500032 00002000:00100000:1.0:1780298643.578435:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (3387392->3391487) req@ffff9cf242863800 x1866777609500032/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.578442:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242863800 with x1866777609500032 ext(3387392->3391487) 00002000:00010000:1.0:1780298643.578445:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf242863800 00000100:00100000:1.0:1780298643.578460:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6170 00000100:00100000:1.0:1780298643.578466:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242863800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609500032:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298643.578540:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242863800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609500032:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 74us (173us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298643.578545:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6170 00002000:00100000:1.0:1780298643.578549:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (3387392->3391487) req@ffff9cf242863800 x1866777609500032/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.578555:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242863800 with x1866777609500032 ext(3387392->3391487) 00002000:00010000:1.0:1780298643.578558:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf242863800 00000100:00100000:3.0:1780298643.579623:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.579651:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609500160 00000100:00100000:2.0:1780298643.579655:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609500160 00002000:00100000:2.0:1780298643.579674:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x12:0x0] (3383296->3391487) req@ffff9cf37f747800 x1866777609500160/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 8680/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.579684:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f747800 with x1866777609500160 ext(3383296->3391487) 00002000:00010000:2.0:1780298643.579687:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f747800 00000100:00100000:2.0:1780298643.579711:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6171 00000100:00100000:2.0:1780298643.579719:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609500160:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:2.0:1780298643.580143:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f747800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2399:x1866777609500160:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 425us (519us total) trans 4294969234 rc 0/0 00000100:00100000:2.0:1780298643.580150:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6171 00002000:00100000:2.0:1780298643.580155:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x12:0x0] (3383296->3391487) req@ffff9cf37f747800 x1866777609500160/t4294969234(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 8680/448 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.580161:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f747800 with x1866777609500160 ext(3383296->3391487) 00002000:00010000:2.0:1780298643.580163:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f747800 00000100:00100000:0.0:1780298643.580853:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.580884:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609500288 00000100:00100000:2.0:1780298643.580887:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609500288 00000100:00100000:2.0:1780298643.580909:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9466 00000100:00100000:2.0:1780298643.580914:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f755880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2398:x1866777609500288:12345-192.168.203.25@tcp:16:fsx.0 00000001:00080000:0.0:1780298643.581583:0:8351:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0000: transno 4294969234 is committed 00000100:00100000:2.0:1780298643.581648:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f755880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609500288:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 733us (797us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.581653:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9466 00000100:00100000:1.0:1780298643.582532:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.582569:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609500416 00000100:00100000:2.0:1780298643.582574:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609500416 00000100:00100000:1.0:1780298643.582584:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.582610:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609500544 00000100:00100000:2.0:1780298643.582613:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609500544 00000100:00100000:2.0:1780298643.582635:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9467 00000100:00100000:2.0:1780298643.582644:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609500416:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298643.582644:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9468 00000100:00100000:0.0:1780298643.582653:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609500544:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.582655:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.582661:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.582671:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e630342f4 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d33f5 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.582671:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e630342fb lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d33fc expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.582715:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242863480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604961024:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.582715:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f757b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604961152:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.582767:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.582768:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00002000:00010000:0.0:1780298643.583591:0:19992:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks: 88 -> 104 00000100:00100000:2.0:1780298643.583597:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242863480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604961024:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.583603:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f757b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604961152:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.583623:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e630342fb lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d33fc expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.583624:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e630342f4 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d33f5 expref: 12 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.583654:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e630342f4 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d33f5 expref: 12 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.583654:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e630342fb lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d33fc expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.583676:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf600, rc 0) 00010000:00010000:2.0:1780298643.583677:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2800, rc 0) 00000100:00100000:0.0:1780298643.583704:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609500544:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1053us (1121us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.583709:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9468 00000100:00100000:2.0:1780298643.583713:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609500416:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1070us (1178us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.583720:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9467 00000100:00100000:1.0:1780298643.584782:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.584827:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609500672 00000100:00100000:1.0:1780298643.584833:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.584833:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609500672 00000100:00100000:0.0:1780298643.584875:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609500800 00000100:00100000:0.0:1780298643.584879:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609500800 00000100:00100000:2.0:1780298643.584882:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9469 00000100:00100000:2.0:1780298643.584895:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609500672:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.584906:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298643.584906:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9470 00000100:00100000:0.0:1780298643.584918:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609500800:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.584919:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000077813dbd/0xaf35132e63034302 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3403 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.584929:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.584939:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c37b15a2/0xaf35132e63034309 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d340a expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.584964:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242863480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604961280:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.584973:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f757b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604961408:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.585019:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.585019:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.585777:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242863480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604961280:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.585795:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000077813dbd/0xaf35132e63034302 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3403 expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.585795:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f757b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604961408:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.585814:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000077813dbd/0xaf35132e63034302 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3403 expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.585821:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c37b15a2/0xaf35132e63034309 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d340a expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.585824:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2200, rc 0) 00010000:00010000:0.0:1780298643.585841:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c37b15a2/0xaf35132e63034309 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d340a expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.585853:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baee00, rc 0) 00000100:00100000:2.0:1780298643.585854:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609500672:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 962us (1072us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.585862:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9469 00000100:00100000:0.0:1780298643.585886:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609500800:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 970us (1051us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.585893:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9470 00000100:00100000:3.0:1780298643.586917:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.586938:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.586951:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609500928 00000100:00100000:1.0:1780298643.586956:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609500928 00000100:00100000:0.0:1780298643.586970:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609501056 00000100:00100000:0.0:1780298643.586974:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609501056 00000100:00100000:1.0:1780298643.586991:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9471 00000100:00100000:1.0:1780298643.587002:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f744700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609500928:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.587012:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298643.587013:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9472 00000100:00100000:2.0:1780298643.587024:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255491500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609501056:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.587030:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e63034310 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3411 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.587033:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.587041:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e63034317 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3418 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.587072:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242863480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604961536:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.587080:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604961664:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.587114:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.587116:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837ae40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.587849:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242863480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604961536:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.587864:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e63034317 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3418 expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.587866:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604961664:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.587882:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e63034317 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3418 expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.587888:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e63034310 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3411 expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.587891:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2000, rc 0) 00010000:00010000:3.0:1780298643.587903:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e63034310 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3411 expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.587912:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92400, rc 0) 00000100:00100000:2.0:1780298643.587921:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255491500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609501056:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 901us (982us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.587927:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9472 00000100:00100000:3.0:1780298643.587933:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f744700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609500928:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 934us (1015us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.587938:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9471 00000100:00100000:0.0:1780298643.588805:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.588831:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609501184 00000100:00100000:3.0:1780298643.588835:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609501184 00000100:00100000:3.0:1780298643.588856:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9473 00000100:00100000:3.0:1780298643.588861:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f757b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:29360:x1866777609501184:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.588867:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.588873:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e6303431e lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d341f expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.588887:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c1a1ebef/0xaf35132e63034206 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1896448->2170879] (req 1896448->1908735) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3307 expref: 10 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.588893:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(150 177) 00010000:00010000:3.0:1780298643.588896:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(151 177) 00010000:00010000:3.0:1780298643.588899:0:20252:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e6303431e lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2158592->2162687] (req 2158592->2162687) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d341f expref: 11 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.588913:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c1a1ebef/0xaf35132e63034206 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1896448->2170879] (req 1896448->1908735) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3307 expref: 10 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.588923:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c1a1ebef/0xaf35132e63034206 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1896448->2170879] (req 1896448->1908735) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3307 expref: 10 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.588929:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255490380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604961792:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298643.588958:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856900 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.589572:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255490380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604961792:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298643.589590:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e6303431e lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2158592->2162687] (req 2158592->2162687) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d341f expref: 11 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.589601:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2400, rc 0) 00000100:00100000:3.0:1780298643.589628:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f757b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:29360:x1866777609501184:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 766us (823us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298643.589635:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9473 00000100:00100000:1.0:1780298643.589893:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.589938:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609501312 00000100:00100000:2.0:1780298643.589944:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609501312 00002000:00100000:2.0:1780298643.589970:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (1896448->1908735) req@ffff9cf2425f1880 x1866777609501312/t0(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 12776/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.589978:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2425f1880 with x1866777609501312 ext(1896448->1908735) 00010000:00010000:2.0:1780298643.589983:0:16972:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000c1a1ebef/0xaf35132e63034206 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1896448->2170879] (req 1896448->1908735) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3307 expref: 11 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.589996:0:16972:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000c1a1ebef/0xaf35132e63034206 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1896448->2170879] (req 1896448->1908735) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3307 expref: 11 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1780298643.590001:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf2425f1880 00000100:00100000:2.0:1780298643.590020:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 196 00000100:00100000:2.0:1780298643.590025:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609501312:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:2.0:1780298643.590506:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609501312:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 480us (614us total) trans 4294969679 rc 0/0 00000100:00100000:2.0:1780298643.590511:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 196 00002000:00100000:2.0:1780298643.590515:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (1896448->1908735) req@ffff9cf2425f1880 x1866777609501312/t4294969679(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 12776/448 e 0 to 0 dl 1780298654 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.590522:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2425f1880 with x1866777609501312 ext(1896448->1908735) 00010000:00010000:2.0:1780298643.590526:0:16972:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000c1a1ebef/0xaf35132e63034206 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1896448->2170879] (req 1896448->1908735) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3307 expref: 13 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.590534:0:16972:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000c1a1ebef/0xaf35132e63034206 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1896448->2170879] (req 1896448->1908735) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3307 expref: 13 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1780298643.590539:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf2425f1880 00000100:00100000:0.0:1780298643.591222:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.591269:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609501440 00000100:00100000:1.0:1780298643.591291:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609501440 00010000:00010000:1.0:1780298643.591314:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c1a1ebef/0xaf35132e63034206 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1896448->2170879] (req 1896448->1908735) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3307 expref: 13 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.591348:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15097 00000100:00100000:1.0:1780298643.591357:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b5ec50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609501440:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298643.591368:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:1.0:1780298643.591377:0:6462:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 360 -> 384 00010000:00010000:1.0:1780298643.591383:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c1a1ebef/0xaf35132e63034206 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1896448->2170879] (req 1896448->1908735) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3307 expref: 13 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.591394:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c1a1ebef/0xaf35132e63034206 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1896448->2170879] (req 1896448->1908735) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3307 expref: 13 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.591416:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c1a1ebef/0xaf35132e63034206 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1896448->2170879] (req 1896448->1908735) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3307 expref: 13 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.591427:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(151 177) 00010000:00010000:1.0:1780298643.591430:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(151 177) 00010000:00010000:1.0:1780298643.591438:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e6303431e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2158592->2170879] (req 2158592->2162687) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d341f expref: 10 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.591447:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e6303431e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2158592->2170879] (req 2158592->2162687) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d341f expref: 10 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.591473:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e6303431e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2158592->2170879] (req 2158592->2162687) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d341f expref: 10 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.591490:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f2300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777604961920:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298643.591533:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a480 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.592191:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f2300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777604961920:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298643.592204:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298643.592230:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b5ec50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609501440:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 875us (1008us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.592238:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15097 00000100:00100000:0.0:1780298643.592451:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.592488:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609501568 00000100:00100000:1.0:1780298643.592493:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609501568 00002000:00100000:1.0:1780298643.592514:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (2158592->2162687) req@ffff9cf37f756680 x1866777609501568/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.592523:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f756680 with x1866777609501568 ext(2158592->2162687) 00002000:00010000:1.0:1780298643.592527:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f756680 00000100:00100000:1.0:1780298643.592549:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6172 00000100:00100000:1.0:1780298643.592558:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f756680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609501568:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298643.592676:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f756680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609501568:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 120us (223us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298643.592684:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6172 00002000:00100000:1.0:1780298643.592691:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (2158592->2162687) req@ffff9cf37f756680 x1866777609501568/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.592700:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f756680 with x1866777609501568 ext(2158592->2162687) 00002000:00010000:1.0:1780298643.592704:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f756680 00000100:00100000:0.0:1780298643.593773:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.593815:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609501696 00000100:00100000:2.0:1780298643.593819:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609501696 00002000:00100000:2.0:1780298643.593835:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (2158592->2174975) req@ffff9cf37f755500 x1866777609501696/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 504/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.593846:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f755500 with x1866777609501696 ext(2158592->2174975) 00002000:00010000:2.0:1780298643.593852:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f755500 00000100:00100000:2.0:1780298643.593868:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6173 00000100:00100000:2.0:1780298643.593873:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f755500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609501696:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:2.0:1780298643.594786:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f755500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2397:x1866777609501696:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 911us (1011us total) trans 4294969680 rc 0/0 00000100:00100000:2.0:1780298643.594791:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6173 00002000:00100000:2.0:1780298643.594797:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (2158592->2174975) req@ffff9cf37f755500 x1866777609501696/t4294969680(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 504/448 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.594803:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f755500 with x1866777609501696 ext(2158592->2174975) 00002000:00010000:2.0:1780298643.594808:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f755500 00000100:00100000:0.0:1780298643.595565:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.595599:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609501824 00000100:00100000:2.0:1780298643.595603:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609501824 00000100:00100000:2.0:1780298643.595629:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9474 00000100:00100000:2.0:1780298643.595634:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f756300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2397:x1866777609501824:12345-192.168.203.25@tcp:16:fsx.0 00000001:00080000:1.0:1780298643.596272:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969680 is committed 00000001:00080000:1.0:1780298643.596280:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969679 is committed 00000100:00100000:2.0:1780298643.596367:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f756300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609501824:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 731us (800us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.596373:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9474 00000100:00100000:0.0:1780298643.597265:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.597309:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609501952 00000100:00100000:2.0:1780298643.597315:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609501952 00000100:00100000:2.0:1780298643.597351:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9475 00000100:00100000:2.0:1780298643.597360:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f754380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2397:x1866777609501952:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.597370:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.597382:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63034325 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3426 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.597430:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242863480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604962048:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.597444:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.597497:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4900 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.597513:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609502080 00000100:00100000:3.0:1780298643.597518:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609502080 00000100:00100000:3.0:1780298643.597543:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9476 00000100:00100000:3.0:1780298643.597548:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e581050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609502080:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.597601:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.597610:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003540747e/0xaf35132e6303432c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d342d expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.597634:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255490380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604962176:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.597661:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856900 going to sleep for 11 seconds 00002000:00010000:2.0:1780298643.598187:0:19992:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks: 384 -> 400 00000100:00100000:2.0:1780298643.598200:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242863480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604962048:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.598225:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63034325 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3426 expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.598246:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63034325 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3426 expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.598260:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2000, rc 0) 00000100:00100000:2.0:1780298643.598313:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f754380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2397:x1866777609501952:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 954us (1047us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.598322:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9475 00000100:00100000:3.0:1780298643.598399:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255490380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604962176:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.598412:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003540747e/0xaf35132e6303432c lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d342d expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.598424:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003540747e/0xaf35132e6303432c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d342d expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.598439:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3c00, rc 0) 00000100:00100000:3.0:1780298643.598467:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e581050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609502080:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 919us (1023us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.598474:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9476 00000100:00100000:1.0:1780298643.599317:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.599340:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609502208 00000100:00100000:3.0:1780298643.599344:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609502208 00000100:00100000:1.0:1780298643.599383:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9477 00000100:00100000:1.0:1780298643.599394:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609502208:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.599403:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.599413:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000038a88423/0xaf35132e63034333 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3434 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.599448:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604962304:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.599500:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a840 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.599506:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.599532:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609502336 00000100:00100000:3.0:1780298643.599535:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609502336 00000100:00100000:3.0:1780298643.599556:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9478 00000100:00100000:3.0:1780298643.599564:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f755f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609502336:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.599570:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.599577:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa41763f/0xaf35132e6303433a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d343b expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.599599:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255491500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604962432:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.599666:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856900 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.600171:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604962304:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.600188:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000038a88423/0xaf35132e63034333 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3434 expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.600200:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000038a88423/0xaf35132e63034333 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3434 expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.600207:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93e00, rc 0) 00000100:00100000:1.0:1780298643.600235:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609502208:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 846us (920us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.600243:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9477 00000100:00100000:2.0:1780298643.600304:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255491500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604962432:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.600324:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000aa41763f/0xaf35132e6303433a lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d343b expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.600337:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa41763f/0xaf35132e6303433a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d343b expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.600349:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3600, rc 0) 00000100:00100000:2.0:1780298643.600366:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f755f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609502336:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 805us (862us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.600371:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9478 00000100:00100000:2.0:1780298643.601199:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.601232:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609502464 00000100:00100000:1.0:1780298643.601238:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609502464 00000100:00100000:1.0:1780298643.601268:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9479 00000100:00100000:1.0:1780298643.601276:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242863480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609502464:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.601296:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.601306:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000056a806f/0xaf35132e63034341 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3442 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.601322:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(108 130) 00010000:00010000:1.0:1780298643.601327:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(108 130) 00010000:00010000:1.0:1780298643.601337:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000056a806f/0xaf35132e63034341 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [516096->614399] (req 516096->524287) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3442 expref: 10 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.601352:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000056a806f/0xaf35132e63034341 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [516096->614399] (req 516096->524287) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3442 expref: 10 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.601360:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92800, rc 0) 00000100:00100000:1.0:1780298643.601390:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242863480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609502464:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 115us (191us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.601397:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9479 00000100:00100000:2.0:1780298643.602004:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.602044:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609502592 00000100:00100000:1.0:1780298643.602049:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609502592 00000100:00100000:1.0:1780298643.602083:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6174 00000100:00100000:1.0:1780298643.602091:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242861f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609502592:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298643.602257:0:19951:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:1.0:1780298643.602261:0:19951:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298643.602263:0:19951:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298643.602266:0:19951:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298643.602268:0:19951:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 104 -> 104 00000100:00100000:1.0:1780298643.602306:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242861f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609502592:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 217us (303us total) trans 4294969235 rc 0/0 00000100:00100000:1.0:1780298643.602315:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6174 00000100:00100000:0.0:1780298643.603095:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.603113:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.603159:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609502720 00000100:00100000:3.0:1780298643.603161:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609502848 00000100:00100000:2.0:1780298643.603164:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609502720 00000100:00100000:3.0:1780298643.603166:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609502848 00000100:00100000:2.0:1780298643.603205:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9480 00000100:00100000:3.0:1780298643.603214:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9481 00000100:00100000:2.0:1780298643.603219:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f757100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609502720:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.603227:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298643.603228:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f754a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609502848:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.603237:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e63034348 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3449 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.603239:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.603250:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d828f4f8/0xaf35132e6303434f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3450 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.603278:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242861180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604962560:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.603310:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255490380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604962688:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.603342:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4900 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.603347:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.604082:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242861180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604962560:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.604106:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e63034348 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3449 expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.604146:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e63034348 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3449 expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.604157:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3600, rc 0) 00000100:00100000:1.0:1780298643.604192:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255490380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604962688:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.604211:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d828f4f8/0xaf35132e6303434f lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3450 expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.604224:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d828f4f8/0xaf35132e6303434f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3450 expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.604236:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2e00, rc 0) 00000100:00100000:3.0:1780298643.604241:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f757100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609502720:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1026us (1143us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.604249:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9480 00000100:00100000:1.0:1780298643.604263:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f754a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609502848:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1038us (1145us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.604271:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9481 00000100:00100000:2.0:1780298643.605273:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.605319:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.605320:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609502976 00000100:00100000:0.0:1780298643.605325:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609502976 00000100:00100000:3.0:1780298643.605351:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609503104 00000100:00100000:3.0:1780298643.605355:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609503104 00000100:00100000:0.0:1780298643.605371:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9482 00000100:00100000:0.0:1780298643.605382:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242861180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609502976:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298643.605391:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9483 00010000:00010000:0.0:1780298643.605401:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298643.605402:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242863b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609503104:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.605412:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.605412:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63034356 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3457 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.605422:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003540747e/0xaf35132e6303435d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d345e expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.605451:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f755c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604962816:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.605456:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255492300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604962944:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.605501:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.605501:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.606267:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f755c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604962816:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.606277:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255492300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604962944:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.606299:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63034356 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3457 expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.606310:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003540747e/0xaf35132e6303435d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d345e expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.606314:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63034356 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3457 expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.606345:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baee00, rc 0) 00010000:00010000:3.0:1780298643.606348:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003540747e/0xaf35132e6303435d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d345e expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.606357:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3c00, rc 0) 00000100:00100000:2.0:1780298643.606374:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242861180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609502976:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 995us (1102us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.606379:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242863b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609503104:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 981us (1061us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.606381:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9482 00000100:00100000:3.0:1780298643.606390:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9483 00000100:00100000:1.0:1780298643.607435:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.607463:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.607483:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609503232 00000100:00100000:0.0:1780298643.607489:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609503232 00000100:00100000:3.0:1780298643.607495:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609503360 00000100:00100000:3.0:1780298643.607499:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609503360 00000100:00100000:0.0:1780298643.607522:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9484 00000100:00100000:3.0:1780298643.607534:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9485 00000100:00100000:0.0:1780298643.607536:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609503232:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298643.607545:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609503360:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.607547:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.607551:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.607557:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63034364 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3465 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.607560:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002c458c77/0xaf35132e6303436b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d346c expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.607601:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255492300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604963072:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.607606:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f755180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604963200:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.607642:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.607642:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.608349:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255492300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604963072:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.608368:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002c458c77/0xaf35132e6303436b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d346c expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.608379:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002c458c77/0xaf35132e6303436b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d346c expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.608385:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f755180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604963200:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.608400:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2a00, rc 0) 00010000:00010000:0.0:1780298643.608407:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63034364 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3465 expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.608419:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63034364 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3465 expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.608427:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf200, rc 0) 00000100:00100000:3.0:1780298643.608428:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609503360:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 886us (963us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.608435:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9485 00000100:00100000:0.0:1780298643.608454:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609503232:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 923us (1028us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.608461:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9484 00000100:00100000:2.0:1780298643.609325:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.609360:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609503488 00000100:00100000:1.0:1780298643.609365:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609503488 00000100:00100000:1.0:1780298643.609398:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9486 00000100:00100000:1.0:1780298643.609406:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242861c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:29360:x1866777609503488:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.609415:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.609424:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e63034372 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3473 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.609441:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(151 177) 00010000:00010000:1.0:1780298643.609445:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(151 177) 00010000:00010000:1.0:1780298643.609455:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e63034372 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [946176->2158591] (req 946176->958463) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3473 expref: 9 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.609470:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e63034372 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [946176->2158591] (req 946176->958463) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3473 expref: 9 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.609479:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92400, rc 0) 00000100:00100000:1.0:1780298643.609509:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242861c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:29360:x1866777609503488:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 104us (184us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.609515:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9486 00000100:00100000:1.0:1780298643.610146:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.610166:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609503616 00000100:00100000:2.0:1780298643.610170:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609503616 00000100:00100000:2.0:1780298643.610200:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6175 00000100:00100000:2.0:1780298643.610205:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609503616:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298643.610373:0:8363:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 5069434 -> 5069434 00002000:00010000:2.0:1780298643.610376:0:8363:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298643.610378:0:8363:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298643.610380:0:8363:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298643.610381:0:8363:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 400 -> 424 00000100:00100000:2.0:1780298643.610404:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609503616:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 200us (262us total) trans 4294969681 rc 0/0 00000100:00100000:2.0:1780298643.610409:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6175 00000100:00100000:2.0:1780298643.611238:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.611269:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609503744 00000100:00100000:2.0:1780298643.611270:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.611275:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609503744 00000100:00100000:3.0:1780298643.611307:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609503872 00000100:00100000:3.0:1780298643.611310:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609503872 00000100:00100000:1.0:1780298643.611319:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9487 00000100:00100000:1.0:1780298643.611330:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242862d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609503744:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298643.611334:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9488 00010000:00010000:1.0:1780298643.611343:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.611353:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e63034379 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d347a expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.611354:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609503872:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.611367:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.611378:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e63034380 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3481 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.611397:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604963328:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.611418:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604963456:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.611459:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4900 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.611462:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.612167:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604963328:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.612190:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e63034379 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d347a expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.612206:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e63034379 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d347a expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.612222:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92600, rc 0) 00000100:00100000:3.0:1780298643.612246:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24310df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604963456:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.612257:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242862d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609503744:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 929us (1018us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.612264:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9487 00010000:00010000:3.0:1780298643.612319:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e63034380 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3481 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.612330:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e63034380 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3481 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.612338:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2400, rc 0) 00000100:00100000:3.0:1780298643.612423:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609503872:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1074us (1155us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.612428:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9488 00000100:00100000:2.0:1780298643.613423:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.613461:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609504000 00000100:00100000:0.0:1780298643.613468:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609504000 00000100:00100000:3.0:1780298643.613496:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.613506:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609504128 00000100:00100000:0.0:1780298643.613511:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609504128 00000100:00100000:2.0:1780298643.613523:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9489 00000100:00100000:2.0:1780298643.613532:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609504000:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.613540:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.613548:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005c47b04a/0xaf35132e63034387 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3488 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.613548:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9490 00000100:00100000:1.0:1780298643.613573:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255492300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609504128:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.613584:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298643.613585:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604963584:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.613595:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000670f33f0/0xaf35132e6303438e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d348f expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.613622:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604963712:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.613635:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.613648:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a840 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.614438:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24310fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604963584:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.614442:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604963712:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.614465:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005c47b04a/0xaf35132e63034387 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3488 expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.614467:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000670f33f0/0xaf35132e6303438e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d348f expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.614494:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005c47b04a/0xaf35132e63034387 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3488 expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.614496:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000670f33f0/0xaf35132e6303438e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d348f expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.614511:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3400, rc 0) 00010000:00010000:1.0:1780298643.614513:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92e00, rc 0) 00000100:00100000:1.0:1780298643.614551:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255492300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609504128:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 983us (1052us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.614551:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609504000:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1018us (1125us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.614565:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9489 00000100:00100000:1.0:1780298643.614572:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9490 00000100:00100000:0.0:1780298643.615635:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.615670:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609504256 00000100:00100000:3.0:1780298643.615674:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609504256 00000100:00100000:3.0:1780298643.615706:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9491 00000100:00100000:3.0:1780298643.615712:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f755180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:29360:x1866777609504256:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.615722:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.615729:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002c458c77/0xaf35132e63034395 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3496 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.615746:0:19992:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000004cdc8249/0xaf35132e630341c7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [876544->1720319] (req 876544->884735) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d32c8 expref: 8 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.615752:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(108 130) 00010000:00010000:3.0:1780298643.615756:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(109 130) 00010000:00010000:3.0:1780298643.615759:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002c458c77/0xaf35132e63034395 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [905216->913407] (req 905216->913407) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3496 expref: 12 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.615775:0:19992:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000004cdc8249/0xaf35132e630341c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [876544->1720319] (req 876544->884735) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d32c8 expref: 8 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.615786:0:19992:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000004cdc8249/0xaf35132e630341c7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [876544->1720319] (req 876544->884735) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d32c8 expref: 8 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.615795:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255493480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604963840:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298643.615830:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.616466:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255493480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604963840:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298643.616484:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002c458c77/0xaf35132e63034395 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [905216->913407] (req 905216->913407) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3496 expref: 12 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.616495:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2a00, rc 0) 00000100:00100000:2.0:1780298643.616523:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f755180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:29360:x1866777609504256:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 811us (888us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.616530:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9491 00000100:00100000:0.0:1780298643.616619:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.616658:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609504384 00000100:00100000:1.0:1780298643.616664:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609504384 00010000:00010000:1.0:1780298643.616686:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000004cdc8249/0xaf35132e630341c7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [876544->1720319] (req 876544->884735) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d32c8 expref: 9 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.616720:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15098 00000100:00100000:1.0:1780298643.616729:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f755880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609504384:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298643.616740:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298643.616755:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000004cdc8249/0xaf35132e630341c7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [876544->1720319] (req 876544->884735) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d32c8 expref: 9 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.616768:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000004cdc8249/0xaf35132e630341c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [876544->1720319] (req 876544->884735) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d32c8 expref: 9 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.616787:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004cdc8249/0xaf35132e630341c7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [876544->1720319] (req 876544->884735) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d32c8 expref: 9 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.616799:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(109 130) 00010000:00010000:1.0:1780298643.616803:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(109 130) 00010000:00010000:1.0:1780298643.616812:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000002c458c77/0xaf35132e63034395 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [905216->1720319] (req 905216->913407) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3496 expref: 11 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.616822:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002c458c77/0xaf35132e63034395 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [905216->1720319] (req 905216->913407) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3496 expref: 11 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.616847:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000002c458c77/0xaf35132e63034395 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [905216->1720319] (req 905216->913407) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3496 expref: 11 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.616859:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f3480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:7400:1866777604963968:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298643.616908:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.617561:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f3480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:7400:1866777604963968:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298643.617572:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298643.617598:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f755880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609504384:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 872us (981us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.617605:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15098 00000100:00100000:3.0:1780298643.617701:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.617743:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609504512 00000100:00100000:2.0:1780298643.617748:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609504512 00000100:00100000:2.0:1780298643.617787:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6176 00000100:00100000:2.0:1780298643.617796:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255491f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2399:x1866777609504512:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298643.618001:0:16972:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:2.0:1780298643.618004:0:16972:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298643.618006:0:16972:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298643.618007:0:16972:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298643.618008:0:16972:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 104 -> 120 00000100:00100000:2.0:1780298643.618031:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255491f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609504512:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 238us (332us total) trans 4294969236 rc 0/0 00000100:00100000:2.0:1780298643.618035:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6176 00000100:00100000:0.0:1780298643.618966:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.619010:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609504640 00000100:00100000:3.0:1780298643.619014:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609504640 00000100:00100000:3.0:1780298643.619043:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9492 00000100:00100000:3.0:1780298643.619051:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f757b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2397:x1866777609504640:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.619061:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298643.619067:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:3.0:1780298643.619075:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003540747e/0xaf35132e6303439c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d349d expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.619090:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609504768 00000100:00100000:0.0:1780298643.619095:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609504768 00000100:00100000:0.0:1780298643.619144:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9493 00000100:00100000:0.0:1780298643.619152:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609504768:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.619160:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298643.619168:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255493100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604964096:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.619168:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d4d8e3d7/0xaf35132e630343a3 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d34a4 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.619193:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f756300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604964224:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.619213:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.619213:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.619909:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255493100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604964096:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.619931:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003540747e/0xaf35132e6303439c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d349d expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.619934:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f756300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604964224:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.619951:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003540747e/0xaf35132e6303439c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d349d expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.619956:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d4d8e3d7/0xaf35132e630343a3 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d34a4 expref: 14 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.619965:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3c00, rc 0) 00010000:00010000:1.0:1780298643.619972:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d4d8e3d7/0xaf35132e630343a3 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d34a4 expref: 14 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.619982:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf000, rc 0) 00000100:00100000:3.0:1780298643.619994:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f757b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2397:x1866777609504640:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 943us (1034us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.620001:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609504768:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 851us (938us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.620004:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9492 00000100:00100000:1.0:1780298643.620012:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9493 00000100:00100000:1.0:1780298643.620900:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.620946:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609504896 00000100:00100000:1.0:1780298643.620946:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.620953:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609504896 00000100:00100000:3.0:1780298643.620971:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609505024 00000100:00100000:3.0:1780298643.620976:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609505024 00000100:00100000:2.0:1780298643.620992:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9494 00000100:00100000:2.0:1780298643.621002:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2400:x1866777609504896:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.621011:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.621020:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e630343aa lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d34ab expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.621025:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9495 00000100:00100000:0.0:1780298643.621037:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609505024:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.621045:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.621052:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b35ce980/0xaf35132e630343b1 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d34b2 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.621060:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604964352:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.621081:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f754380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604964480:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.621116:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417d80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.621117:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.621920:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24310ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604964352:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.621939:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f754380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604964480:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.621942:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e630343aa lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d34ab expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.621960:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ced6368f/0xaf35132e630343aa lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d34ab expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.621966:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b35ce980/0xaf35132e630343b1 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d34b2 expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.621992:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2a00, rc 0) 00010000:00010000:1.0:1780298643.621997:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b35ce980/0xaf35132e630343b1 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d34b2 expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.622009:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bafa00, rc 0) 00000100:00100000:2.0:1780298643.622024:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2400:x1866777609504896:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1024us (1123us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.622033:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9494 00000100:00100000:1.0:1780298643.622038:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609505024:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1002us (1088us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.622048:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9495 00000100:00100000:1.0:1780298643.623085:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.623144:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609505152 00000100:00100000:2.0:1780298643.623148:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609505152 00000100:00100000:1.0:1780298643.623167:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.623180:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609505280 00000100:00100000:2.0:1780298643.623183:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609505280 00000100:00100000:3.0:1780298643.623194:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9496 00000100:00100000:2.0:1780298643.623211:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9497 00000100:00100000:2.0:1780298643.623218:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609505280:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.623225:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.623233:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005c47b04a/0xaf35132e630343b8 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d34c0 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.623237:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548bad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609505152:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.623249:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.623257:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000fa1945a/0xaf35132e630343bf lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d34b9 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.623299:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604964608:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.623300:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255493100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604964736:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.623358:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.623360:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.623996:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24310ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604964608:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.624009:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005c47b04a/0xaf35132e630343b8 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d34c0 expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.624019:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005c47b04a/0xaf35132e630343b8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d34c0 expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.624026:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3400, rc 0) 00000100:00100000:2.0:1780298643.624051:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609505280:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 835us (886us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.624056:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9497 00000100:00100000:3.0:1780298643.624116:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255493100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604964736:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.624177:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000fa1945a/0xaf35132e630343bf lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d34b9 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.624187:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000fa1945a/0xaf35132e630343bf lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d34b9 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.624194:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3000, rc 0) 00000100:00100000:3.0:1780298643.624216:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548bad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609505152:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1015us (1133us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.624220:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9496 00000100:00100000:1.0:1780298643.625068:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.625116:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609505408 00000100:00100000:3.0:1780298643.625159:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609505408 00000100:00100000:0.0:1780298643.625178:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.625202:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609505536 00000100:00100000:3.0:1780298643.625207:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609505536 00000100:00100000:2.0:1780298643.625223:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9498 00000100:00100000:2.0:1780298643.625237:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548bb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609505408:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298643.625237:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9499 00010000:00010000:2.0:1780298643.625249:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298643.625253:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f754380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609505536:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.625260:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e630343c6 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d34c7 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.625262:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.625277:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000670f33f0/0xaf35132e630343cd lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d34ce expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.625306:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604964864:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.625322:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548bbb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604964992:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.625349:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.625358:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a840 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.626057:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24310ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604964864:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.626072:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e630343c6 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d34c7 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.626084:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e630343c6 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d34c7 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.626088:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548bbb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604964992:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.626101:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2400, rc 0) 00010000:00010000:0.0:1780298643.626109:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000670f33f0/0xaf35132e630343cd lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d34ce expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.626121:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000670f33f0/0xaf35132e630343cd lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d34ce expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.626147:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92e00, rc 0) 00000100:00100000:2.0:1780298643.626152:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548bb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609505408:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 917us (1080us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.626161:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9498 00000100:00100000:0.0:1780298643.626172:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f754380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609505536:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 923us (994us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.626181:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9499 00000100:00100000:0.0:1780298643.627440:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.627487:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609505664 00000100:00100000:1.0:1780298643.627494:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609505664 00002000:00100000:1.0:1780298643.627528:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (1310720->1314815) req@ffff9cf37f756300 x1866777609505664/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.627539:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f756300 with x1866777609505664 ext(1310720->1314815) 00002000:00010000:1.0:1780298643.627546:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f756300 00000100:00100000:1.0:1780298643.627571:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6177 00000100:00100000:1.0:1780298643.627581:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f756300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609505664:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298643.627756:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f756300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609505664:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 177us (316us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298643.627763:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6177 00002000:00100000:1.0:1780298643.627771:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (1310720->1314815) req@ffff9cf37f756300 x1866777609505664/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.627786:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f756300 with x1866777609505664 ext(1310720->1314815) 00002000:00010000:1.0:1780298643.627791:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f756300 00000100:00100000:1.0:1780298643.628716:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.628752:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609505792 00000100:00100000:2.0:1780298643.628760:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609505792 00002000:00100000:2.0:1780298643.628778:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (1290240->1355775) req@ffff9cf2548b9500 x1866777609505792/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 504/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.628785:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2548b9500 with x1866777609505792 ext(1290240->1355775) 00002000:00010000:2.0:1780298643.628787:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2548b9500 00000100:00100000:2.0:1780298643.628802:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6178 00000100:00100000:2.0:1780298643.628807:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609505792:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298643.629763:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609505792:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 954us (1047us total) trans 0 rc 57344/57344 00000100:00100000:2.0:1780298643.629769:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6178 00002000:00100000:2.0:1780298643.629774:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (1290240->1355775) req@ffff9cf2548b9500 x1866777609505792/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 504/440 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 57344/57344 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.629779:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2548b9500 with x1866777609505792 ext(1290240->1355775) 00002000:00010000:2.0:1780298643.629782:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2548b9500 00000100:00100000:1.0:1780298643.630821:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.630847:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.630866:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609505920 00000100:00100000:0.0:1780298643.630873:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609505920 00000100:00100000:2.0:1780298643.630878:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609506048 00000100:00100000:2.0:1780298643.630883:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609506048 00000100:00100000:0.0:1780298643.630917:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9500 00000100:00100000:2.0:1780298643.630924:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9501 00000100:00100000:0.0:1780298643.630931:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2400:x1866777609505920:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298643.630936:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609506048:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.630947:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.630947:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.630963:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ced6368f/0xaf35132e630343d4 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d34dc expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.630963:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630343db lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d34d5 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.631029:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604965120:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.631031:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f755c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604965248:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.631090:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.631091:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417e40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.631847:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24310ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604965120:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.631847:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f755c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604965248:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.631883:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630343db lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d34d5 expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.631885:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ced6368f/0xaf35132e630343d4 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d34dc expref: 14 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.631909:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630343db lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d34d5 expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.631911:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ced6368f/0xaf35132e630343d4 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d34dc expref: 14 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.631922:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf000, rc 0) 00010000:00010000:2.0:1780298643.631928:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2a00, rc 0) 00000100:00100000:1.0:1780298643.631957:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2400:x1866777609505920:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1028us (1135us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.631958:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609506048:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1023us (1105us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.631970:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9500 00000100:00100000:2.0:1780298643.631977:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9501 00000100:00100000:1.0:1780298643.632909:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.632928:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.632941:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609506176 00000100:00100000:0.0:1780298643.632946:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609506176 00000100:00100000:2.0:1780298643.632958:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609506304 00000100:00100000:2.0:1780298643.632963:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609506304 00000100:00100000:0.0:1780298643.632976:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9502 00000100:00100000:0.0:1780298643.632983:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2398:x1866777609506176:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.632990:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.632998:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e630343e2 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d34e3 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.633003:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9503 00000100:00100000:1.0:1780298643.633015:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609506304:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.633022:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298643.633029:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f755c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604965376:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.633030:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c1a1ebef/0xaf35132e630343e9 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d34ea expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.633059:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548b8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604965504:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.633074:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.633086:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837af00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.633809:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f755c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604965376:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.633824:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e630343e2 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d34e3 expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.633834:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e630343e2 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d34e3 expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.633839:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548b8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604965504:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.633843:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf200, rc 0) 00010000:00010000:2.0:1780298643.633856:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c1a1ebef/0xaf35132e630343e9 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d34ea expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.633870:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2398:x1866777609506176:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 888us (961us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.633878:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9502 00010000:00010000:2.0:1780298643.633880:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c1a1ebef/0xaf35132e630343e9 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d34ea expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.633892:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93a00, rc 0) 00000100:00100000:2.0:1780298643.633914:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609506304:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 902us (983us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.633921:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9503 00000100:00100000:2.0:1780298643.634856:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.634893:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609506432 00000100:00100000:1.0:1780298643.634900:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609506432 00000100:00100000:1.0:1780298643.634934:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6179 00000100:00100000:1.0:1780298643.634942:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609506432:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298643.635082:0:19951:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:1.0:1780298643.635085:0:19951:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298643.635087:0:19951:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298643.635088:0:19951:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298643.635090:0:19951:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 120 -> 120 00000100:00100000:1.0:1780298643.635115:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609506432:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 174us (258us total) trans 4294969237 rc 0/0 00000100:00100000:1.0:1780298643.635121:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6179 00000100:00100000:0.0:1780298643.635988:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.636010:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.636026:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609506560 00000100:00100000:3.0:1780298643.636032:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609506560 00000100:00100000:1.0:1780298643.636042:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609506688 00000100:00100000:1.0:1780298643.636047:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609506688 00000100:00100000:3.0:1780298643.636069:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9504 00000100:00100000:3.0:1780298643.636079:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f755c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609506560:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298643.636085:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9505 00010000:00010000:3.0:1780298643.636087:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298643.636094:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b42680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609506688:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.636097:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000fa1945a/0xaf35132e630343f0 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d34f1 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.636105:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.636114:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e630343f7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d34f8 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.636155:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255493100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604965632:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.636161:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604965760:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.636214:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.636215:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.637004:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24310fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604965760:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.637024:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e630343f7 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d34f8 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.637033:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255493100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604965632:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.637044:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e630343f7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d34f8 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.637053:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2400, rc 0) 00010000:00010000:3.0:1780298643.637057:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000fa1945a/0xaf35132e630343f0 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d34f1 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.637073:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000fa1945a/0xaf35132e630343f0 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d34f1 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.637080:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b42680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609506688:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 986us (1066us total) trans 0 rc 301/301 00010000:00010000:3.0:1780298643.637089:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3000, rc 0) 00000100:00100000:2.0:1780298643.637091:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9505 00000100:00100000:3.0:1780298643.637115:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f755c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609506560:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1038us (1126us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.637121:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9504 00000100:00100000:0.0:1780298643.637891:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.637924:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609506816 00000100:00100000:2.0:1780298643.637929:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609506816 00000100:00100000:2.0:1780298643.637957:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9506 00000100:00100000:2.0:1780298643.637964:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b41c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609506816:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.637973:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.637982:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005c47b04a/0xaf35132e630343fe lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d34ff expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.638013:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604965888:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.638032:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.638053:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.638070:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609506944 00000100:00100000:1.0:1780298643.638075:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609506944 00000100:00100000:1.0:1780298643.638113:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9507 00000100:00100000:1.0:1780298643.638152:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609506944:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.638161:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.638171:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e63034405 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3506 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.638201:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548b8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604966016:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.638244:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.638706:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24310fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604965888:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.638718:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005c47b04a/0xaf35132e630343fe lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d34ff expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.638727:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005c47b04a/0xaf35132e630343fe lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d34ff expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.638733:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3400, rc 0) 00000100:00100000:2.0:1780298643.638756:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b41c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609506816:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 793us (865us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.638761:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9506 00000100:00100000:1.0:1780298643.638886:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548b8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604966016:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.638902:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e63034405 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3506 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.638917:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e63034405 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3506 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.638927:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92a00, rc 0) 00000100:00100000:1.0:1780298643.638956:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609506944:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 837us (924us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.638965:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9507 00000100:00100000:3.0:1780298643.639740:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.639773:0:9976:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609507200 00000100:00100000:2.0:1780298643.639777:0:9976:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609507200 00000100:00100000:2.0:1780298643.639804:0:9976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61984 00000100:00100000:2.0:1780298643.639808:0:9976:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255493100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+109:29360:x1866777609507200:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:2.0:1780298643.639842:0:9976:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298643.639857:0:9976:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3e00/0xaf35132e630340e7 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 75 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d31ef expref: 109 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.639868:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298643.639872:0:9976:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298643.640011:0:9976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2a00/0xaf35132e6303440c lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 74 type: IBT flags: 0x40000000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.640025:0:9976:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93000/0xaf35132e63034188 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 74 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3289 expref: 142 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.640044:0:9976:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2a00/0xaf35132e6303440c lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 74 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.640066:0:9976:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93000/0xaf35132e63034188 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 74 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3289 expref: 142 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.640080:0:9976:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93000/0xaf35132e63034188 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 74 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3289 expref: 142 pid: 6476 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.640091:0:9976:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310fb80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:9976:1866777604966144:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298643.640160:0:9976:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.640755:0:9976:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24310fb80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:9976:1866777604966144:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298643.640764:0:9976:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2a00/0xaf35132e6303440c lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 74 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.640953:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.640981:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609507328 00000100:00100000:2.0:1780298643.640985:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609507328 00010000:00010000:2.0:1780298643.640998:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93000/0xaf35132e63034188 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 74 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3289 expref: 143 pid: 6476 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.641019:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15099 00000100:00100000:2.0:1780298643.641024:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255493800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+143:2400:x1866777609507328:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298643.641033:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298643.641037:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93000/0xaf35132e63034188 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 75 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3289 expref: 143 pid: 6476 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.641044:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93000/0xaf35132e63034188 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 75 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3289 expref: 143 pid: 6476 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.641053:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93000/0xaf35132e63034188 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 75 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3289 expref: 143 pid: 6476 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.641060:0:8374:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298643.641063:0:8374:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2a00/0xaf35132e6303440c lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 74 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.641070:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2a00/0xaf35132e6303440c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 74 type: IBT flags: 0x40290000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.641075:0:8374:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2a00/0xaf35132e6303440c lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 74 type: IBT flags: 0x40290000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.641088:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298643.641109:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255493800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+142:2400:x1866777609507328:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 85us (158us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.641114:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15099 00010000:00010000:0.0:1780298643.641116:0:9976:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2a00/0xaf35132e6303440c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 73 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.641151:0:9976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2a00/0xaf35132e6303440c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 73 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1780298643.641209:0:9976:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991050 00000004:00010000:0.0:1780298643.641235:0:9976:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 0000000089df40df reply state 00000000bd6432b0 transno 4294991051 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2a00/0xaf35132e6303440c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 73 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.641249:0:9976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2a00/0xaf35132e6303440c lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 73 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.641254:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298643.641256:0:9976:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1780298643.641257:0:9976:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1780298643.641326:0:9976:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255493100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+111:29360:x1866777609507200:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1515us (1587us total) trans 4294991051 rc 0/0 00000100:00100000:0.0:1780298643.641333:0:9976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61984 00000020:00080000:0.0:1780298643.641337:0:9976:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1780298643.641898:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.641940:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609507456 00000100:00100000:0.0:1780298643.641945:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609507456 00000100:00100000:0.0:1780298643.641977:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9508 00000100:00100000:0.0:1780298643.641982:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255493480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:29360:x1866777609507456:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.641990:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.641998:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e63034413 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d350d expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.642011:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e63034372 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [946176->2158591] (req 946176->958463) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3473 expref: 11 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.642017:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e6303431e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2158592->2170879] (req 2158592->2162687) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d341f expref: 11 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.642022:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000075e99a4b/0xaf35132e6303410a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2170880->35725311] (req 2170880->2174975) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3212 expref: 11 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.642030:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 3 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(151 177) 00010000:00010000:0.0:1780298643.642033:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(154 177) 00010000:00010000:0.0:1780298643.642035:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e63034413 lrc: 6/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d350d expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.642048:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000075e99a4b/0xaf35132e6303410a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2170880->35725311] (req 2170880->2174975) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3212 expref: 11 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.642055:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000075e99a4b/0xaf35132e6303410a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2170880->35725311] (req 2170880->2174975) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3212 expref: 11 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.642061:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604966272:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298643.642097:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e6303431e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2158592->2170879] (req 2158592->2162687) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d341f expref: 11 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.642104:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e6303431e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2158592->2170879] (req 2158592->2162687) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d341f expref: 11 pid: 20252 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.642111:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604966400:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298643.642159:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e63034372 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [946176->2158591] (req 946176->958463) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3473 expref: 11 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.642167:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e63034372 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [946176->2158591] (req 946176->958463) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3473 expref: 11 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.642175:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b41180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604966528:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298643.642193:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417e40 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.642879:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604966272:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298643.642890:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604966400:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298643.642997:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b41180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604966528:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298643.643012:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e63034413 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d350d expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.643021:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae600, rc 0) 00000100:00100000:0.0:1780298643.643046:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255493480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:29360:x1866777609507456:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1062us (1147us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298643.643054:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9508 00000100:00100000:3.0:1780298643.643146:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.643181:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609507584 00000100:00100000:2.0:1780298643.643186:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609507584 00010000:00010000:2.0:1780298643.643204:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000075e99a4b/0xaf35132e6303410a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2170880->35725311] (req 2170880->2174975) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3212 expref: 11 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.643241:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15100 00000100:00100000:2.0:1780298643.643256:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255490380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609507584:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298643.643269:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298643.643300:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000075e99a4b/0xaf35132e6303410a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [2170880->35725311] (req 2170880->2174975) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3212 expref: 11 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.643311:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000075e99a4b/0xaf35132e6303410a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [2170880->35725311] (req 2170880->2174975) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3212 expref: 11 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.643328:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000075e99a4b/0xaf35132e6303410a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [2170880->35725311] (req 2170880->2174975) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3212 expref: 11 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.643335:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:2.0:1780298643.643339:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298643.643343:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.643365:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255490380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609507584:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 111us (239us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.643373:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15100 00000100:00100000:1.0:1780298643.643373:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609507840 00000100:00100000:1.0:1780298643.643379:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609507840 00000100:00100000:2.0:1780298643.643389:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609507712 00000100:00100000:2.0:1780298643.643394:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609507712 00002000:00100000:1.0:1780298643.643410:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (1310720->1323007) req@ffff9cf242b41180 x1866777609507840/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 12776/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00010000:00010000:2.0:1780298643.643412:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e6303431e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2158592->2170879] (req 2158592->2162687) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d341f expref: 11 pid: 20252 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00100000:1.0:1780298643.643434:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b41180 with x1866777609507840 ext(1310720->1323007) 00010000:00010000:1.0:1780298643.643442:0:8363:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e63034372 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [946176->2158591] (req 946176->958463) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3473 expref: 11 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.643451:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15101 00010000:00010000:1.0:1780298643.643456:0:8363:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e63034372 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [946176->2158591] (req 946176->958463) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3473 expref: 11 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.643463:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255491500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609507712:12345-192.168.203.25@tcp:103:ldlm_bl.0 00002000:00010000:1.0:1780298643.643465:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf242b41180 00010000:00010000:2.0:1780298643.643476:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298643.643486:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e6303431e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2158592->2170879] (req 2158592->2162687) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d341f expref: 11 pid: 20252 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.643486:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 197 00000100:00100000:1.0:1780298643.643497:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b41180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609507840:12345-192.168.203.25@tcp:4:fsx.0 00010000:00010000:2.0:1780298643.643502:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e6303431e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2158592->2170879] (req 2158592->2162687) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d341f expref: 11 pid: 20252 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.643515:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e6303431e lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2158592->2170879] (req 2158592->2162687) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d341f expref: 11 pid: 20252 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.643527:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298643.643550:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255491500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609507712:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 92us (211us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.643557:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15101 00000100:00100000:1.0:1780298643.643930:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b41180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609507840:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 435us (596us total) trans 4294969682 rc 0/0 00000100:00100000:1.0:1780298643.643935:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 197 00002000:00100000:1.0:1780298643.643941:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (1310720->1323007) req@ffff9cf242b41180 x1866777609507840/t4294969682(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 12776/448 e 0 to 0 dl 1780298654 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.643952:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b41180 with x1866777609507840 ext(1310720->1323007) 00010000:00010000:1.0:1780298643.643956:0:8363:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e63034372 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [946176->2158591] (req 946176->958463) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3473 expref: 11 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.643966:0:8363:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e63034372 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [946176->2158591] (req 946176->958463) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3473 expref: 11 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1780298643.643972:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf242b41180 00000100:00100000:0.0:1780298643.644644:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.644673:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609507968 00000100:00100000:2.0:1780298643.644676:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609507968 00010000:00010000:2.0:1780298643.644689:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e63034372 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [946176->2158591] (req 946176->958463) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3473 expref: 11 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.644709:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15102 00000100:00100000:2.0:1780298643.644714:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b43100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609507968:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298643.644719:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:2.0:1780298643.644725:0:6461:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 424 -> 448 00010000:00010000:2.0:1780298643.644729:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e63034372 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [946176->2158591] (req 946176->958463) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3473 expref: 11 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.644736:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e63034372 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [946176->2158591] (req 946176->958463) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3473 expref: 11 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.644748:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e63034372 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [946176->2158591] (req 946176->958463) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3473 expref: 11 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.644755:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(154 177) 00010000:00010000:2.0:1780298643.644758:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(154 177) 00010000:00010000:2.0:1780298643.644762:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e63034413 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d350d expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.644767:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e63034413 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d350d expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.644784:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e63034413 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d350d expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.644791:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310fb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777604966656:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298643.644824:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.645474:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24310fb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777604966656:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298643.645482:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298643.645503:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609507968:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 789us (860us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.645513:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15102 00000100:00100000:0.0:1780298643.645576:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.645630:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609508096 00000100:00100000:3.0:1780298643.645634:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609508096 00000100:00100000:3.0:1780298643.645660:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9509 00000100:00100000:3.0:1780298643.645665:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:29360:x1866777609508096:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.645672:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.645681:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003540747e/0xaf35132e6303441a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3514 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.645719:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000056a806f/0xaf35132e63034341 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [516096->614399] (req 516096->524287) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3442 expref: 12 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.645731:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e630342a7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [618496->876543] (req 618496->630783) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d33a8 expref: 10 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.645740:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000002c458c77/0xaf35132e63034395 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [905216->1720319] (req 905216->913407) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3496 expref: 12 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.645748:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e6303417a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [1720320->35274751] (req 1720320->1728511) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3282 expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.645755:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 4 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(109 130) 00010000:00010000:1.0:1780298643.645758:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(113 130) 00010000:00010000:1.0:1780298643.645763:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003540747e/0xaf35132e6303441a lrc: 7/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3514 expref: 10 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.645787:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e6303417a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [1720320->35274751] (req 1720320->1728511) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3282 expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.645796:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e6303417a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [1720320->35274751] (req 1720320->1728511) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3282 expref: 10 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.645803:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548b8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604966784:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298643.645839:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000002c458c77/0xaf35132e63034395 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [905216->1720319] (req 905216->913407) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3496 expref: 12 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.645845:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000002c458c77/0xaf35132e63034395 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [905216->1720319] (req 905216->913407) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3496 expref: 12 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.645852:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548bbb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604966912:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298643.645884:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e630342a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [618496->876543] (req 618496->630783) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d33a8 expref: 10 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.645893:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e630342a7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [618496->876543] (req 618496->630783) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d33a8 expref: 10 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.645903:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2556ab800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604967040:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298643.645934:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000056a806f/0xaf35132e63034341 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [516096->614399] (req 516096->524287) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3442 expref: 12 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.645944:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000056a806f/0xaf35132e63034341 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [516096->614399] (req 516096->524287) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3442 expref: 12 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.645952:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2556a8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604967168:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298643.645973:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.646422:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548b8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604966784:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298643.646681:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548bbb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604966912:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298643.646691:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2556ab800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604967040:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298643.646835:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2556a8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604967168:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298643.646856:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003540747e/0xaf35132e6303441a lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3514 expref: 10 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.646868:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3c00, rc 0) 00000100:00100000:2.0:1780298643.646897:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:29360:x1866777609508096:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1231us (1320us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.646904:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9509 00000100:00100000:3.0:1780298643.646974:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.647009:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609508224 00000100:00100000:1.0:1780298643.647014:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609508224 00010000:00010000:1.0:1780298643.647034:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e6303417a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [1720320->35274751] (req 1720320->1728511) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3282 expref: 10 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.647064:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15103 00000100:00100000:1.0:1780298643.647073:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348007100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609508224:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298643.647083:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298643.647096:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e6303417a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [1720320->35274751] (req 1720320->1728511) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3282 expref: 10 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.647110:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:1.0:1780298643.647111:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e6303417a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [1720320->35274751] (req 1720320->1728511) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3282 expref: 10 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.647151:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e6303417a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [1720320->35274751] (req 1720320->1728511) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3282 expref: 10 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.647165:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:1.0:1780298643.647165:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298643.647166:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609508480 00000100:00100000:0.0:1780298643.647173:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609508480 00000100:00100000:2.0:1780298643.647189:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609508352 00000100:00100000:2.0:1780298643.647194:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609508352 00000100:00100000:1.0:1780298643.647195:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348007100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609508224:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 124us (220us total) trans 0 rc 0/0 00010000:00010000:0.0:1780298643.647202:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e630342a7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [618496->876543] (req 618496->630783) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d33a8 expref: 10 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.647206:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15103 00010000:00010000:2.0:1780298643.647215:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000002c458c77/0xaf35132e63034395 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [905216->1720319] (req 905216->913407) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3496 expref: 13 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.647239:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15104 00000100:00100000:0.0:1780298643.647247:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15105 00000100:00100000:1.0:1780298643.647252:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b5cc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609508480:12345-192.168.203.25@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1780298643.647262:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348004e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609508352:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298643.647264:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298643.647277:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e630342a7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [618496->876543] (req 618496->630783) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d33a8 expref: 9 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.647277:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298643.647301:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e630342a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [618496->876543] (req 618496->630783) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d33a8 expref: 9 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.647313:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000002c458c77/0xaf35132e63034395 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [905216->1720319] (req 905216->913407) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3496 expref: 13 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.647317:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e630342a7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 8 type: EXT [618496->876543] (req 618496->630783) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d33a8 expref: 9 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.647331:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000002c458c77/0xaf35132e63034395 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 8 type: EXT [905216->1720319] (req 905216->913407) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3496 expref: 13 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.647338:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1780298643.647352:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002c458c77/0xaf35132e63034395 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [905216->1720319] (req 905216->913407) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3496 expref: 13 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.647356:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:0.0:1780298643.647365:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298643.647373:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b5cc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609508480:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 125us (263us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.647380:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609508608 00000100:00100000:1.0:1780298643.647384:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15104 00000100:00100000:2.0:1780298643.647386:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609508608 00000100:00100000:0.0:1780298643.647388:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348004e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609508352:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 129us (224us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298643.647395:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15105 00010000:00010000:2.0:1780298643.647405:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000056a806f/0xaf35132e63034341 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [516096->614399] (req 516096->524287) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3442 expref: 13 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.647434:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15106 00000100:00100000:2.0:1780298643.647443:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609508608:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298643.647451:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298643.647460:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000056a806f/0xaf35132e63034341 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [516096->614399] (req 516096->524287) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3442 expref: 12 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.647471:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000056a806f/0xaf35132e63034341 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [516096->614399] (req 516096->524287) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3442 expref: 12 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.647483:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000056a806f/0xaf35132e63034341 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [516096->614399] (req 516096->524287) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3442 expref: 12 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.647494:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(113 130) 00010000:00010000:2.0:1780298643.647497:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(113 130) 00010000:00010000:2.0:1780298643.647502:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000003540747e/0xaf35132e6303441a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3514 expref: 7 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.647511:0:6463:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003540747e/0xaf35132e6303441a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3514 expref: 7 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.647533:0:6463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000003540747e/0xaf35132e6303441a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3514 expref: 7 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.647542:0:6463:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310df80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777604967296:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298643.647570:0:6463:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.648166:0:6463:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24310df80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777604967296:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298643.648175:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298643.648193:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609508608:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 754us (844us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.648198:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15106 00000100:00100000:2.0:1780298643.648359:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.648378:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.648396:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609508736 00000100:00100000:1.0:1780298643.648402:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609508736 00000100:00100000:3.0:1780298643.648408:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609508864 00000100:00100000:3.0:1780298643.648412:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609508864 00002000:00010000:1.0:1780298643.648420:0:16972:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (1543402->18446744073709551615) 00002000:00100000:1.0:1780298643.648425:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24310df80 with x1866777609508736 ext(1543402->18446744073709551615) 00002000:00010000:1.0:1780298643.648434:0:16972:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24310df80. 00002000:00010000:3.0:1780298643.648435:0:19928:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00002000:00100000:3.0:1780298643.648440:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2548bbb80 with x1866777609508864 ext(0->18446744073709551615) 00002000:00010000:3.0:1780298643.648446:0:19928:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2548bbb80. 00000100:00100000:1.0:1780298643.648458:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6180 00000100:00100000:1.0:1780298643.648467:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609508736:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:3.0:1780298643.648472:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6181 00000100:00100000:3.0:1780298643.648480:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609508864:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:1.0:1780298643.648788:0:16972:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 5069434 -> 1543402 00002000:00010000:1.0:1780298643.648791:0:16972:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298643.648793:0:16972:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298643.648796:0:16972:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298643.648798:0:16972:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 448 -> 376 00000100:00100000:1.0:1780298643.648831:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609508736:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 366us (471us total) trans 4294969683 rc 0/0 00000100:00100000:1.0:1780298643.648839:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6180 00002000:00010000:1.0:1780298643.648844:0:16972:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (1543402->18446744073709551615) 00002000:00100000:1.0:1780298643.648847:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24310df80 with x1866777609508736 ext(1543402->18446744073709551615) 00002000:00010000:1.0:1780298643.648852:0:16972:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24310df80. 00002000:00010000:3.0:1780298643.648880:0:19928:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4194304 -> 0 00002000:00010000:3.0:1780298643.648883:0:19928:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:3.0:1780298643.648885:0:19928:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:3.0:1780298643.648886:0:19928:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:3.0:1780298643.648888:0:19928:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 120 -> 0 00000100:00100000:3.0:1780298643.648911:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609508864:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 433us (532us total) trans 4294969238 rc 0/0 00000100:00100000:3.0:1780298643.648917:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6181 00002000:00010000:3.0:1780298643.648920:0:19928:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00002000:00100000:3.0:1780298643.648986:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2548bbb80 with x1866777609508864 ext(0->18446744073709551615) 00002000:00010000:3.0:1780298643.648990:0:19928:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2548bbb80. 00000100:00100000:2.0:1780298643.649655:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.649683:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609508992 00000100:00100000:1.0:1780298643.649687:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609508992 00000100:00100000:1.0:1780298643.649713:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61985 00000100:00100000:1.0:1780298643.649718:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+142:29360:x1866777609508992:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.649726:0:6477:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.649736:0:6477:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93600/0xaf35132e63034421 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 74 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d351b expref: -99 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.649795:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93e00/0xaf35132e63034428 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 75 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.649805:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93e00/0xaf35132e63034428 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 75 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.649814:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93e00/0xaf35132e63034428 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 75 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1780298643.649831:0:6477:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 1543402 00000004:00200000:1.0:1780298643.649834:0:6477:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1780298643.649845:0:6477:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93e00/0xaf35132e63034428 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 75 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.649859:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93600/0xaf35132e63034421 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 75 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d351b expref: 144 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.649867:0:6477:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93e00/0xaf35132e63034428 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 74 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d351b expref: 143 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.649872:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298643.649874:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1780298643.649876:0:6477:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93e00, rc 0) 00000100:00100000:1.0:1780298643.649896:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+143:29360:x1866777609508992:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 178us (242us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.649900:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61985 00000100:00100000:1.0:1780298643.650746:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.650772:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.650789:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609509120 00000100:00100000:0.0:1780298643.650795:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609509120 00000100:00100000:2.0:1780298643.650801:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609509248 00000100:00100000:2.0:1780298643.650805:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609509248 00000100:00100000:0.0:1780298643.650838:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9510 00000100:00100000:2.0:1780298643.650846:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9511 00000100:00100000:0.0:1780298643.650853:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609509120:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298643.650860:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556abb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609509248:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.650865:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.650867:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.650877:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e63034436 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3529 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.650877:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000441cddf1/0xaf35132e6303442f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3522 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.650922:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b6b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604967424:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.650931:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604967552:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.650962:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.650967:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417e40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.651661:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b6b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604967424:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.651677:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e63034436 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3529 expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.651681:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604967552:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.651701:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000441cddf1/0xaf35132e6303442f lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3522 expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.651707:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e63034436 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3529 expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.651714:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000441cddf1/0xaf35132e6303442f lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3522 expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.651717:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2400, rc 0) 00010000:00010000:1.0:1780298643.651726:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf400, rc 0) 00000100:00100000:2.0:1780298643.651745:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556abb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609509248:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 889us (971us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.651746:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609509120:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 900us (999us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.651757:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9511 00000100:00100000:1.0:1780298643.651765:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9510 00000100:00100000:3.0:1780298643.652609:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.652640:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609509376 00000100:00100000:0.0:1780298643.652652:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609509376 00000100:00100000:3.0:1780298643.652653:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.652686:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609509504 00000100:00100000:0.0:1780298643.652690:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609509504 00000100:00100000:1.0:1780298643.652694:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9512 00000100:00100000:1.0:1780298643.652704:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348007b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2400:x1866777609509376:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.652713:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298643.652718:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9513 00010000:00010000:1.0:1780298643.652723:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e6303443d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3530 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.652731:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348007100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609509504:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.652740:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.652748:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e63034444 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3537 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.652760:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2556aa300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604967680:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.652773:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b42300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604967808:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.652811:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417d80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.652813:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.653442:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2556aa300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604967680:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.653460:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e6303443d lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3530 expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.653469:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b42300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604967808:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.653472:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e6303443d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3530 expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.653483:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92a00, rc 0) 00010000:00010000:0.0:1780298643.653488:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e63034444 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3537 expref: 12 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.653499:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e63034444 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3537 expref: 12 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.653507:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348007b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2400:x1866777609509376:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 805us (937us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.653513:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9512 00010000:00010000:0.0:1780298643.653515:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf600, rc 0) 00000100:00100000:0.0:1780298643.653537:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348007100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609509504:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 812us (885us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.653542:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9513 00000100:00100000:3.0:1780298643.654394:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.654421:0:7411:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609509632 00000100:00100000:2.0:1780298643.654425:0:7411:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609509632 00000100:00100000:2.0:1780298643.654452:0:7411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61986 00000100:00100000:2.0:1780298643.654458:0:7411:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b00a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+111:29360:x1866777609509632:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.654468:0:7411:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.654477:0:7411:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3e00/0xaf35132e6303444b lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 75 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d353e expref: -99 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.654533:0:7411:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3600/0xaf35132e63034452 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 76 type: IBT flags: 0x40000000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.654542:0:7411:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3600/0xaf35132e63034452 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 76 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.654549:0:7411:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3600/0xaf35132e63034452 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 76 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298643.654563:0:7411:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 1543402 00000004:00200000:2.0:1780298643.654566:0:7411:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298643.654577:0:7411:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3600/0xaf35132e63034452 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 76 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.654589:0:7411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3e00/0xaf35132e6303444b lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 76 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d353e expref: 113 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.654598:0:7411:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3600/0xaf35132e63034452 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 75 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d353e expref: 112 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.654603:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298643.654605:0:7411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298643.654606:0:7411:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3600, rc 0) 00000100:00100000:2.0:1780298643.654629:0:7411:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b00a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+112:29360:x1866777609509632:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 172us (236us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.654633:0:7411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61986 00000100:00100000:3.0:1780298643.655931:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.655965:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609509760 00000100:00100000:2.0:1780298643.655969:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609509760 00000100:00100000:2.0:1780298643.656000:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6182 00000100:00100000:2.0:1780298643.656007:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b01c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609509760:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298643.656116:0:19949:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 1543402 -> 1543402 00002000:00010000:2.0:1780298643.656119:0:19949:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298643.656121:0:19949:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298643.656148:0:19949:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298643.656150:0:19949:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 376 -> 376 00000100:00100000:2.0:1780298643.656176:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b01c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609509760:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 170us (248us total) trans 4294969684 rc 0/0 00000100:00100000:2.0:1780298643.656181:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6182 00000100:00100000:3.0:1780298643.657004:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.657034:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609509888 00000100:00100000:3.0:1780298643.657037:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.657039:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609509888 00000100:00100000:1.0:1780298643.657066:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609510016 00000100:00100000:1.0:1780298643.657068:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609510016 00000100:00100000:2.0:1780298643.657073:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9514 00000100:00100000:2.0:1780298643.657081:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b00000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2400:x1866777609509888:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.657088:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298643.657093:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9515 00010000:00010000:2.0:1780298643.657096:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e63034459 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3545 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.657105:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b03b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609510016:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.657112:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.657120:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b87f9a7e/0xaf35132e63034460 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d354c expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.657155:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b6b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604967936:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.657188:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2556aa300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604968064:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.657198:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.657221:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.657845:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b6b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604967936:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.657857:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2556aa300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604968064:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.657865:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e63034459 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3545 expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.657875:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b87f9a7e/0xaf35132e63034460 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d354c expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.657880:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e63034459 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3545 expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.657892:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b87f9a7e/0xaf35132e63034460 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d354c expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.657898:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2400, rc 0) 00010000:00010000:1.0:1780298643.657901:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93600, rc 0) 00000100:00100000:2.0:1780298643.657926:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b00000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2400:x1866777609509888:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 844us (922us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.657926:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b03b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609510016:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 825us (889us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.657939:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9514 00000100:00100000:1.0:1780298643.657947:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9515 00000100:00100000:3.0:1780298643.658825:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.658851:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.658865:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609510144 00000100:00100000:2.0:1780298643.658869:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609510144 00000100:00100000:1.0:1780298643.658883:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609510272 00000100:00100000:1.0:1780298643.658887:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609510272 00000100:00100000:2.0:1780298643.658895:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9516 00000100:00100000:2.0:1780298643.658902:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255093800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2397:x1866777609510144:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.658908:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.658916:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63034467 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3553 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.658923:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9517 00000100:00100000:1.0:1780298643.658932:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255091180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609510272:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298643.658939:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b6b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604968192:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.658939:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.658947:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000670f33f0/0xaf35132e6303446e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d355a expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.658972:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2556aa300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604968320:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.658976:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.658996:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.659744:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2556aa300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604968320:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.659761:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b6b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604968192:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.659761:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000670f33f0/0xaf35132e6303446e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d355a expref: 12 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.659777:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000670f33f0/0xaf35132e6303446e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d355a expref: 12 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.659785:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92e00, rc 0) 00010000:00010000:2.0:1780298643.659787:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63034467 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3553 expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.659801:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63034467 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3553 expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.659810:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255091180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609510272:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 879us (957us total) trans 0 rc 301/301 00010000:00010000:2.0:1780298643.659814:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2000, rc 0) 00000100:00100000:1.0:1780298643.659816:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9517 00000100:00100000:2.0:1780298643.659842:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255093800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2397:x1866777609510144:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 939us (1017us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.659848:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9516 00000100:00100000:3.0:1780298643.660906:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.660944:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.660962:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609510400 00000100:00100000:0.0:1780298643.660967:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609510400 00000100:00100000:2.0:1780298643.660975:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609510528 00000100:00100000:2.0:1780298643.660980:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609510528 00000100:00100000:0.0:1780298643.661003:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9518 00000100:00100000:2.0:1780298643.661014:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9519 00000100:00100000:0.0:1780298643.661017:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255093100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2400:x1866777609510400:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.661029:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298643.661031:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255090700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609510528:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.661041:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.661041:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e63034475 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3561 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.661052:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e6303447c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3568 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.661074:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b42300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604968448:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.661084:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b6b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604968576:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.661114:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417d80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.661115:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.661839:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b42300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604968448:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.661854:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e63034475 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3561 expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.661865:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e63034475 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3561 expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.661873:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf600, rc 0) 00000100:00100000:0.0:1780298643.661897:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255093100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2400:x1866777609510400:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 887us (993us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.661902:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9518 00000100:00100000:2.0:1780298643.661917:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b6b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604968576:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.661933:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e6303447c lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3568 expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.661947:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e6303447c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3568 expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.661956:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3e00, rc 0) 00000100:00100000:2.0:1780298643.661976:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255090700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609510528:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 949us (1032us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.661983:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9519 00000100:00100000:1.0:1780298643.662812:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.662831:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.662845:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609510656 00000100:00100000:0.0:1780298643.662849:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609510656 00000100:00100000:2.0:1780298643.662876:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609510784 00000100:00100000:2.0:1780298643.662880:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609510784 00000100:00100000:0.0:1780298643.662881:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9520 00000100:00100000:0.0:1780298643.662891:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556aa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2397:x1866777609510656:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.662898:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.662905:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e63034483 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d356f expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.662912:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9521 00000100:00100000:2.0:1780298643.662921:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609510784:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.662928:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298643.662933:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b42300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604968704:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.662936:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e6303448a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3576 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.662963:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b6b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604968832:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.662975:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417d80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.662991:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4780 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.663694:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b6b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604968832:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.663712:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e6303448a lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3576 expref: 12 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.663723:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e6303448a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3576 expref: 12 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.663728:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b42300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604968704:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.663731:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3c00, rc 0) 00010000:00010000:0.0:1780298643.663752:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e63034483 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d356f expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.663760:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609510784:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 840us (927us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.663767:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9521 00010000:00010000:0.0:1780298643.663771:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e63034483 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d356f expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.663783:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae400, rc 0) 00000100:00100000:0.0:1780298643.663807:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556aa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2397:x1866777609510656:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 919us (995us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.663815:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9520 00000100:00100000:3.0:1780298643.666190:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.666247:0:6476:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609511040 00000100:00100000:1.0:1780298643.666255:0:6476:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609511040 00000100:00100000:1.0:1780298643.666308:0:6476:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61987 00000100:00100000:1.0:1780298643.666316:0:6476:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255091500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+143:29360:x1866777609511040:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:1.0:1780298643.666364:0:6476:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298643.666387:0:6476:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93e00/0xaf35132e63034428 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 76 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d351b expref: 143 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.666400:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298643.666403:0:6476:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298643.666555:0:6476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92a00/0xaf35132e63034491 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 75 type: IBT flags: 0x40000000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.666569:0:6476:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3600/0xaf35132e63034452 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 75 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d353e expref: 111 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.666601:0:6476:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92a00/0xaf35132e63034491 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 75 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.666634:0:6476:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3600/0xaf35132e63034452 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 75 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d353e expref: 111 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.666646:0:6476:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3600/0xaf35132e63034452 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 75 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d353e expref: 111 pid: 7411 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.666655:0:6476:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2556aad80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6476:1866777604968960:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298643.666714:0:6476:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837af00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.667379:0:6476:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2556aad80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6476:1866777604968960:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298643.667396:0:6476:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92a00/0xaf35132e63034491 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 75 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.667530:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.667554:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609511168 00000100:00100000:3.0:1780298643.667588:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609511168 00010000:00010000:3.0:1780298643.667613:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3600/0xaf35132e63034452 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 75 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d353e expref: 112 pid: 7411 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.667641:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15107 00000100:00100000:3.0:1780298643.667648:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b6b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+112:2400:x1866777609511168:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298643.667655:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298643.667659:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3600/0xaf35132e63034452 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 76 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d353e expref: 112 pid: 7411 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.667668:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3600/0xaf35132e63034452 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 76 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d353e expref: 112 pid: 7411 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.667677:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3600/0xaf35132e63034452 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 76 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d353e expref: 112 pid: 7411 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.667687:0:8374:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298643.667689:0:8374:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92a00/0xaf35132e63034491 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 75 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.667697:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92a00/0xaf35132e63034491 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 75 type: IBT flags: 0x40290000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.667702:0:8374:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92a00/0xaf35132e63034491 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 75 type: IBT flags: 0x40290000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.667716:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298643.667723:0:6476:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92a00/0xaf35132e63034491 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 74 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.667732:0:6476:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92a00/0xaf35132e63034491 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 74 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.667740:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b6b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+111:2400:x1866777609511168:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 93us (210us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298643.667745:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15107 00000001:00080000:2.0:1780298643.667778:0:6476:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991052 00000004:00010000:2.0:1780298643.667802:0:6476:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000a4ac32a6 reply state 00000000f31bff4b transno 4294991053 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92a00/0xaf35132e63034491 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 74 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.667814:0:6476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92a00/0xaf35132e63034491 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 74 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.667822:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298643.667825:0:6476:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1780298643.667827:0:6476:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1780298643.667888:0:6476:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255091500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+145:29360:x1866777609511040:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1572us (1704us total) trans 4294991053 rc 0/0 00000100:00100000:2.0:1780298643.667896:0:6476:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61987 00000020:00080000:2.0:1780298643.667900:0:6476:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1780298643.668538:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.668562:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609511296 00000100:00100000:3.0:1780298643.668568:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609511296 00000100:00100000:3.0:1780298643.668609:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9522 00000100:00100000:3.0:1780298643.668617:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b6a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:29360:x1866777609511296:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.668626:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.668637:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000052b89172/0xaf35132e63034498 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d357d expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.668655:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e63034413 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d350d expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.668662:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(154 177) 00010000:00010000:3.0:1780298643.668665:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(155 177) 00010000:00010000:3.0:1780298643.668668:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000052b89172/0xaf35132e63034498 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d357d expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.668682:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e63034413 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d350d expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.668691:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e63034413 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d350d expref: 11 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.668708:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604969088:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298643.668753:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.669301:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604969088:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298643.669314:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000052b89172/0xaf35132e63034498 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d357d expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.669324:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2800, rc 0) 00000100:00100000:3.0:1780298643.669344:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b6a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609511296:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 730us (807us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298643.669349:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9522 00000100:00100000:1.0:1780298643.669521:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.669560:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609511424 00000100:00100000:0.0:1780298643.669567:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609511424 00010000:00010000:0.0:1780298643.669584:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e63034413 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d350d expref: 12 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.669608:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15108 00000100:00100000:0.0:1780298643.669615:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609511424:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298643.669626:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298643.669641:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e63034413 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d350d expref: 12 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.669652:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e63034413 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d350d expref: 12 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.669674:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e63034413 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d350d expref: 12 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.669685:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(155 177) 00010000:00010000:0.0:1780298643.669688:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(155 177) 00010000:00010000:0.0:1780298643.669701:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000052b89172/0xaf35132e63034498 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d357d expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.669713:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000052b89172/0xaf35132e63034498 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d357d expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.669734:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000052b89172/0xaf35132e63034498 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d357d expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.669744:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b42300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777604969216:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298643.669776:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417d80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.670336:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b42300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777604969216:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298643.670345:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298643.670368:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609511424:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 753us (848us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298643.670375:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15108 00000100:00100000:1.0:1780298643.670450:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.670478:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609511552 00000100:00100000:2.0:1780298643.670484:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609511552 00000100:00100000:2.0:1780298643.670519:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9523 00000100:00100000:2.0:1780298643.670529:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e580450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:29360:x1866777609511552:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.670540:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.670551:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e6303449f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3584 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.670570:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003540747e/0xaf35132e6303441a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3514 expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.670584:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(113 130) 00010000:00010000:2.0:1780298643.670587:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(114 130) 00010000:00010000:2.0:1780298643.670592:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e6303449f lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3584 expref: 12 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.670618:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003540747e/0xaf35132e6303441a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3514 expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.670631:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003540747e/0xaf35132e6303441a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3514 expref: 8 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.670641:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b68380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604969344:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298643.670691:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4900 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.671259:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b68380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604969344:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298643.671275:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e6303449f lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3584 expref: 12 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.671303:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3c00, rc 0) 00000100:00100000:1.0:1780298643.671331:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e580450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:29360:x1866777609511552:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 805us (881us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.671338:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9523 00000100:00100000:3.0:1780298643.671439:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.671482:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609511680 00000100:00100000:2.0:1780298643.671486:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609511680 00010000:00010000:2.0:1780298643.671501:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000003540747e/0xaf35132e6303441a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3514 expref: 9 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.671521:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15109 00000100:00100000:2.0:1780298643.671526:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255092680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609511680:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298643.671533:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298643.671542:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000003540747e/0xaf35132e6303441a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3514 expref: 9 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.671552:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000003540747e/0xaf35132e6303441a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3514 expref: 9 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.671562:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003540747e/0xaf35132e6303441a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3514 expref: 9 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.671568:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(114 130) 00010000:00010000:2.0:1780298643.671571:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(114 130) 00010000:00010000:2.0:1780298643.671574:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e6303449f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3584 expref: 11 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.671579:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e6303449f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3584 expref: 11 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.671592:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e6303449f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3584 expref: 11 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.671598:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b69f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777604969472:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298643.671625:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4900 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.672261:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b69f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777604969472:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298643.672273:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298643.672313:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255092680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609511680:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 784us (874us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.672325:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15109 00000100:00100000:3.0:1780298643.672367:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.672395:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609511936 00000100:00100000:2.0:1780298643.672398:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609511936 00002000:00010000:2.0:1780298643.672413:0:19951:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (3650406->18446744073709551615) 00002000:00100000:2.0:1780298643.672416:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255091f80 with x1866777609511936 ext(3650406->18446744073709551615) 00002000:00010000:2.0:1780298643.672421:0:19951:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf255091f80. 00000100:00100000:3.0:1780298643.672443:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.672447:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6183 00000100:00100000:2.0:1780298643.672457:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255091f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609511936:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:0.0:1780298643.672460:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609511808 00000100:00100000:0.0:1780298643.672465:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609511808 00002000:00010000:0.0:1780298643.672480:0:19950:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:0.0:1780298643.672485:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255090000 with x1866777609511808 ext(4194304->18446744073709551615) 00002000:00010000:0.0:1780298643.672490:0:19950:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255090000. 00000100:00100000:0.0:1780298643.672510:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6184 00000100:00100000:0.0:1780298643.672518:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255090000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609511808:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:2.0:1780298643.672587:0:19951:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 0 -> 3650406 00002000:00010000:2.0:1780298643.672592:0:19951:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298643.672595:0:19951:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298643.672597:0:19951:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298643.672599:0:19951:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 0 -> 0 00002000:00010000:0.0:1780298643.672619:0:19950:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 1543402 -> 4194304 00002000:00010000:0.0:1780298643.672623:0:19950:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298643.672625:0:19950:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298643.672628:0:19950:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298643.672630:0:19950:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 376 -> 376 00000100:00100000:2.0:1780298643.672642:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255091f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609511936:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 186us (306us total) trans 4294969239 rc 0/0 00000100:00100000:2.0:1780298643.672651:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6183 00000100:00100000:0.0:1780298643.672655:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255090000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609511808:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 136us (211us total) trans 4294969685 rc 0/0 00002000:00010000:2.0:1780298643.672657:0:19951:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (3650406->18446744073709551615) 00002000:00100000:2.0:1780298643.672666:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255091f80 with x1866777609511936 ext(3650406->18446744073709551615) 00000100:00100000:0.0:1780298643.672670:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6184 00002000:00010000:2.0:1780298643.672674:0:19951:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf255091f80. 00002000:00010000:0.0:1780298643.672674:0:19950:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:0.0:1780298643.672679:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255090000 with x1866777609511808 ext(4194304->18446744073709551615) 00002000:00010000:0.0:1780298643.672685:0:19950:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255090000. 00000100:00100000:1.0:1780298643.674063:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.674098:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609512064 00000100:00100000:0.0:1780298643.674103:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609512064 00002000:00100000:0.0:1780298643.674147:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x12:0x0] (3649536->3653631) req@ffff9cf2556abb80 x1866777609512064/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.674158:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2556abb80 with x1866777609512064 ext(3649536->3653631) 00002000:00010000:0.0:1780298643.674162:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2556abb80 00000100:00100000:0.0:1780298643.674180:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6185 00000100:00100000:0.0:1780298643.674188:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556abb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609512064:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298643.674311:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556abb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609512064:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 124us (249us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298643.674319:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6185 00002000:00100000:0.0:1780298643.674325:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x12:0x0] (3649536->3653631) req@ffff9cf2556abb80 x1866777609512064/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.674333:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2556abb80 with x1866777609512064 ext(3649536->3653631) 00002000:00010000:0.0:1780298643.674337:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2556abb80 00000100:00100000:1.0:1780298643.675513:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.675543:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609512192 00000100:00100000:3.0:1780298643.675547:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609512192 00002000:00100000:3.0:1780298643.675563:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x12:0x0] (3649536->3650405) req@ffff9cf2556a9880 x1866777609512192/t0(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 1360/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298643.675570:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2556a9880 with x1866777609512192 ext(3649536->3650405) 00002000:00010000:3.0:1780298643.675572:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2556a9880 00000100:00100000:3.0:1780298643.675588:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6186 00000100:00100000:3.0:1780298643.675595:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609512192:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:3.0:1780298643.675950:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2400:x1866777609512192:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 356us (439us total) trans 4294969240 rc 0/0 00000100:00100000:3.0:1780298643.675958:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6186 00002000:00100000:3.0:1780298643.675962:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x12:0x0] (3649536->3650405) req@ffff9cf2556a9880 x1866777609512192/t4294969240(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 1360/448 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298643.675968:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2556a9880 with x1866777609512192 ext(3649536->3650405) 00002000:00010000:3.0:1780298643.675971:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2556a9880 00000100:00100000:3.0:1780298643.676702:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.676753:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609512320 00000100:00100000:1.0:1780298643.676758:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609512320 00000100:00100000:1.0:1780298643.676801:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9524 00000100:00100000:1.0:1780298643.676811:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255091180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2400:x1866777609512320:12345-192.168.203.25@tcp:16:fsx.0 00000001:00080000:0.0:1780298643.677526:0:8351:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0000: transno 4294969240 is committed 00000001:00080000:0.0:1780298643.677535:0:8351:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0000: transno 4294969238 is committed 00000100:00100000:1.0:1780298643.677613:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255091180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609512320:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 803us (910us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.677622:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9524 00000100:00100000:1.0:1780298643.678377:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.678418:0:9976:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609512448 00000100:00100000:0.0:1780298643.678423:0:9976:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609512448 00000100:00100000:0.0:1780298643.678455:0:9976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61988 00000100:00100000:0.0:1780298643.678461:0:9976:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+145:29360:x1866777609512448:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.678474:0:9976:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.678489:0:9976:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf000/0xaf35132e630344a6 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 75 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d358b expref: -99 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.678557:0:9976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae000/0xaf35132e630344ad lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 76 type: IBT flags: 0x40000000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.678571:0:9976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae000/0xaf35132e630344ad lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 76 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.678579:0:9976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae000/0xaf35132e630344ad lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 76 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1780298643.678602:0:9976:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 7844710 00000004:00200000:0.0:1780298643.678606:0:9976:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1780298643.678622:0:9976:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae000/0xaf35132e630344ad lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 76 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.678648:0:9976:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf000/0xaf35132e630344a6 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 76 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d358b expref: 147 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.678661:0:9976:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae000/0xaf35132e630344ad lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 75 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d358b expref: 146 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.678666:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298643.678668:0:9976:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1780298643.678670:0:9976:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae000, rc 0) 00000100:00100000:0.0:1780298643.678695:0:9976:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+146:29360:x1866777609512448:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 234us (317us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298643.678702:0:9976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61988 00000100:00100000:0.0:1780298643.679896:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.679923:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609512576 00000100:00100000:3.0:1780298643.679927:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609512576 00000100:00100000:3.0:1780298643.679950:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9525 00000100:00100000:3.0:1780298643.679955:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:29360:x1866777609512576:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.679962:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.679970:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d828f4f8/0xaf35132e630344b4 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3592 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.679986:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e6303449f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3584 expref: 9 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.679993:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(114 130) 00010000:00010000:3.0:1780298643.679996:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(115 130) 00010000:00010000:3.0:1780298643.679999:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d828f4f8/0xaf35132e630344b4 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [176128->184319] (req 176128->184319) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3592 expref: 7 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.680028:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e6303449f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3584 expref: 9 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.680042:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e6303449f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3584 expref: 9 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.680052:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255093800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604969600:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298643.680092:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.680759:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255093800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604969600:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298643.680775:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d828f4f8/0xaf35132e630344b4 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [176128->184319] (req 176128->184319) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3592 expref: 7 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.680784:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2e00, rc 0) 00000100:00100000:0.0:1780298643.680805:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609512576:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 851us (913us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298643.680810:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9525 00000100:00100000:1.0:1780298643.681016:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.681089:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609512704 00000100:00100000:2.0:1780298643.681096:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609512704 00010000:00010000:2.0:1780298643.681184:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e6303449f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3584 expref: 10 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.681229:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15110 00000100:00100000:2.0:1780298643.681244:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556aa300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609512704:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298643.681257:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:2.0:1780298643.681272:0:6461:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 0 -> 8 00010000:00010000:2.0:1780298643.681280:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e6303449f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3584 expref: 10 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.681311:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e6303449f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3584 expref: 10 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.681333:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e6303449f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3584 expref: 10 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.681347:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(115 130) 00010000:00010000:2.0:1780298643.681351:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(115 130) 00010000:00010000:2.0:1780298643.681360:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000d828f4f8/0xaf35132e630344b4 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 176128->184319) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3592 expref: 6 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.681370:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d828f4f8/0xaf35132e630344b4 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 176128->184319) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3592 expref: 6 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.681404:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000d828f4f8/0xaf35132e630344b4 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 176128->184319) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3592 expref: 6 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.681419:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b69f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6461:1866777604969728:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298643.681488:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4900 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.682196:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b69f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6461:1866777604969728:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298643.682207:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298643.682241:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556aa300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609512704:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1003us (1226us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.682250:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15110 00000100:00100000:3.0:1780298643.682486:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.682548:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609512832 00000100:00100000:2.0:1780298643.682555:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609512832 00002000:00100000:2.0:1780298643.682608:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x12:0x0] (176128->180223) req@ffff9cf255093100 x1866777609512832/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.682620:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255093100 with x1866777609512832 ext(176128->180223) 00002000:00010000:2.0:1780298643.682625:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf255093100 00000100:00100000:2.0:1780298643.682661:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6187 00000100:00100000:2.0:1780298643.682674:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255093100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609512832:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298643.682863:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255093100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609512832:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 190us (373us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298643.682872:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6187 00002000:00100000:2.0:1780298643.682882:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x12:0x0] (176128->180223) req@ffff9cf255093100 x1866777609512832/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.682891:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255093100 with x1866777609512832 ext(176128->180223) 00002000:00010000:2.0:1780298643.682896:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf255093100 00000100:00100000:1.0:1780298643.683700:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.683741:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609512960 00000100:00100000:3.0:1780298643.683745:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609512960 00002000:00100000:3.0:1780298643.683763:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (180224->184319) req@ffff9cf2556ab100 x1866777609512960/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298643.683770:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2556ab100 with x1866777609512960 ext(180224->184319) 00002000:00010000:3.0:1780298643.683773:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2556ab100 00000100:00100000:3.0:1780298643.683787:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6188 00000100:00100000:3.0:1780298643.683792:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556ab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609512960:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298643.683867:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556ab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609512960:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 75us (169us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1780298643.683872:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6188 00002000:00100000:3.0:1780298643.683876:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (180224->184319) req@ffff9cf2556ab100 x1866777609512960/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298643.683883:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2556ab100 with x1866777609512960 ext(180224->184319) 00002000:00010000:3.0:1780298643.683886:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2556ab100 00000100:00100000:1.0:1780298643.684642:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.684679:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609513088 00000100:00100000:2.0:1780298643.684683:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609513088 00000100:00100000:2.0:1780298643.684712:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61989 00000100:00100000:2.0:1780298643.684717:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556aaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+111:29360:x1866777609513088:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.684727:0:6477:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.684742:0:6477:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2000/0xaf35132e630344bb lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 76 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3599 expref: -99 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.684809:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2400/0xaf35132e630344c2 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 77 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.684824:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2400/0xaf35132e630344c2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 77 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.684835:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2400/0xaf35132e630344c2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 77 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298643.684855:0:6477:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 7844710 00000004:00200000:2.0:1780298643.684858:0:6477:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298643.684870:0:6477:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2400/0xaf35132e630344c2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 77 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.684888:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2000/0xaf35132e630344bb lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 77 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3599 expref: 113 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.684901:0:6477:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2400/0xaf35132e630344c2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 76 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3599 expref: 112 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.684907:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298643.684909:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298643.684911:0:6477:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2400, rc 0) 00000100:00100000:2.0:1780298643.684934:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556aaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+112:29360:x1866777609513088:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 217us (293us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.684938:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61989 00000100:00100000:1.0:1780298643.685868:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.685909:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609513216 00000100:00100000:3.0:1780298643.685915:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609513216 00000100:00100000:3.0:1780298643.685949:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9526 00000100:00100000:3.0:1780298643.685957:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556aa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609513216:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.685967:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.685978:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000fa1945a/0xaf35132e630344c9 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d35a0 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.686021:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255090700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604969856:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.686074:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856900 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.686848:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255090700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604969856:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.686867:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000fa1945a/0xaf35132e630344c9 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35a0 expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.686881:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000fa1945a/0xaf35132e630344c9 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35a0 expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.686888:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3000, rc 0) 00000100:00100000:3.0:1780298643.686910:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556aa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609513216:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 955us (1043us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.686915:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9526 00000100:00100000:1.0:1780298643.687734:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.687770:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609513344 00000100:00100000:2.0:1780298643.687775:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609513344 00000100:00100000:2.0:1780298643.687801:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9527 00000100:00100000:2.0:1780298643.687807:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609513344:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.687814:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.687822:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e630344d0 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d35a7 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.687848:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b69f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604969984:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.687880:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.688493:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b69f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604969984:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.688513:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e630344d0 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35a7 expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.688523:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e630344d0 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35a7 expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.688532:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2800, rc 0) 00000100:00100000:2.0:1780298643.688554:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609513344:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 748us (821us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.688559:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9527 00000100:00100000:1.0:1780298643.689388:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.689419:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609513472 00000100:00100000:2.0:1780298643.689424:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609513472 00000100:00100000:2.0:1780298643.689457:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9528 00000100:00100000:2.0:1780298643.689467:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556aad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609513472:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.689476:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.689487:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e630344d7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d35ae expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.689504:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(115 130) 00010000:00010000:2.0:1780298643.689508:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(115 130) 00010000:00010000:2.0:1780298643.689517:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e630344d7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1384448->1392639) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35ae expref: 10 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.689529:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e630344d7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1384448->1392639) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35ae expref: 10 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.689538:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3c00, rc 0) 00000100:00100000:2.0:1780298643.689563:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556aad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609513472:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 99us (176us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.689571:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9528 00000100:00100000:1.0:1780298643.690390:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.690432:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609513600 00000100:00100000:0.0:1780298643.690438:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609513600 00002000:00100000:0.0:1780298643.690465:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (1384448->1392639) req@ffff9cf2556a8a80 x1866777609513600/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.690480:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2556a8a80 with x1866777609513600 ext(1384448->1392639) 00002000:00010000:0.0:1780298643.690484:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2556a8a80 00000100:00100000:0.0:1780298643.690501:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6189 00000100:00100000:0.0:1780298643.690507:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609513600:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298643.690663:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609513600:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 155us (272us total) trans 0 rc 8192/8192 00000100:00100000:0.0:1780298643.690669:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6189 00002000:00100000:0.0:1780298643.690675:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (1384448->1392639) req@ffff9cf2556a8a80 x1866777609513600/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/8632 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 8192/8192 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.690681:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2556a8a80 with x1866777609513600 ext(1384448->1392639) 00002000:00010000:0.0:1780298643.690685:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2556a8a80 00000100:00100000:1.0:1780298643.692095:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.692159:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609513728 00000100:00100000:0.0:1780298643.692165:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609513728 00000100:00100000:0.0:1780298643.692203:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6190 00000100:00100000:0.0:1780298643.692211:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556ab800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609513728:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298643.692432:0:19943:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:0.0:1780298643.692436:0:19943:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298643.692438:0:19943:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298643.692441:0:19943:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298643.692443:0:19943:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 376 -> 384 00000100:00100000:0.0:1780298643.692478:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556ab800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609513728:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 267us (386us total) trans 4294969686 rc 0/0 00000100:00100000:0.0:1780298643.692486:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6190 00000100:00100000:1.0:1780298643.693803:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.693837:0:7411:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609513984 00000100:00100000:2.0:1780298643.693841:0:7411:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609513984 00000100:00100000:2.0:1780298643.693873:0:7411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61990 00000100:00100000:2.0:1780298643.693880:0:7411:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242863480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+112:29360:x1866777609513984:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:2.0:1780298643.693915:0:7411:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298643.693934:0:7411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2400/0xaf35132e630344c2 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 77 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3599 expref: 112 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.693951:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298643.693954:0:7411:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298643.694074:0:7411:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2200/0xaf35132e630344de lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 76 type: IBT flags: 0x40000000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.694094:0:7411:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae000/0xaf35132e630344ad lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 76 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d358b expref: 145 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.694113:0:7411:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2200/0xaf35132e630344de lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 76 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.694160:0:7411:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae000/0xaf35132e630344ad lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 76 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d358b expref: 145 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.694175:0:7411:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae000/0xaf35132e630344ad lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 76 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d358b expref: 145 pid: 9976 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.694190:0:7411:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b69f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7411:1866777604970112:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298643.694245:0:7411:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4900 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.694968:0:7411:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b69f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7411:1866777604970112:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298643.694986:0:7411:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2200/0xaf35132e630344de lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 76 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.695165:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.695255:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609514112 00000100:00100000:3.0:1780298643.695261:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609514112 00010000:00010000:3.0:1780298643.695303:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae000/0xaf35132e630344ad lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 76 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d358b expref: 146 pid: 9976 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.695331:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15111 00000100:00100000:3.0:1780298643.695340:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b5fc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+146:2400:x1866777609514112:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298643.695351:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298643.695357:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae000/0xaf35132e630344ad lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 77 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d358b expref: 146 pid: 9976 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.695370:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae000/0xaf35132e630344ad lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 77 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d358b expref: 146 pid: 9976 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.695386:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae000/0xaf35132e630344ad lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 77 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d358b expref: 146 pid: 9976 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.695399:0:20253:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298643.695403:0:20253:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2200/0xaf35132e630344de lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 76 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.695415:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2200/0xaf35132e630344de lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 76 type: IBT flags: 0x40290000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.695423:0:20253:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2200/0xaf35132e630344de lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 76 type: IBT flags: 0x40290000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.695447:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298643.695478:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b5fc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+145:2400:x1866777609514112:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 140us (315us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298643.695487:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15111 00010000:00010000:2.0:1780298643.695497:0:7411:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2200/0xaf35132e630344de lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 75 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.695510:0:7411:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2200/0xaf35132e630344de lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 75 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1780298643.695583:0:7411:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991054 00000004:00010000:2.0:1780298643.695622:0:7411:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 0000000081775bd6 reply state 0000000078e3efa0 transno 4294991055 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2200/0xaf35132e630344de lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 75 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.695644:0:7411:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2200/0xaf35132e630344de lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 75 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.695652:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298643.695655:0:7411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1780298643.695658:0:7411:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1780298643.695755:0:7411:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242863480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+114:29360:x1866777609513984:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1871us (1952us total) trans 4294991055 rc 0/0 00000100:00100000:2.0:1780298643.695765:0:7411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61990 00000020:00080000:2.0:1780298643.695770:0:7411:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1780298643.696547:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.696587:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609514240 00000100:00100000:0.0:1780298643.696591:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609514240 00000100:00100000:0.0:1780298643.696615:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9529 00000100:00100000:0.0:1780298643.696620:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242861180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:29360:x1866777609514240:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.696634:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.696645:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630344e5 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d35b5 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.696662:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000052b89172/0xaf35132e63034498 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d357d expref: 10 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.696668:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(155 177) 00010000:00010000:0.0:1780298643.696672:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(156 177) 00010000:00010000:0.0:1780298643.696675:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630344e5 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35b5 expref: 12 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.696690:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000052b89172/0xaf35132e63034498 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d357d expref: 10 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.696697:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000052b89172/0xaf35132e63034498 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d357d expref: 10 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.696703:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b41880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604970240:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298643.696736:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.697428:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b41880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604970240:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298643.697440:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630344e5 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35b5 expref: 12 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.697451:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf000, rc 0) 00000100:00100000:0.0:1780298643.697474:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242861180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:29360:x1866777609514240:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 854us (928us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298643.697479:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9529 00000100:00100000:1.0:1780298643.697672:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.697715:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609514368 00000100:00100000:0.0:1780298643.697720:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609514368 00010000:00010000:0.0:1780298643.697738:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000052b89172/0xaf35132e63034498 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d357d expref: 11 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.697764:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15112 00000100:00100000:0.0:1780298643.697771:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242863b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609514368:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298643.697781:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298643.697798:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000052b89172/0xaf35132e63034498 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d357d expref: 11 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.697822:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000052b89172/0xaf35132e63034498 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d357d expref: 11 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.697840:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000052b89172/0xaf35132e63034498 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d357d expref: 11 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.697852:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(156 177) 00010000:00010000:0.0:1780298643.697856:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(156 177) 00010000:00010000:0.0:1780298643.697862:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630344e5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35b5 expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.697872:0:6463:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630344e5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35b5 expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.697894:0:6463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630344e5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35b5 expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.697906:0:6463:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b41880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6463:1866777604970368:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298643.697942:0:6463:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.698676:0:6463:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b41880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6463:1866777604970368:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298643.698694:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298643.698757:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.698761:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242863b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609514368:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 978us (1077us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298643.698774:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15112 00000100:00100000:3.0:1780298643.698791:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609514496 00000100:00100000:3.0:1780298643.698795:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609514496 00000100:00100000:3.0:1780298643.698832:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9530 00000100:00100000:3.0:1780298643.698843:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242861c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609514496:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.698854:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.698897:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000fa1945a/0xaf35132e630344ec lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d35bc expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.698946:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d828f4f8/0xaf35132e630344b4 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 176128->184319) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3592 expref: 8 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.698958:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e630344d7 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1384448->1392639) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35ae expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.698966:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(115 130) 00010000:00010000:2.0:1780298643.698969:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(115 130) 00010000:00010000:2.0:1780298643.698973:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000fa1945a/0xaf35132e630344ec lrc: 5/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35bc expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.698994:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e630344d7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1384448->1392639) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35ae expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.699006:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e630344d7 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1384448->1392639) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35ae expref: 9 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.699022:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b69f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604970496:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298643.699071:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d828f4f8/0xaf35132e630344b4 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 176128->184319) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3592 expref: 8 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.699082:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d828f4f8/0xaf35132e630344b4 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 176128->184319) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3592 expref: 8 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.699092:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b69880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604970624:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298643.699117:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.699785:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b69f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604970496:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298643.699794:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b69880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604970624:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298643.699810:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000fa1945a/0xaf35132e630344ec lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35bc expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.699821:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3000, rc 0) 00000100:00100000:2.0:1780298643.699852:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242861c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:29360:x1866777609514496:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1013us (1097us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.699859:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9530 00000100:00100000:1.0:1780298643.700087:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.700169:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609514624 00000100:00100000:3.0:1780298643.700174:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609514624 00010000:00010000:3.0:1780298643.700191:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e630344d7 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1384448->1392639) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35ae expref: 10 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.700207:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.700248:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609514752 00000100:00100000:3.0:1780298643.700251:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609514752 00010000:00010000:3.0:1780298643.700263:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d828f4f8/0xaf35132e630344b4 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 176128->184319) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3592 expref: 8 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.700297:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15113 00000100:00100000:0.0:1780298643.700307:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15114 00000100:00100000:3.0:1780298643.700308:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242862d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609514624:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298643.700322:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:0.0:1780298643.700328:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609514752:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298643.700337:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e630344d7 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1384448->1392639) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35ae expref: 10 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.700342:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298643.700347:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e630344d7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 1384448->1392639) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35ae expref: 10 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.700358:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e630344d7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1384448->1392639) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35ae expref: 10 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.700464:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1780298643.700470:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d828f4f8/0xaf35132e630344b4 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 176128->184319) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3592 expref: 8 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.700495:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d828f4f8/0xaf35132e630344b4 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 176128->184319) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3592 expref: 8 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.700516:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d828f4f8/0xaf35132e630344b4 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 176128->184319) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3592 expref: 8 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.700523:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242862d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609514624:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 218us (435us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.700537:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15113 00010000:00010000:0.0:1780298643.700543:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(115 130) 00010000:00010000:0.0:1780298643.700548:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(115 130) 00010000:00010000:0.0:1780298643.700556:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000000fa1945a/0xaf35132e630344ec lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35bc expref: 7 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.700567:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000fa1945a/0xaf35132e630344ec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35bc expref: 7 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.700599:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000000fa1945a/0xaf35132e630344ec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35bc expref: 7 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.700614:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b41880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:7400:1866777604970752:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298643.700675:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.701348:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b41880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:7400:1866777604970752:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298643.701359:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298643.701387:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609514752:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1067us (1180us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298643.701397:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15114 00000100:00100000:1.0:1780298643.701468:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.701550:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609514880 00000100:00100000:3.0:1780298643.701554:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609514880 00002000:00010000:3.0:1780298643.701573:0:19928:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4697678->18446744073709551615) 00002000:00100000:3.0:1780298643.701577:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24310ed80 with x1866777609514880 ext(4697678->18446744073709551615) 00000100:00100000:1.0:1780298643.701582:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00002000:00010000:3.0:1780298643.701589:0:19928:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24310ed80. 00000100:00100000:3.0:1780298643.701614:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609515008 00000100:00100000:3.0:1780298643.701618:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609515008 00000100:00100000:2.0:1780298643.701625:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6191 00002000:00010000:3.0:1780298643.701629:0:19928:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4194304->18446744073709551615) 00002000:00100000:3.0:1780298643.701633:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24310c000 with x1866777609515008 ext(4194304->18446744073709551615) 00000100:00100000:2.0:1780298643.701639:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609514880:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:3.0:1780298643.701641:0:19928:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24310c000. 00000100:00100000:3.0:1780298643.701669:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6192 00000100:00100000:3.0:1780298643.701681:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609515008:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:3.0:1780298643.701779:0:19928:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 3650406 -> 4194304 00002000:00010000:3.0:1780298643.701782:0:19928:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:3.0:1780298643.701784:0:19928:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:3.0:1780298643.701785:0:19928:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:3.0:1780298643.701787:0:19928:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 8 -> 8 00002000:00010000:2.0:1780298643.701799:0:8363:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4697678 00002000:00010000:2.0:1780298643.701804:0:8363:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298643.701806:0:8363:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298643.701808:0:8363:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298643.701811:0:8363:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 384 -> 384 00000100:00100000:3.0:1780298643.701813:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609515008:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 136us (234us total) trans 4294969241 rc 0/0 00000100:00100000:3.0:1780298643.701819:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6192 00002000:00010000:3.0:1780298643.701822:0:19928:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4194304->18446744073709551615) 00002000:00100000:3.0:1780298643.701824:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24310c000 with x1866777609515008 ext(4194304->18446744073709551615) 00002000:00010000:3.0:1780298643.701828:0:19928:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24310c000. 00000100:00100000:2.0:1780298643.701844:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2399:x1866777609514880:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 208us (374us total) trans 4294969687 rc 0/0 00000100:00100000:2.0:1780298643.701852:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6191 00002000:00010000:2.0:1780298643.701856:0:8363:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4697678->18446744073709551615) 00002000:00100000:2.0:1780298643.701859:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24310ed80 with x1866777609514880 ext(4697678->18446744073709551615) 00002000:00010000:2.0:1780298643.701864:0:8363:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24310ed80. 00000100:00100000:1.0:1780298643.710150:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.710197:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609515136 00000100:00100000:2.0:1780298643.710206:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609515136 00000100:00100000:1.0:1780298643.710211:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00002000:00100000:2.0:1780298643.710241:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (0->4694015) req@ffff9cf2425f0380 x1866777609515136/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 504/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00000100:00100000:0.0:1780298643.710246:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609515264 00000100:00100000:0.0:1780298643.710252:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609515264 00002000:00100000:2.0:1780298643.710255:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2425f0380 with x1866777609515136 ext(0->4694015) 00002000:00010000:2.0:1780298643.710263:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2425f0380 00002000:00100000:0.0:1780298643.710278:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (4190208->4194303) req@ffff9cf2425f2300 x1866777609515264/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00000100:00100000:2.0:1780298643.710301:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6193 00002000:00100000:0.0:1780298643.710308:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2425f2300 with x1866777609515264 ext(4190208->4194303) 00002000:00010000:0.0:1780298643.710314:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2425f2300 00000100:00100000:2.0:1780298643.710316:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609515136:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298643.710338:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6194 00000100:00100000:0.0:1780298643.710345:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609515264:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298643.710481:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609515264:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 136us (270us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298643.710487:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6194 00002000:00100000:0.0:1780298643.710492:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (4190208->4194303) req@ffff9cf2425f2300 x1866777609515264/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.710498:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2425f2300 with x1866777609515264 ext(4190208->4194303) 00002000:00010000:0.0:1780298643.710501:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2425f2300 00000100:00100000:2.0:1780298643.721039:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609515136:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 10714us (10907us total) trans 0 rc 4194304/4194304 00000100:00100000:2.0:1780298643.721049:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6193 00002000:00100000:2.0:1780298643.721061:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (0->4694015) req@ffff9cf2425f0380 x1866777609515136/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 504/440 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4194304/4194304 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.721069:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2425f0380 with x1866777609515136 ext(0->4694015) 00002000:00010000:2.0:1780298643.721075:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2425f0380 00000100:00100000:2.0:1780298643.731969:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.732030:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609515392 00000100:00100000:0.0:1780298643.732041:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609515392 00002000:00100000:0.0:1780298643.732083:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (4694016->4698111) req@ffff9cf242b69880 x1866777609515392/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.732092:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b69880 with x1866777609515392 ext(4694016->4698111) 00002000:00010000:0.0:1780298643.732097:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf242b69880 00000100:00100000:0.0:1780298643.732120:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6195 00000100:00100000:0.0:1780298643.732146:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b69880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2397:x1866777609515392:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298643.732326:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b69880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2397:x1866777609515392:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 197us (365us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298643.732333:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6195 00002000:00100000:0.0:1780298643.732338:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (4694016->4698111) req@ffff9cf242b69880 x1866777609515392/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.732349:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b69880 with x1866777609515392 ext(4694016->4698111) 00002000:00010000:0.0:1780298643.732352:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf242b69880 00000100:00100000:3.0:1780298643.732401:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.732434:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609515520 00000100:00100000:0.0:1780298643.732437:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609515520 00002000:00100000:0.0:1780298643.732451:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x12:0x0] (0->4194303) req@ffff9cf255092680 x1866777609515520/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.732457:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255092680 with x1866777609515520 ext(0->4194303) 00002000:00010000:0.0:1780298643.732460:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf255092680 00000100:00100000:0.0:1780298643.732474:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6196 00000100:00100000:0.0:1780298643.732478:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255092680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609515520:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298643.733869:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.733905:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609515648 00000100:00100000:1.0:1780298643.733909:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609515648 00002000:00100000:1.0:1780298643.733927:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (4689920->4697677) req@ffff9cf242b69f80 x1866777609515648/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 8248/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.733934:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b69f80 with x1866777609515648 ext(4689920->4697677) 00002000:00010000:1.0:1780298643.733937:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf242b69f80 00000100:00100000:1.0:1780298643.733952:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6197 00000100:00100000:1.0:1780298643.733958:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b69f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2398:x1866777609515648:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:1.0:1780298643.734414:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b69f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2398:x1866777609515648:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 456us (546us total) trans 4294969688 rc 0/0 00000100:00100000:1.0:1780298643.734420:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6197 00002000:00100000:1.0:1780298643.734425:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (4689920->4697677) req@ffff9cf242b69f80 x1866777609515648/t4294969688(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 8248/448 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.734432:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b69f80 with x1866777609515648 ext(4689920->4697677) 00002000:00010000:1.0:1780298643.734436:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf242b69f80 00000100:00100000:1.0:1780298643.735341:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.735391:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609515776 00000100:00100000:2.0:1780298643.735396:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609515776 00000100:00100000:2.0:1780298643.735435:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9531 00000100:00100000:2.0:1780298643.735443:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2398:x1866777609515776:12345-192.168.203.25@tcp:16:fsx.0 00000001:00080000:1.0:1780298643.736228:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969688 is committed 00000001:00080000:1.0:1780298643.736240:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969686 is committed 00000100:00100000:2.0:1780298643.736337:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609515776:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 893us (996us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.736346:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9531 00000100:00100000:1.0:1780298643.737530:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.737586:0:6476:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609515904 00000100:00100000:2.0:1780298643.737593:0:6476:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609515904 00000100:00100000:2.0:1780298643.737647:0:6476:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61991 00000100:00100000:2.0:1780298643.737657:0:6476:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+114:29360:x1866777609515904:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.737674:0:6476:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.737693:0:6476:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2800/0xaf35132e630344f3 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 76 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d35c3 expref: -99 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.737788:0:6476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2000/0xaf35132e630344fa lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 77 type: IBT flags: 0x40000000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.737805:0:6476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2000/0xaf35132e630344fa lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 77 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.737816:0:6476:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2000/0xaf35132e630344fa lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 77 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298643.737842:0:6476:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 8891982 00000004:00200000:2.0:1780298643.737846:0:6476:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298643.737864:0:6476:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2000/0xaf35132e630344fa lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 77 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.737891:0:6476:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2800/0xaf35132e630344f3 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 77 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35c3 expref: 116 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.737911:0:6476:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2000/0xaf35132e630344fa lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 76 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35c3 expref: 115 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.737921:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298643.737925:0:6476:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298643.737930:0:6476:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2000, rc 0) 00000100:00100000:2.0:1780298643.737966:0:6476:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+115:29360:x1866777609515904:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 311us (441us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.737973:0:6476:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61991 00000100:00100000:2.0:1780298643.739953:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.740006:0:9976:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609516032 00000100:00100000:0.0:1780298643.740015:0:9976:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609516032 00000100:00100000:0.0:1780298643.740061:0:9976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61992 00000100:00100000:0.0:1780298643.740072:0:9976:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b6bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+145:29360:x1866777609516032:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.740089:0:9976:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.740105:0:9976:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baee00/0xaf35132e63034501 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 77 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d35ca expref: -99 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.740210:0:9976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baea00/0xaf35132e63034508 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 78 type: IBT flags: 0x40000000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.740225:0:9976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baea00/0xaf35132e63034508 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 78 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.740237:0:9976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baea00/0xaf35132e63034508 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 78 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1780298643.740263:0:9976:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 8891982 00000004:00200000:0.0:1780298643.740268:0:9976:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1780298643.740301:0:9976:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baea00/0xaf35132e63034508 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 78 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.740320:0:9976:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baee00/0xaf35132e63034501 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 78 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35ca expref: 147 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.740332:0:9976:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baea00/0xaf35132e63034508 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 77 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35ca expref: 146 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.740337:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298643.740340:0:9976:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1780298643.740342:0:9976:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baea00, rc 0) 00000100:00100000:0.0:1780298643.740376:0:9976:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b6bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+146:29360:x1866777609516032:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 308us (429us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298643.740380:0:9976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61992 00000100:00100000:1.0:1780298643.741933:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.741982:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609516160 00000100:00100000:2.0:1780298643.741991:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609516160 00000100:00100000:2.0:1780298643.742030:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9532 00000100:00100000:2.0:1780298643.742039:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609516160:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.742052:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.742065:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002de18a35/0xaf35132e6303450f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d35d1 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.742116:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b68700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604970880:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.742232:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.742330:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.742413:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609516288 00000100:00100000:1.0:1780298643.742417:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609516288 00000100:00100000:1.0:1780298643.742445:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9533 00000100:00100000:1.0:1780298643.742450:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b41880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609516288:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.742458:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.742467:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c18c87d/0xaf35132e63034516 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d35d8 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.742490:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604971008:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.742527:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a840 going to sleep for 11 seconds 00002000:00010000:1.0:1780298643.743309:0:19992:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks: 384 -> 400 00000100:00100000:1.0:1780298643.743324:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b68700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604970880:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.743345:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002de18a35/0xaf35132e6303450f lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35d1 expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.743362:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002de18a35/0xaf35132e6303450f lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35d1 expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.743371:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf2445bfa00, rc 0) 00000100:00100000:1.0:1780298643.743399:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609516160:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1362us (1471us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.743404:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9532 00000100:00100000:0.0:1780298643.743835:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255092680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609515520:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 11354us (11510us total) trans 0 rc 4194304/4194304 00000100:00100000:0.0:1780298643.743846:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6196 00002000:00100000:0.0:1780298643.743862:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x12:0x0] (0->4194303) req@ffff9cf255092680 x1866777609515520/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/440 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4194304/4194304 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.743873:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255092680 with x1866777609515520 ext(0->4194303) 00002000:00010000:0.0:1780298643.743879:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf255092680 00000100:00100000:1.0:1780298643.743950:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604971008:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.743969:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004c18c87d/0xaf35132e63034516 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35d8 expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.743983:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c18c87d/0xaf35132e63034516 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35d8 expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.743991:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34841ce00, rc 0) 00000100:00100000:1.0:1780298643.744035:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b41880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609516288:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1583us (1704us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.744043:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9533 00000100:00100000:2.0:1780298643.745662:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.745696:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.745715:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609516544 00000100:00100000:0.0:1780298643.745721:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609516544 00000100:00100000:1.0:1780298643.745735:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609516416 00000100:00100000:1.0:1780298643.745741:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609516416 00000100:00100000:0.0:1780298643.745758:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9534 00000100:00100000:0.0:1780298643.745766:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b68700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609516544:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.745778:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298643.745786:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9535 00010000:00010000:0.0:1780298643.745789:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a3f03a/0xaf35132e6303451d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d35e6 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.745799:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b6b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2397:x1866777609516416:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.745808:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.745819:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000213a6cb7/0xaf35132e63034524 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d35df expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.745832:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b40000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604971136:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.745855:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604971264:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.745887:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417e40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.745890:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a840 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.746652:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b40000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604971136:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.746669:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000001a3f03a/0xaf35132e6303451d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35e6 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.746682:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a3f03a/0xaf35132e6303451d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35e6 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.746689:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae600, rc 0) 00000100:00100000:1.0:1780298643.746708:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604971264:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.746716:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b68700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609516544:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 951us (1056us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.746721:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9534 00010000:00010000:1.0:1780298643.746730:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000213a6cb7/0xaf35132e63034524 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35df expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.746744:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000213a6cb7/0xaf35132e63034524 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35df expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.746755:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34841c800, rc 0) 00000100:00100000:1.0:1780298643.746786:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b6b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2397:x1866777609516416:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 992us (1088us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.746795:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9535 00000100:00100000:2.0:1780298643.747770:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.747810:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609516672 00000100:00100000:1.0:1780298643.747815:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609516672 00000100:00100000:1.0:1780298643.747845:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9536 00000100:00100000:1.0:1780298643.747853:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b6a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2397:x1866777609516672:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.747863:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.747874:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000004503c2f/0xaf35132e6303452b lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d35ed expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.747905:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.747909:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604971392:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.747944:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609516800 00000100:00100000:0.0:1780298643.747950:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609516800 00000100:00100000:1.0:1780298643.747961:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a840 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.747985:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9537 00000100:00100000:0.0:1780298643.747993:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b68380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609516800:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.748001:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.748011:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63034532 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d35f4 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.748037:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b40000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604971520:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.748067:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417e40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.748692:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604971392:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.748708:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000004503c2f/0xaf35132e6303452b lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35ed expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.748724:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000004503c2f/0xaf35132e6303452b lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35ed expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.748735:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34841ca00, rc 0) 00000100:00100000:0.0:1780298643.748747:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b40000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604971520:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.748767:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b6a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2397:x1866777609516672:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 913us (995us total) trans 0 rc 301/301 00010000:00010000:0.0:1780298643.748776:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63034532 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35f4 expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.748778:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9536 00010000:00010000:0.0:1780298643.748801:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63034532 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35f4 expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.748814:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae000, rc 0) 00000100:00100000:0.0:1780298643.748864:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b68380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609516800:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 868us (955us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.748874:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9537 00000100:00100000:2.0:1780298643.749744:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.749769:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.749777:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609516928 00000100:00100000:3.0:1780298643.749782:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609516928 00000100:00100000:3.0:1780298643.749804:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609517056 00000100:00100000:3.0:1780298643.749806:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609517056 00000100:00100000:3.0:1780298643.749825:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9538 00000100:00100000:3.0:1780298643.749832:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254919500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609516928:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.749840:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298643.749846:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9539 00010000:00010000:3.0:1780298643.749847:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e63034539 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d35fb expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.749865:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254918700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609517056:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298643.749875:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255090700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604971648:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.749877:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.749890:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b35ce980/0xaf35132e63034540 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3602 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.749915:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.749937:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b40000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604971776:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.749991:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417e40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.750717:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255090700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604971648:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.750740:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e63034539 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35fb expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.750755:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e63034539 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35fb expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.750763:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b40000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604971776:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.750767:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157400, rc 0) 00010000:00010000:0.0:1780298643.750792:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b35ce980/0xaf35132e63034540 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3602 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.750802:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254919500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609516928:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 969us (1057us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.750812:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9538 00010000:00010000:0.0:1780298643.750818:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b35ce980/0xaf35132e63034540 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3602 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.750831:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bafa00, rc 0) 00000100:00100000:0.0:1780298643.750879:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254918700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609517056:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1019us (1107us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.750887:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9539 00000100:00100000:2.0:1780298643.752094:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.752114:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609517184 00000100:00100000:3.0:1780298643.752119:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609517184 00000100:00100000:3.0:1780298643.752171:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9540 00000100:00100000:3.0:1780298643.752180:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254919c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609517184:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.752190:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.752199:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c841b52/0xaf35132e63034547 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3609 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.752259:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000000fa1945a/0xaf35132e630344ec lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35bc expref: 7 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.752279:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(115 130) 00010000:00010000:1.0:1780298643.752293:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(116 130) 00010000:00010000:1.0:1780298643.752300:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c841b52/0xaf35132e63034547 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2084864->2093055] (req 2084864->2093055) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3609 expref: 10 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.752321:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000000fa1945a/0xaf35132e630344ec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35bc expref: 7 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.752333:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000000fa1945a/0xaf35132e630344ec lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35bc expref: 7 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.752341:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604971904:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298643.752377:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.753091:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604971904:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298643.753110:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009c841b52/0xaf35132e63034547 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2084864->2093055] (req 2084864->2093055) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3609 expref: 10 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.753148:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157600, rc 0) 00000100:00100000:1.0:1780298643.753184:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254919c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609517184:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1005us (1089us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.753197:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9540 00000100:00100000:2.0:1780298643.757515:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.757562:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609517312 00000100:00100000:1.0:1780298643.757573:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609517312 00010000:00010000:1.0:1780298643.757612:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000000fa1945a/0xaf35132e630344ec lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35bc expref: 8 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.757648:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15115 00000100:00100000:1.0:1780298643.757657:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609517312:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298643.757670:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298643.757687:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000000fa1945a/0xaf35132e630344ec lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35bc expref: 8 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.757697:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000000fa1945a/0xaf35132e630344ec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35bc expref: 8 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.757716:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000fa1945a/0xaf35132e630344ec lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35bc expref: 8 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.757728:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(116 130) 00010000:00010000:1.0:1780298643.757731:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(116 130) 00010000:00010000:1.0:1780298643.757738:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009c841b52/0xaf35132e63034547 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2084864->35639295] (req 2084864->2093055) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3609 expref: 9 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.757746:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c841b52/0xaf35132e63034547 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2084864->35639295] (req 2084864->2093055) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3609 expref: 9 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.757774:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000009c841b52/0xaf35132e63034547 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2084864->35639295] (req 2084864->2093055) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3609 expref: 9 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.757784:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f1f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777604972032:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298643.757851:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.758765:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f1f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777604972032:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298643.758782:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298643.758814:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609517312:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1158us (1311us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.758822:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15115 00000100:00100000:2.0:1780298643.759048:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.759115:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609517440 00000100:00100000:3.0:1780298643.759158:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609517440 00002000:00100000:2.0:1780298643.759244:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (2084864->2088959) req@ffff9cf25491b480 x1866777609517440/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.759255:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25491b480 with x1866777609517440 ext(2084864->2088959) 00002000:00010000:2.0:1780298643.759260:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf25491b480 00000100:00100000:2.0:1780298643.759279:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6198 00000100:00100000:2.0:1780298643.759297:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609517440:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298643.759416:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609517440:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 119us (368us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298643.759423:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6198 00002000:00100000:2.0:1780298643.759430:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (2084864->2088959) req@ffff9cf25491b480 x1866777609517440/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.759439:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25491b480 with x1866777609517440 ext(2084864->2088959) 00002000:00010000:2.0:1780298643.759443:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf25491b480 00000100:00100000:2.0:1780298643.760354:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.760397:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609517568 00000100:00100000:3.0:1780298643.760401:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609517568 00002000:00100000:3.0:1780298643.760420:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (2088960->2093055) req@ffff9cf254918a80 x1866777609517568/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298643.760426:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf254918a80 with x1866777609517568 ext(2088960->2093055) 00002000:00010000:3.0:1780298643.760429:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf254918a80 00000100:00100000:3.0:1780298643.760453:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6199 00000100:00100000:3.0:1780298643.760461:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254918a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609517568:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298643.760533:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254918a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609517568:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 74us (181us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1780298643.760537:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6199 00002000:00100000:3.0:1780298643.760541:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (2088960->2093055) req@ffff9cf254918a80 x1866777609517568/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298643.760546:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf254918a80 with x1866777609517568 ext(2088960->2093055) 00002000:00010000:3.0:1780298643.760549:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf254918a80 00000100:00100000:0.0:1780298643.761779:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.761817:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609517696 00000100:00100000:1.0:1780298643.761821:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609517696 00000100:00100000:1.0:1780298643.761847:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9541 00000100:00100000:1.0:1780298643.761853:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b40000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609517696:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.761860:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.761873:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000004503c2f/0xaf35132e6303454e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3610 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.761909:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604972160:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.761970:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a840 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.762160:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.762210:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609517824 00000100:00100000:2.0:1780298643.762216:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609517824 00000100:00100000:2.0:1780298643.762252:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9542 00000100:00100000:2.0:1780298643.762264:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255090700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609517824:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.762272:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.762293:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63034555 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3617 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.762320:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25491ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604972288:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.762358:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2449b4480 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.762846:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604972160:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.762872:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000004503c2f/0xaf35132e6303454e lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3610 expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.762898:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000004503c2f/0xaf35132e6303454e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3610 expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.762910:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34841ca00, rc 0) 00000100:00100000:1.0:1780298643.762940:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b40000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609517696:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1087us (1161us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.762947:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9541 00000100:00100000:2.0:1780298643.763106:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25491ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604972288:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.763152:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63034555 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3617 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.763166:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63034555 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3617 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.763177:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf2445be600, rc 0) 00000100:00100000:2.0:1780298643.763205:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255090700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609517824:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 944us (1048us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.763211:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9542 00000100:00100000:0.0:1780298643.764393:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.764430:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609518080 00000100:00100000:0.0:1780298643.764435:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.764437:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609518080 00000100:00100000:2.0:1780298643.764475:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609517952 00000100:00100000:2.0:1780298643.764478:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609517952 00000100:00100000:1.0:1780298643.764483:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9543 00000100:00100000:1.0:1780298643.764496:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609518080:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.764508:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298643.764509:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9544 00010000:00010000:1.0:1780298643.764520:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000213a6cb7/0xaf35132e6303455c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3625 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.764525:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2400:x1866777609517952:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.764539:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.764550:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e63034563 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d361e expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.764567:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604972416:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.764586:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255093800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604972544:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.764639:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.764639:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.765590:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604972416:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.765616:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000213a6cb7/0xaf35132e6303455c lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3625 expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.765634:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000213a6cb7/0xaf35132e6303455c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3625 expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.765645:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34841c800, rc 0) 00000100:00100000:1.0:1780298643.765679:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609518080:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1187us (1288us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.765688:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9543 00000100:00100000:3.0:1780298643.765769:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255093800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604972544:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.765787:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e63034563 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d361e expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.765801:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e63034563 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d361e expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.765811:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157400, rc 0) 00000100:00100000:3.0:1780298643.765840:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2400:x1866777609517952:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1321us (1406us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.765847:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9544 00000100:00100000:0.0:1780298643.767065:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.767119:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609518336 00000100:00100000:2.0:1780298643.767146:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609518336 00000100:00100000:2.0:1780298643.767189:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61993 00000100:00100000:2.0:1780298643.767199:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b40a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+146:29360:x1866777609518336:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:2.0:1780298643.767254:0:6477:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298643.767278:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baea00/0xaf35132e63034508 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 78 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35ca expref: 146 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.767303:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298643.767307:0:6477:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298643.767455:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf000/0xaf35132e6303456a lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 77 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.767472:0:6477:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2000/0xaf35132e630344fa lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 77 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35c3 expref: 114 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.767491:0:6477:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf000/0xaf35132e6303456a lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 77 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.767517:0:6477:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2000/0xaf35132e630344fa lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 77 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35c3 expref: 114 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.767531:0:6477:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2000/0xaf35132e630344fa lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 77 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35c3 expref: 114 pid: 6476 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.767544:0:6477:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25491ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6477:1866777604972672:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298643.767596:0:6477:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff49c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.768508:0:6477:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25491ad80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6477:1866777604972672:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298643.768527:0:6477:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf000/0xaf35132e6303456a lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 77 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.768591:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.768634:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609518464 00000100:00100000:1.0:1780298643.768639:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609518464 00010000:00010000:1.0:1780298643.768659:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2000/0xaf35132e630344fa lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 77 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35c3 expref: 115 pid: 6476 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.768694:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15116 00000100:00100000:1.0:1780298643.768702:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b41f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+115:2399:x1866777609518464:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298643.768712:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298643.768717:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2000/0xaf35132e630344fa lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 78 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35c3 expref: 115 pid: 6476 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.768729:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2000/0xaf35132e630344fa lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 78 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35c3 expref: 115 pid: 6476 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.768744:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2000/0xaf35132e630344fa lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 78 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35c3 expref: 115 pid: 6476 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.768758:0:20253:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298643.768762:0:20253:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf000/0xaf35132e6303456a lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 77 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.768775:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf000/0xaf35132e6303456a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 77 type: IBT flags: 0x40290000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.768783:0:20253:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf000/0xaf35132e6303456a lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 77 type: IBT flags: 0x40290000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.768804:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1780298643.768819:0:6477:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf000/0xaf35132e6303456a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 76 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.768830:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf000/0xaf35132e6303456a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 76 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.768837:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b41f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+114:2399:x1866777609518464:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 135us (246us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.768847:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15116 00000001:00080000:3.0:1780298643.768900:0:6477:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991056 00000004:00010000:3.0:1780298643.768931:0:6477:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000e6f4a90e reply state 000000008f92792e transno 4294991057 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf000/0xaf35132e6303456a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 76 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.768983:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf000/0xaf35132e6303456a lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 76 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.768990:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298643.768992:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1780298643.768995:0:6477:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1780298643.769059:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b40a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+148:29360:x1866777609518336:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1861us (1995us total) trans 4294991057 rc 0/0 00000100:00100000:3.0:1780298643.769066:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61993 00000020:00080000:3.0:1780298643.769070:0:6477:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1780298643.769965:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.770007:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609518592 00000100:00100000:3.0:1780298643.770012:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609518592 00000100:00100000:3.0:1780298643.770044:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9545 00000100:00100000:3.0:1780298643.770053:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b42680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:29360:x1866777609518592:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.770064:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.770075:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034571 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d362c expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.770098:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630344e5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35b5 expref: 6 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.770107:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(156 177) 00010000:00010000:3.0:1780298643.770111:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(157 177) 00010000:00010000:3.0:1780298643.770116:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034571 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d362c expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.770157:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630344e5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35b5 expref: 6 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.770170:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630344e5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35b5 expref: 6 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.770181:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255093800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604972800:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298643.770224:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.771028:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255093800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604972800:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298643.771046:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034571 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d362c expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.771059:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157c00, rc 0) 00000100:00100000:3.0:1780298643.771090:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b42680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609518592:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1038us (1129us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298643.771097:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9545 00000100:00100000:0.0:1780298643.775568:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.775615:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609518720 00000100:00100000:2.0:1780298643.775625:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609518720 00010000:00010000:2.0:1780298643.775658:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630344e5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35b5 expref: 7 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.775689:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15117 00000100:00100000:2.0:1780298643.775699:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b41c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609518720:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298643.775715:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298643.775736:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630344e5 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35b5 expref: 7 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.775750:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630344e5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35b5 expref: 7 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.775772:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630344e5 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d35b5 expref: 7 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.775786:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(157 177) 00010000:00010000:2.0:1780298643.775789:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(157 177) 00010000:00010000:2.0:1780298643.775795:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034571 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d362c expref: 8 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.775804:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034571 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d362c expref: 8 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.775842:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034571 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d362c expref: 8 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.775856:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25491ad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777604972928:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298643.775926:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff49c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.776903:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25491ad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777604972928:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298643.776923:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298643.776956:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b41c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609518720:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1259us (1395us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.776963:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15117 00000100:00100000:0.0:1780298643.777033:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.777079:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609518848 00000100:00100000:2.0:1780298643.777085:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609518848 00000100:00100000:2.0:1780298643.777121:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9546 00000100:00100000:2.0:1780298643.777156:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609518848:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.777169:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.777181:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63034578 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3633 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.777199:0:19992:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000009c841b52/0xaf35132e63034547 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2084864->35639295] (req 2084864->2093055) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3609 expref: 11 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.777204:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(116 130) 00010000:00010000:2.0:1780298643.777215:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(117 130) 00010000:00010000:2.0:1780298643.777218:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63034578 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3633 expref: 11 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.777243:0:19992:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000009c841b52/0xaf35132e63034547 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2084864->35639295] (req 2084864->2093055) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3609 expref: 11 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.777256:0:19992:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000009c841b52/0xaf35132e63034547 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2084864->35639295] (req 2084864->2093055) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3609 expref: 11 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.777267:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25491ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604973056:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298643.777308:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff49c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.778012:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25491ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604973056:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298643.778028:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63034578 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3633 expref: 11 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.778041:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf2445be600, rc 0) 00000100:00100000:2.0:1780298643.778073:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:29360:x1866777609518848:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 920us (1041us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.778080:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9546 00000100:00100000:0.0:1780298643.778470:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.778553:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609518976 00000100:00100000:1.0:1780298643.778561:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609518976 00002000:00100000:1.0:1780298643.778619:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x12:0x0] (2084864->2093055) req@ffff9cf242b43480 x1866777609518976/t0(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 8680/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.778630:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b43480 with x1866777609518976 ext(2084864->2093055) 00010000:00010000:1.0:1780298643.778639:0:19943:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000009c841b52/0xaf35132e63034547 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2084864->35639295] (req 2084864->2093055) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3609 expref: 11 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.778654:0:19943:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000009c841b52/0xaf35132e63034547 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2084864->35639295] (req 2084864->2093055) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3609 expref: 11 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1780298643.778662:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9cf242b43480 00000100:00100000:1.0:1780298643.778700:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 198 00000100:00100000:1.0:1780298643.778714:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609518976:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:1.0:1780298643.779436:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609518976:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 725us (964us total) trans 4294969242 rc 0/0 00000100:00100000:1.0:1780298643.779446:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 198 00002000:00100000:1.0:1780298643.779455:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x12:0x0] (2084864->2093055) req@ffff9cf242b43480 x1866777609518976/t4294969242(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 8680/448 e 0 to 0 dl 1780298654 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.779464:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b43480 with x1866777609518976 ext(2084864->2093055) 00010000:00010000:1.0:1780298643.779472:0:19943:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000009c841b52/0xaf35132e63034547 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2084864->35639295] (req 2084864->2093055) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3609 expref: 13 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.779490:0:19943:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000009c841b52/0xaf35132e63034547 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2084864->35639295] (req 2084864->2093055) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3609 expref: 13 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1780298643.779498:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9cf242b43480 00000100:00100000:3.0:1780298643.780271:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.780320:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609519104 00000100:00100000:0.0:1780298643.780325:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609519104 00010000:00010000:0.0:1780298643.780344:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000009c841b52/0xaf35132e63034547 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2084864->35639295] (req 2084864->2093055) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3609 expref: 13 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.780374:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15118 00000100:00100000:0.0:1780298643.780380:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b5e850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609519104:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298643.780391:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:0.0:1780298643.780398:0:6463:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 8 -> 24 00010000:00010000:0.0:1780298643.780402:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000009c841b52/0xaf35132e63034547 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2084864->35639295] (req 2084864->2093055) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3609 expref: 13 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.780411:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000009c841b52/0xaf35132e63034547 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2084864->35639295] (req 2084864->2093055) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3609 expref: 13 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.780425:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c841b52/0xaf35132e63034547 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2084864->35639295] (req 2084864->2093055) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3609 expref: 13 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.780434:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(117 130) 00010000:00010000:0.0:1780298643.780437:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(117 130) 00010000:00010000:0.0:1780298643.780442:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63034578 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3633 expref: 12 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.780448:0:6463:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63034578 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3633 expref: 12 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.780465:0:6463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63034578 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3633 expref: 12 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.780474:0:6463:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777604973184:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298643.780512:0:6463:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417e40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.781264:0:6463:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777604973184:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298643.781277:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298643.781315:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b5e850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609519104:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 934us (1044us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.781323:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15118 00000100:00100000:2.0:1780298643.781523:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.781546:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.781555:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609519232 00000100:00100000:3.0:1780298643.781561:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609519232 00002000:00010000:3.0:1780298643.781584:0:19950:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (1027283->18446744073709551615) 00002000:00100000:3.0:1780298643.781589:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25491a680 with x1866777609519232 ext(1027283->18446744073709551615) 00002000:00010000:3.0:1780298643.781598:0:19950:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf25491a680. 00000100:00100000:3.0:1780298643.781624:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609519360 00000100:00100000:3.0:1780298643.781628:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609519360 00002000:00010000:3.0:1780298643.781640:0:19950:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00000100:00100000:1.0:1780298643.781645:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6200 00002000:00100000:3.0:1780298643.781646:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25491ad80 with x1866777609519360 ext(0->18446744073709551615) 00002000:00010000:3.0:1780298643.781654:0:19950:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf25491ad80. 00000100:00100000:1.0:1780298643.781668:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609519232:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:0.0:1780298643.781669:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6201 00000100:00100000:0.0:1780298643.781683:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2399:x1866777609519360:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:0.0:1780298643.785108:0:19949:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4194304 -> 0 00002000:00010000:0.0:1780298643.785114:0:19949:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298643.785116:0:19949:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298643.785118:0:19949:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298643.785120:0:19949:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 24 -> 0 00000100:00100000:0.0:1780298643.785184:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609519360:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 3503us (3635us total) trans 4294969243 rc 0/0 00000100:00100000:0.0:1780298643.785191:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6201 00002000:00010000:0.0:1780298643.785194:0:19949:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00002000:00100000:0.0:1780298643.785197:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25491ad80 with x1866777609519360 ext(0->18446744073709551615) 00002000:00010000:0.0:1780298643.785203:0:19949:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf25491ad80. 00002000:00010000:1.0:1780298643.786004:0:8363:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4697678 -> 1027283 00002000:00010000:1.0:1780298643.786010:0:8363:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298643.786012:0:8363:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298643.786014:0:8363:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298643.786016:0:8363:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 400 -> 352 00000100:00100000:1.0:1780298643.786066:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609519232:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 4407us (4541us total) trans 4294969689 rc 0/0 00000100:00100000:1.0:1780298643.786076:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6200 00002000:00010000:1.0:1780298643.786080:0:8363:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (1027283->18446744073709551615) 00002000:00100000:1.0:1780298643.786083:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25491a680 with x1866777609519232 ext(1027283->18446744073709551615) 00002000:00010000:1.0:1780298643.786089:0:8363:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf25491a680. 00000100:00100000:2.0:1780298643.786918:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.786975:0:7411:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609519488 00000100:00100000:1.0:1780298643.786985:0:7411:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609519488 00000100:00100000:1.0:1780298643.787042:0:7411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61994 00000100:00100000:1.0:1780298643.787055:0:7411:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254918000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+148:29360:x1866777609519488:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.787076:0:7411:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.787098:0:7411:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841c800/0xaf35132e6303457f lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 77 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d363a expref: -99 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.787236:0:7411:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841ca00/0xaf35132e63034586 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 78 type: IBT flags: 0x40000000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.787273:0:7411:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841ca00/0xaf35132e63034586 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 78 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.787300:0:7411:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841ca00/0xaf35132e63034586 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 78 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1780298643.787335:0:7411:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 1027283 00000004:00200000:1.0:1780298643.787340:0:7411:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1780298643.787361:0:7411:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841ca00/0xaf35132e63034586 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 78 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.787394:0:7411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841c800/0xaf35132e6303457f lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 78 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d363a expref: 150 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.787413:0:7411:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841ca00/0xaf35132e63034586 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 77 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d363a expref: 149 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.787423:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298643.787425:0:7411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1780298643.787430:0:7411:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34841ca00, rc 0) 00000100:00100000:1.0:1780298643.787478:0:7411:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254918000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+149:29360:x1866777609519488:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 425us (558us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.787487:0:7411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61994 00000100:00100000:3.0:1780298643.790249:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.790310:0:6476:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609519744 00000100:00100000:2.0:1780298643.790319:0:6476:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609519744 00000100:00100000:2.0:1780298643.790365:0:6476:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61995 00000100:00100000:2.0:1780298643.790374:0:6476:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255093800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+149:29360:x1866777609519744:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:2.0:1780298643.790416:0:6476:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298643.790437:0:6476:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841ca00/0xaf35132e63034586 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 78 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d363a expref: 149 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.790454:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298643.790458:0:6476:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298643.790606:0:6476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfa00/0xaf35132e6303458d lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 77 type: IBT flags: 0x40000000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.790630:0:6476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfa00/0xaf35132e6303458d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 77 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.790640:0:6476:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfa00/0xaf35132e6303458d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 77 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1780298643.790684:0:6476:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991058 00000004:00010000:2.0:1780298643.790706:0:6476:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000e87ff114 reply state 000000007ee13030 transno 4294991059 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfa00/0xaf35132e6303458d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 77 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.790719:0:6476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfa00/0xaf35132e6303458d lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 77 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.790725:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298643.790728:0:6476:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1780298643.790730:0:6476:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1780298643.790796:0:6476:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255093800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+151:29360:x1866777609519744:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 422us (550us total) trans 4294991059 rc 0/0 00000100:00100000:2.0:1780298643.790804:0:6476:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61995 00000020:00080000:2.0:1780298643.790809:0:6476:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1780298643.791772:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.791791:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.791811:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609519872 00000100:00100000:2.0:1780298643.791817:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609519872 00002000:00010000:2.0:1780298643.791835:0:8364:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:2.0:1780298643.791840:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2556a9f80 with x1866777609519872 ext(4194304->18446744073709551615) 00000100:00100000:1.0:1780298643.791846:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609520000 00002000:00010000:2.0:1780298643.791848:0:8364:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2556a9f80. 00000100:00100000:1.0:1780298643.791855:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609520000 00000100:00100000:2.0:1780298643.791874:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6202 00002000:00010000:1.0:1780298643.791882:0:19951:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (2111883->18446744073709551615) 00000100:00100000:2.0:1780298643.791883:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609519872:12345-192.168.203.25@tcp:10:fsx.0 00002000:00100000:1.0:1780298643.791892:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2556aa300 with x1866777609520000 ext(2111883->18446744073709551615) 00002000:00010000:1.0:1780298643.791900:0:19951:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2556aa300. 00000100:00100000:1.0:1780298643.791938:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6203 00000100:00100000:1.0:1780298643.791950:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556aa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609520000:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:2.0:1780298643.791996:0:8364:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 1027283 -> 4194304 00002000:00010000:2.0:1780298643.792000:0:8364:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298643.792003:0:8364:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298643.792005:0:8364:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298643.792008:0:8364:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 352 -> 352 00000100:00100000:2.0:1780298643.792040:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609519872:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 159us (267us total) trans 4294969690 rc 0/0 00000100:00100000:2.0:1780298643.792048:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6202 00002000:00010000:2.0:1780298643.792052:0:8364:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:2.0:1780298643.792055:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2556a9f80 with x1866777609519872 ext(4194304->18446744073709551615) 00002000:00010000:2.0:1780298643.792061:0:8364:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2556a9f80. 00002000:00010000:1.0:1780298643.792112:0:19951:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 0 -> 2111883 00002000:00010000:1.0:1780298643.792117:0:19951:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298643.792119:0:19951:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298643.792158:0:19951:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298643.792163:0:19951:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:1.0:1780298643.792212:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556aa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609520000:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 264us (416us total) trans 4294969244 rc 0/0 00000100:00100000:1.0:1780298643.792221:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6203 00002000:00010000:1.0:1780298643.792226:0:19951:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (2111883->18446744073709551615) 00002000:00100000:1.0:1780298643.792229:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2556aa300 with x1866777609520000 ext(2111883->18446744073709551615) 00002000:00010000:1.0:1780298643.792235:0:19951:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2556aa300. 00000100:00100000:3.0:1780298643.793029:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.793077:0:17578:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609520128 00000100:00100000:1.0:1780298643.793083:0:17578:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609520128 00000100:00100000:1.0:1780298643.793115:0:17578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61996 00000100:00100000:1.0:1780298643.793144:0:17578:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5db800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+151:29360:x1866777609520128:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.793158:0:17578:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.793170:0:17578:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841ce00/0xaf35132e63034594 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 78 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3641 expref: -99 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.793243:0:17578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841c000/0xaf35132e6303459b lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 79 type: IBT flags: 0x40000000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.793258:0:17578:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841c000/0xaf35132e6303459b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 79 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.793268:0:17578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841c000/0xaf35132e6303459b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 79 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1780298643.793305:0:17578:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 6306187 00000004:00200000:1.0:1780298643.793311:0:17578:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1780298643.793331:0:17578:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841c000/0xaf35132e6303459b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 79 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.793365:0:17578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841ce00/0xaf35132e63034594 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 79 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3641 expref: 153 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.793384:0:17578:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841c000/0xaf35132e6303459b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 78 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3641 expref: 152 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.793394:0:17578:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298643.793397:0:17578:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1780298643.793400:0:17578:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34841c000, rc 0) 00000100:00100000:1.0:1780298643.793449:0:17578:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5db800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+152:29360:x1866777609520128:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 325us (419us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.793470:0:17578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61996 00000100:00100000:0.0:1780298643.795236:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.795298:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609520256 00000100:00100000:1.0:1780298643.795305:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609520256 00000100:00100000:1.0:1780298643.795362:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6204 00000100:00100000:1.0:1780298643.795375:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b43100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609520256:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298643.795580:0:19943:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:1.0:1780298643.795585:0:19943:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298643.795588:0:19943:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298643.795590:0:19943:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298643.795592:0:19943:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 352 -> 368 00000100:00100000:1.0:1780298643.795645:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609520256:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 272us (410us total) trans 4294969691 rc 0/0 00000100:00100000:1.0:1780298643.795654:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6204 00000100:00100000:3.0:1780298643.797515:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.797569:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609520384 00000100:00100000:2.0:1780298643.797575:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609520384 00000100:00100000:2.0:1780298643.797613:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61997 00000100:00100000:2.0:1780298643.797622:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+114:29360:x1866777609520384:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.797637:0:6477:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.797650:0:6477:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be000/0xaf35132e630345a2 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 79 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3648 expref: -99 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.797715:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bec00/0xaf35132e630345a9 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 80 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.797729:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bec00/0xaf35132e630345a9 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 80 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.797739:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bec00/0xaf35132e630345a9 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 80 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298643.797760:0:6477:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 6306187 00000004:00200000:2.0:1780298643.797763:0:6477:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298643.797778:0:6477:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bec00/0xaf35132e630345a9 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 80 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.797800:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be000/0xaf35132e630345a2 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 80 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3648 expref: 116 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.797818:0:6477:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bec00/0xaf35132e630345a9 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 79 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3648 expref: 115 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.797825:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298643.797828:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298643.797831:0:6477:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf2445bec00, rc 0) 00000100:00100000:2.0:1780298643.797864:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+115:29360:x1866777609520384:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 244us (350us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.797871:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61997 00000100:00100000:3.0:1780298643.798899:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.798930:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.798954:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609520512 00000100:00100000:0.0:1780298643.798960:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609520512 00000100:00100000:1.0:1780298643.798971:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609520640 00000100:00100000:1.0:1780298643.798977:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609520640 00000100:00100000:0.0:1780298643.798994:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9547 00000100:00100000:0.0:1780298643.799002:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5dbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609520512:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.799010:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.799019:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e630345b0 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d364f expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.799029:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9548 00000100:00100000:3.0:1780298643.799041:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5da300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609520640:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.799052:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.799063:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000004b2c7ae/0xaf35132e630345b7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3656 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.799065:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b41f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604973312:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.799107:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5db480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604973440:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.799227:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.799228:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417e40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.800033:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5db480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604973440:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.800034:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b41f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604973312:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.800069:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000004b2c7ae/0xaf35132e630345b7 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3656 expref: 8 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.800070:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e630345b0 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d364f expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.800098:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e630345b0 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d364f expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.800103:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000004b2c7ae/0xaf35132e630345b7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3656 expref: 8 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.800112:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf200, rc 0) 00010000:00010000:3.0:1780298643.800118:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156400, rc 0) 00000100:00100000:1.0:1780298643.800162:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5dbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609520512:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1161us (1264us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.800174:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9547 00000100:00100000:3.0:1780298643.800175:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5da300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609520640:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1132us (1239us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.800189:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9548 00000100:00100000:2.0:1780298643.801211:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.801235:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609520768 00000100:00100000:3.0:1780298643.801240:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609520768 00000100:00100000:3.0:1780298643.801273:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9549 00000100:00100000:3.0:1780298643.801279:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:29360:x1866777609520768:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.801300:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.801310:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e630345be lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d365d expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.801329:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034571 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d362c expref: 11 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.801339:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(157 177) 00010000:00010000:3.0:1780298643.801343:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(158 177) 00010000:00010000:3.0:1780298643.801348:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e630345be lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1490944->1499135] (req 1490944->1499135) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d365d expref: 7 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.801368:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034571 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d362c expref: 11 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.801382:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034571 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d362c expref: 11 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.801393:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5db480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604973568:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298643.801438:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.802276:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5db480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604973568:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298643.802366:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e630345be lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1490944->1499135] (req 1490944->1499135) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d365d expref: 7 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.802383:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157400, rc 0) 00000100:00100000:3.0:1780298643.802429:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609520768:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1146us (1215us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.802430:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.802441:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9549 00000100:00100000:1.0:1780298643.802494:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609520896 00000100:00100000:1.0:1780298643.802499:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609520896 00010000:00010000:1.0:1780298643.802522:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034571 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d362c expref: 12 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.802551:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15119 00000100:00100000:1.0:1780298643.802560:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609520896:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298643.802570:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298643.802582:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034571 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d362c expref: 12 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.802595:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034571 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d362c expref: 12 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.802612:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034571 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d362c expref: 12 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.802622:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(158 177) 00010000:00010000:1.0:1780298643.802626:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(158 177) 00010000:00010000:1.0:1780298643.802634:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e630345be lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1490944->35045375] (req 1490944->1499135) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d365d expref: 6 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.802642:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e630345be lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1490944->35045375] (req 1490944->1499135) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d365d expref: 6 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.802667:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e630345be lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1490944->35045375] (req 1490944->1499135) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d365d expref: 6 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.802677:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f1f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777604973696:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298643.802717:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.803470:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f1f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777604973696:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298643.803481:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298643.803506:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609520896:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 949us (1081us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.803513:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15119 00000100:00100000:0.0:1780298643.803597:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.803636:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609521024 00000100:00100000:2.0:1780298643.803641:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609521024 00000100:00100000:2.0:1780298643.803679:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6205 00000100:00100000:2.0:1780298643.803687:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b41c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609521024:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298643.803848:0:16972:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:2.0:1780298643.803852:0:16972:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298643.803855:0:16972:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298643.803857:0:16972:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298643.803860:0:16972:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 368 -> 384 00000100:00100000:2.0:1780298643.803894:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b41c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609521024:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 207us (296us total) trans 4294969692 rc 0/0 00000100:00100000:2.0:1780298643.803901:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6205 00000100:00100000:3.0:1780298643.804972:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.805014:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609521152 00000100:00100000:2.0:1780298643.805019:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609521152 00000100:00100000:2.0:1780298643.805054:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9550 00000100:00100000:2.0:1780298643.805071:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609521152:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.805083:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.805094:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006dfdf24b/0xaf35132e630345c5 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3664 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.805151:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254919880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604973824:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.805185:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.805205:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4540 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.805217:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609521280 00000100:00100000:1.0:1780298643.805223:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609521280 00000100:00100000:1.0:1780298643.805262:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9551 00000100:00100000:1.0:1780298643.805271:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b43800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609521280:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.805289:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.805299:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c18c87d/0xaf35132e630345cc lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d366b expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.805321:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604973952:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.805347:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a840 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.805865:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254919880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604973824:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.805886:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006dfdf24b/0xaf35132e630345c5 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3664 expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.805902:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006dfdf24b/0xaf35132e630345c5 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3664 expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.805913:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d08200, rc 0) 00000100:00100000:2.0:1780298643.805940:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609521152:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 880us (968us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.805947:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9550 00000100:00100000:1.0:1780298643.806069:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604973952:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.806080:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004c18c87d/0xaf35132e630345cc lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d366b expref: 8 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.806090:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c18c87d/0xaf35132e630345cc lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d366b expref: 8 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.806096:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34841ce00, rc 0) 00000100:00100000:1.0:1780298643.806121:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609521280:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 852us (939us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.806162:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9551 00000100:00100000:3.0:1780298643.807046:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.807090:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609521408 00000100:00100000:1.0:1780298643.807095:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609521408 00000100:00100000:1.0:1780298643.807161:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9552 00000100:00100000:1.0:1780298643.807171:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609521408:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298643.807181:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:1.0:1780298643.807188:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.807199:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000213a6cb7/0xaf35132e630345d3 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3672 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.807206:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609521536 00000100:00100000:2.0:1780298643.807211:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609521536 00000100:00100000:1.0:1780298643.807236:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604974080:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.807248:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9553 00000100:00100000:2.0:1780298643.807257:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609521536:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.807266:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.807275:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fba32a6f/0xaf35132e630345da lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3679 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.807298:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a840 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.807317:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254919880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604974208:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.807348:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4540 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.808206:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604974080:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.808227:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000213a6cb7/0xaf35132e630345d3 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3672 expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.808255:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000213a6cb7/0xaf35132e630345d3 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3672 expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.808269:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34841c800, rc 0) 00000100:00100000:2.0:1780298643.808270:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254919880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604974208:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.808303:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000fba32a6f/0xaf35132e630345da lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3679 expref: 8 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.808317:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609521408:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1147us (1271us total) trans 0 rc 301/301 00010000:00010000:2.0:1780298643.808320:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fba32a6f/0xaf35132e630345da lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3679 expref: 8 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.808332:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9552 00010000:00010000:2.0:1780298643.808347:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d08600, rc 0) 00000100:00100000:2.0:1780298643.808369:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609521536:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1114us (1194us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.808376:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9553 00000100:00100000:2.0:1780298643.809400:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.809436:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609521664 00000100:00100000:1.0:1780298643.809441:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609521664 00000100:00100000:1.0:1780298643.809468:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9554 00000100:00100000:1.0:1780298643.809472:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254919880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:29360:x1866777609521664:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.809482:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.809489:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000af0426c7/0xaf35132e630345e1 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3680 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.809505:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e630345be lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1490944->35045375] (req 1490944->1499135) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d365d expref: 7 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.809511:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(158 177) 00010000:00010000:1.0:1780298643.809514:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(159 177) 00010000:00010000:1.0:1780298643.809517:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000af0426c7/0xaf35132e630345e1 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1773568->1777663] (req 1773568->1777663) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3680 expref: 12 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.809532:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e630345be lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1490944->35045375] (req 1490944->1499135) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d365d expref: 7 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.809541:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e630345be lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1490944->35045375] (req 1490944->1499135) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d365d expref: 7 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.809547:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604974336:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298643.809584:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.810259:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604974336:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298643.810274:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000af0426c7/0xaf35132e630345e1 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1773568->1777663] (req 1773568->1777663) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3680 expref: 12 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.810292:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34841d200, rc 0) 00000100:00100000:1.0:1780298643.810321:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254919880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:29360:x1866777609521664:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 848us (923us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.810328:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9554 00000100:00100000:3.0:1780298643.810518:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.810561:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609521792 00000100:00100000:0.0:1780298643.810565:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609521792 00010000:00010000:0.0:1780298643.810583:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e630345be lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1490944->35045375] (req 1490944->1499135) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d365d expref: 8 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.810610:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15120 00000100:00100000:0.0:1780298643.810617:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d9f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609521792:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298643.810628:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298643.810640:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e630345be lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1490944->35045375] (req 1490944->1499135) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d365d expref: 8 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.810648:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e630345be lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1490944->35045375] (req 1490944->1499135) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d365d expref: 8 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.810661:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e630345be lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1490944->35045375] (req 1490944->1499135) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d365d expref: 8 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.810668:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(159 177) 00010000:00010000:0.0:1780298643.810670:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(159 177) 00010000:00010000:0.0:1780298643.810676:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000af0426c7/0xaf35132e630345e1 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1773568->1777663) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3680 expref: 11 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.810681:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000af0426c7/0xaf35132e630345e1 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1773568->1777663) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3680 expref: 11 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.810697:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000af0426c7/0xaf35132e630345e1 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1773568->1777663) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3680 expref: 11 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.810705:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b41f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777604974464:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298643.810735:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.811769:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b41f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777604974464:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298643.811772:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:0.0:1780298643.811786:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298643.811814:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609521920 00000100:00100000:0.0:1780298643.811818:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d9f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609521792:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1202us (1302us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.811819:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609521920 00000100:00100000:0.0:1780298643.811825:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15120 00002000:00100000:1.0:1780298643.811846:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (1773568->1777663) req@ffff9cf25491b800 x1866777609521920/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.811855:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25491b800 with x1866777609521920 ext(1773568->1777663) 00002000:00010000:1.0:1780298643.811860:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf25491b800 00000100:00100000:1.0:1780298643.811880:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6206 00000100:00100000:1.0:1780298643.811886:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609521920:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298643.812114:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609521920:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 226us (342us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298643.812143:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6206 00002000:00100000:1.0:1780298643.812151:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (1773568->1777663) req@ffff9cf25491b800 x1866777609521920/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.812159:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25491b800 with x1866777609521920 ext(1773568->1777663) 00002000:00010000:1.0:1780298643.812162:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf25491b800 00000100:00100000:2.0:1780298643.813209:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.813231:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.813251:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609522048 00000100:00100000:1.0:1780298643.813257:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609522048 00000100:00100000:0.0:1780298643.813276:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609522176 00000100:00100000:0.0:1780298643.813291:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609522176 00000100:00100000:1.0:1780298643.813307:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9555 00000100:00100000:2.0:1780298643.813316:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9556 00000100:00100000:1.0:1780298643.813321:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254919180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609522048:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.813331:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298643.813339:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609522176:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.813345:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008556a893/0xaf35132e630345e8 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3687 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.813349:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.813362:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042649f54/0xaf35132e630345ef lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d368e expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.813373:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(159 177) 00010000:00010000:1.0:1780298643.813378:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(159 177) 00010000:00010000:1.0:1780298643.813386:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008556a893/0xaf35132e630345e8 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3687 expref: 8 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.813401:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008556a893/0xaf35132e630345e8 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3687 expref: 8 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.813403:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242863b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604974592:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.813418:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34841c400, rc 0) 00000100:00100000:1.0:1780298643.813452:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254919180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609522048:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 137us (243us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.813461:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9555 00000100:00100000:2.0:1780298643.813467:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4540 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.814241:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242863b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604974592:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.814262:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000042649f54/0xaf35132e630345ef lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d368e expref: 8 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.814278:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042649f54/0xaf35132e630345ef lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d368e expref: 8 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.814301:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d08400, rc 0) 00000100:00100000:1.0:1780298643.814330:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609522176:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1004us (1096us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.814337:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9556 00000100:00100000:3.0:1780298643.815261:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.815303:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609522304 00000100:00100000:2.0:1780298643.815308:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609522304 00000100:00100000:2.0:1780298643.815339:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9557 00000100:00100000:2.0:1780298643.815347:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609522304:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.815356:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.815365:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e31d8c5/0xaf35132e630345f6 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3695 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.815400:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242862d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604974720:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.815450:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.816161:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242862d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604974720:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.816177:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003e31d8c5/0xaf35132e630345f6 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3695 expref: 8 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.816187:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e31d8c5/0xaf35132e630345f6 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3695 expref: 8 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.816195:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d08a00, rc 0) 00000100:00100000:3.0:1780298643.816220:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609522304:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 876us (966us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.816225:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9557 00000100:00100000:0.0:1780298643.817087:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.817122:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609522432 00000100:00100000:1.0:1780298643.817139:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609522432 00000100:00100000:1.0:1780298643.817178:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9558 00000100:00100000:1.0:1780298643.817187:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b41f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:29360:x1866777609522432:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.817195:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.817205:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000213a6cb7/0xaf35132e630345fd lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d369c expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.817225:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000af0426c7/0xaf35132e630345e1 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1773568->1777663) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3680 expref: 13 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.817236:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000008556a893/0xaf35132e630345e8 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3687 expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.817244:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(159 177) 00010000:00010000:1.0:1780298643.817248:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(159 177) 00010000:00010000:1.0:1780298643.817252:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000213a6cb7/0xaf35132e630345fd lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1957888->1966079] (req 1957888->1966079) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d369c expref: 13 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.817275:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000008556a893/0xaf35132e630345e8 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3687 expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.817300:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000008556a893/0xaf35132e630345e8 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3687 expref: 7 pid: 9128 timeout: 933 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.817311:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604974848:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298643.817361:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000af0426c7/0xaf35132e630345e1 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1773568->1777663) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3680 expref: 13 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.817373:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000af0426c7/0xaf35132e630345e1 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1773568->1777663) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3680 expref: 13 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.817384:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548ba300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604974976:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298643.817409:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.817963:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604974848:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298643.818159:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548ba300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604974976:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298643.818169:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:1.0:1780298643.818185:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000213a6cb7/0xaf35132e630345fd lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1957888->1966079] (req 1957888->1966079) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d369c expref: 13 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.818198:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34841c800, rc 0) 00000100:00100000:2.0:1780298643.818206:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609522560 00000100:00100000:2.0:1780298643.818212:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609522560 00010000:00010000:2.0:1780298643.818233:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000008556a893/0xaf35132e630345e8 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3687 expref: 8 pid: 9128 timeout: 933 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.818246:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b41f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:29360:x1866777609522432:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1059us (1156us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.818254:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9558 00000100:00100000:2.0:1780298643.818271:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15121 00000100:00100000:2.0:1780298643.818292:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343965880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609522560:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298643.818300:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298643.818310:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000008556a893/0xaf35132e630345e8 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3687 expref: 8 pid: 9128 timeout: 933 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.818319:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000008556a893/0xaf35132e630345e8 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3687 expref: 8 pid: 9128 timeout: 933 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.818329:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008556a893/0xaf35132e630345e8 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3687 expref: 8 pid: 9128 timeout: 933 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.818337:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298643.818350:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343965880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609522560:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 73us (187us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.818354:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15121 00000100:00100000:3.0:1780298643.818453:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.818490:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609522688 00000100:00100000:2.0:1780298643.818495:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609522688 00010000:00010000:2.0:1780298643.818509:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000af0426c7/0xaf35132e630345e1 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1773568->1777663) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3680 expref: 13 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.818532:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15122 00000100:00100000:2.0:1780298643.818537:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609522688:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298643.818543:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298643.818550:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000af0426c7/0xaf35132e630345e1 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1773568->1777663) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3680 expref: 13 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.818557:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000af0426c7/0xaf35132e630345e1 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1773568->1777663) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3680 expref: 13 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.818567:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000af0426c7/0xaf35132e630345e1 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1773568->1777663) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3680 expref: 13 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.818574:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(159 177) 00010000:00010000:2.0:1780298643.818577:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(159 177) 00010000:00010000:2.0:1780298643.818582:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000213a6cb7/0xaf35132e630345fd lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1957888->35512319] (req 1957888->1966079) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d369c expref: 12 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.818588:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000213a6cb7/0xaf35132e630345fd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1957888->35512319] (req 1957888->1966079) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d369c expref: 12 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.818602:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000213a6cb7/0xaf35132e630345fd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1957888->35512319] (req 1957888->1966079) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d369c expref: 12 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.818613:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242862d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777604975104:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298643.818643:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4b40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.819271:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242862d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777604975104:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298643.819294:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298643.819323:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609522688:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 784us (869us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.819331:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15122 00000100:00100000:3.0:1780298643.819502:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.819532:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609522816 00000100:00100000:2.0:1780298643.819536:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609522816 00000100:00100000:2.0:1780298643.819565:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6207 00000100:00100000:2.0:1780298643.819571:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609522816:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298643.819770:0:19928:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:2.0:1780298643.819773:0:19928:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298643.819776:0:19928:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298643.819779:0:19928:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298643.819781:0:19928:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 384 -> 400 00000100:00100000:2.0:1780298643.819811:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609522816:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 239us (309us total) trans 4294969693 rc 0/0 00000100:00100000:2.0:1780298643.819818:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6207 00000100:00100000:0.0:1780298643.820738:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.820758:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.820774:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609522944 00000100:00100000:3.0:1780298643.820778:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609522944 00000100:00100000:1.0:1780298643.820788:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609523072 00000100:00100000:1.0:1780298643.820793:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609523072 00000100:00100000:3.0:1780298643.820809:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9559 00000100:00100000:3.0:1780298643.820817:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343964380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609522944:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.820824:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298643.820827:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9560 00010000:00010000:3.0:1780298643.820833:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e63034604 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d36a3 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.820840:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343964a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609523072:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.820850:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.820859:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c18c87d/0xaf35132e6303460b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d36aa expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.820868:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604975232:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.820900:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548ba300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604975360:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.820906:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.820927:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.821526:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604975232:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.821539:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e63034604 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d36a3 expref: 8 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.821550:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e63034604 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d36a3 expref: 8 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.821597:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156400, rc 0) 00000100:00100000:3.0:1780298643.821622:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343964380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609522944:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 808us (885us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.821627:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9559 00000100:00100000:1.0:1780298643.821705:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548ba300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604975360:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.821723:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004c18c87d/0xaf35132e6303460b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d36aa expref: 8 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.821736:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c18c87d/0xaf35132e6303460b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d36aa expref: 8 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.821743:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34841ce00, rc 0) 00000100:00100000:1.0:1780298643.821768:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343964a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609523072:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 936us (1010us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.821774:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9560 00000100:00100000:2.0:1780298643.822618:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.822639:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.822640:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609523200 00000100:00100000:3.0:1780298643.822644:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609523200 00000100:00100000:3.0:1780298643.822666:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609523328 00000100:00100000:3.0:1780298643.822668:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609523328 00000100:00100000:3.0:1780298643.822684:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9561 00000100:00100000:3.0:1780298643.822692:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242862d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609523200:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298643.822696:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9562 00010000:00010000:3.0:1780298643.822700:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298643.822708:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242863b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609523328:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.822709:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e63034612 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d36b1 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.822717:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.822727:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e63034619 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d36b8 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.822733:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604975488:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.822758:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343965c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604975616:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.822771:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.822786:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.823407:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604975488:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.823475:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e63034612 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d36b1 expref: 8 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.823487:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e63034612 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d36b1 expref: 8 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.823493:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157200, rc 0) 00000100:00100000:3.0:1780298643.823514:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242862d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609523200:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 825us (897us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.823519:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9561 00000100:00100000:0.0:1780298643.823556:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343965c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604975616:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.823569:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e63034619 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d36b8 expref: 8 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.823586:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e63034619 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d36b8 expref: 8 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.823592:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baea00, rc 0) 00000100:00100000:0.0:1780298643.823612:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242863b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609523328:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 910us (974us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.823617:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9562 00000100:00100000:2.0:1780298643.824490:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.824525:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609523456 00000100:00100000:0.0:1780298643.824529:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609523456 00000100:00100000:0.0:1780298643.824554:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9563 00000100:00100000:0.0:1780298643.824559:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609523456:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.824565:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.824572:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63034620 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d36bf expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.824591:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343965c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604975744:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.824622:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.825229:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343965c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604975744:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.825240:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63034620 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d36bf expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.825249:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63034620 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d36bf expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.825259:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf200, rc 0) 00000100:00100000:0.0:1780298643.825291:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609523456:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 723us (792us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.825296:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9563 00000100:00100000:2.0:1780298643.825967:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.826004:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609523584 00000100:00100000:1.0:1780298643.826009:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609523584 00000100:00100000:1.0:1780298643.826039:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9564 00000100:00100000:1.0:1780298643.826047:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f734a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609523584:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.826055:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.826064:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000af0426c7/0xaf35132e63034627 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d36c6 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.826087:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548ba300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604975872:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.826117:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.826711:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548ba300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604975872:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.826723:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000af0426c7/0xaf35132e63034627 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d36c6 expref: 14 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.826731:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000af0426c7/0xaf35132e63034627 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d36c6 expref: 14 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.826738:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34841d200, rc 0) 00000100:00100000:1.0:1780298643.826760:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f734a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609523584:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 716us (794us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.826765:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9564 00000100:00100000:3.0:1780298643.827551:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.827584:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609523712 00000100:00100000:1.0:1780298643.827589:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609523712 00000100:00100000:1.0:1780298643.827615:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9565 00000100:00100000:1.0:1780298643.827622:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609523712:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.827630:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.827638:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008556a893/0xaf35132e6303462e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d36cd expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.827659:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63034578 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3633 expref: 13 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.827667:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(117 130) 00010000:00010000:1.0:1780298643.827671:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(118 130) 00010000:00010000:1.0:1780298643.827674:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008556a893/0xaf35132e6303462e lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [32768->40959] (req 32768->40959) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d36cd expref: 8 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.827692:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63034578 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3633 expref: 13 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.827702:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63034578 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3633 expref: 13 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.827710:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548ba300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604976000:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298643.827739:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.828277:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548ba300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604976000:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298643.828301:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008556a893/0xaf35132e6303462e lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [32768->40959] (req 32768->40959) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d36cd expref: 8 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.828310:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34841c400, rc 0) 00000100:00100000:1.0:1780298643.828334:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:29360:x1866777609523712:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 713us (782us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.828340:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9565 00000100:00100000:2.0:1780298643.828502:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.828534:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609523840 00000100:00100000:3.0:1780298643.828537:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609523840 00010000:00010000:3.0:1780298643.828552:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63034578 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3633 expref: 14 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.828570:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15123 00000100:00100000:3.0:1780298643.828575:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f736680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609523840:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298643.828582:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298643.828590:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63034578 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3633 expref: 14 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.828597:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63034578 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3633 expref: 14 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.828609:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63034578 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3633 expref: 14 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.828616:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(118 130) 00010000:00010000:3.0:1780298643.828618:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(118 130) 00010000:00010000:3.0:1780298643.828623:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000008556a893/0xaf35132e6303462e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [32768->33587199] (req 32768->40959) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d36cd expref: 7 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.828628:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008556a893/0xaf35132e6303462e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [32768->33587199] (req 32768->40959) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d36cd expref: 7 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.828644:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000008556a893/0xaf35132e6303462e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [32768->33587199] (req 32768->40959) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d36cd expref: 7 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.828650:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f745c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:8374:1866777604976128:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298643.828677:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.829301:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f745c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:8374:1866777604976128:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298643.829313:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298643.829342:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f736680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609523840:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 765us (839us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.829350:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15123 00000100:00100000:3.0:1780298643.829379:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.829407:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609523968 00000100:00100000:1.0:1780298643.829412:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609523968 00000100:00100000:1.0:1780298643.829442:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6208 00000100:00100000:1.0:1780298643.829449:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609523968:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298643.829577:0:19951:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 2111883 -> 2111883 00002000:00010000:1.0:1780298643.829580:0:19951:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298643.829583:0:19951:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298643.829586:0:19951:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298643.829588:0:19951:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:1.0:1780298643.829617:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609523968:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 168us (237us total) trans 4294969245 rc 0/0 00000100:00100000:1.0:1780298643.829625:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6208 00000100:00100000:0.0:1780298643.830308:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.830327:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.830349:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609524096 00000100:00100000:2.0:1780298643.830352:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609524224 00000100:00100000:3.0:1780298643.830353:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609524096 00000100:00100000:2.0:1780298643.830359:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609524224 00000100:00100000:3.0:1780298643.830381:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9566 00000100:00100000:3.0:1780298643.830389:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343965c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609524096:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.830395:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298643.830397:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9567 00010000:00010000:3.0:1780298643.830403:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e63034635 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d36d4 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.830407:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343966d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609524224:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.830419:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.830430:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e6303463c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d36db expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.830441:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f745f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604976256:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.830463:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343965f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604976384:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.830490:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.830490:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.831101:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f745f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604976256:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.831112:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343965f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604976384:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.831122:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e63034635 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d36d4 expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.831148:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e6303463c lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d36db expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.831170:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e6303463c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d36db expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.831172:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e63034635 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d36d4 expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.831183:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae000, rc 0) 00010000:00010000:1.0:1780298643.831192:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156400, rc 0) 00000100:00100000:0.0:1780298643.831215:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343966d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609524224:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 810us (884us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.831217:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343965c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609524096:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 829us (908us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.831229:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9567 00000100:00100000:1.0:1780298643.831237:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9566 00000100:00100000:1.0:1780298643.832112:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.832156:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609524352 00000100:00100000:2.0:1780298643.832161:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609524352 00000100:00100000:2.0:1780298643.832191:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9568 00000100:00100000:2.0:1780298643.832197:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609524352:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.832204:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.832210:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000debe9bb3/0xaf35132e63034643 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d36e2 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.832233:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f734000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604976512:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.832239:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.832277:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4b40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.832279:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609524480 00000100:00100000:1.0:1780298643.832294:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609524480 00000100:00100000:1.0:1780298643.832322:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9569 00000100:00100000:1.0:1780298643.832331:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609524480:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.832339:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.832347:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004c18c87d/0xaf35132e6303464a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d36e9 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.832373:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548bad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604976640:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.832397:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.833006:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f734000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604976512:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.833021:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000debe9bb3/0xaf35132e63034643 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d36e2 expref: 14 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.833031:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000debe9bb3/0xaf35132e63034643 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d36e2 expref: 14 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.833038:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d09600, rc 0) 00000100:00100000:2.0:1780298643.833062:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609524352:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 866us (950us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.833071:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9568 00000100:00100000:1.0:1780298643.833102:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548bad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604976640:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.833117:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004c18c87d/0xaf35132e6303464a lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d36e9 expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.833153:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004c18c87d/0xaf35132e6303464a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d36e9 expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.833163:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34841ce00, rc 0) 00000100:00100000:1.0:1780298643.833203:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609524480:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 873us (970us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.833210:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9569 00000100:00100000:0.0:1780298643.834005:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.834032:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609524608 00000100:00100000:1.0:1780298643.834037:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609524608 00000100:00100000:1.0:1780298643.834060:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9570 00000100:00100000:1.0:1780298643.834066:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e4f1c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609524608:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.834071:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.834078:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000004503c2f/0xaf35132e63034651 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d36f0 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.834098:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548bad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604976768:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.834110:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.834152:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609524736 00000100:00100000:0.0:1780298643.834157:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609524736 00000100:00100000:1.0:1780298643.834162:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.834191:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9571 00000100:00100000:0.0:1780298643.834200:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f747100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609524736:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.834208:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.834216:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e63034658 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d36f7 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.834237:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343965f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604976896:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.834264:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.834777:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548bad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604976768:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.834788:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000004503c2f/0xaf35132e63034651 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d36f0 expref: 14 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.834802:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000004503c2f/0xaf35132e63034651 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d36f0 expref: 14 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.834809:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34841ca00, rc 0) 00000100:00100000:1.0:1780298643.834835:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e4f1c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609524608:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 768us (829us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.834842:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9570 00000100:00100000:0.0:1780298643.834882:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343965f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604976896:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.834905:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e63034658 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d36f7 expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.834922:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e63034658 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d36f7 expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.834933:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf200, rc 0) 00000100:00100000:0.0:1780298643.834954:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f747100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609524736:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 756us (846us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.834959:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9571 00000100:00100000:0.0:1780298643.835685:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.835717:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609524864 00000100:00100000:2.0:1780298643.835721:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609524864 00000100:00100000:2.0:1780298643.835751:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9572 00000100:00100000:2.0:1780298643.835760:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343965f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609524864:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298643.835768:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:2.0:1780298643.835770:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.835779:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006dfdf24b/0xaf35132e6303465f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d36fe expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.835789:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609524992 00000100:00100000:1.0:1780298643.835793:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609524992 00000100:00100000:2.0:1780298643.835807:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f734000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604977024:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.835818:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9573 00000100:00100000:1.0:1780298643.835824:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f744700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609524992:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.835832:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.835839:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000af0426c7/0xaf35132e63034666 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3705 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.835843:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4b40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.835864:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548ba300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604977152:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.835890:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.836429:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f734000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604977024:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.836445:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006dfdf24b/0xaf35132e6303465f lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d36fe expref: 14 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.836459:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006dfdf24b/0xaf35132e6303465f lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d36fe expref: 14 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.836469:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d08200, rc 0) 00000100:00100000:2.0:1780298643.836499:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343965f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609524864:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 741us (812us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.836507:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9572 00000100:00100000:1.0:1780298643.836614:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548ba300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604977152:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.836629:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000af0426c7/0xaf35132e63034666 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3705 expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.836644:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000af0426c7/0xaf35132e63034666 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3705 expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.836659:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34841d200, rc 0) 00000100:00100000:1.0:1780298643.836687:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f744700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609524992:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 863us (922us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.836695:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9573 00000100:00100000:0.0:1780298643.837465:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.837495:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609525120 00000100:00100000:1.0:1780298643.837499:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609525120 00000100:00100000:1.0:1780298643.837520:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9574 00000100:00100000:1.0:1780298643.837525:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343964000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609525120:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.837531:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.837537:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e6303466d lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d370c expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.837549:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000213a6cb7/0xaf35132e630345fd lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1957888->35512319] (req 1957888->1966079) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d369c expref: 12 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.837555:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(159 177) 00010000:00010000:1.0:1780298643.837557:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(160 177) 00010000:00010000:1.0:1780298643.837560:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e6303466d lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3723264->3735551] (req 3723264->3735551) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d370c expref: 8 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.837572:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000213a6cb7/0xaf35132e630345fd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1957888->35512319] (req 1957888->1966079) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d369c expref: 12 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.837580:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000213a6cb7/0xaf35132e630345fd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1957888->35512319] (req 1957888->1966079) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d369c expref: 12 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.837587:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548ba300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604977280:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298643.837614:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.838211:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548ba300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604977280:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298643.838224:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e6303466d lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3723264->3735551] (req 3723264->3735551) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d370c expref: 8 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.838235:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278400, rc 0) 00000100:00100000:1.0:1780298643.838262:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343964000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:29360:x1866777609525120:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 735us (796us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.838269:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9574 00000100:00100000:3.0:1780298643.838448:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.838477:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609525248 00000100:00100000:0.0:1780298643.838482:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609525248 00010000:00010000:0.0:1780298643.838503:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000213a6cb7/0xaf35132e630345fd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1957888->35512319] (req 1957888->1966079) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d369c expref: 13 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.838532:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15124 00000100:00100000:0.0:1780298643.838540:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f746300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609525248:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298643.838550:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298643.838561:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000213a6cb7/0xaf35132e630345fd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1957888->35512319] (req 1957888->1966079) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d369c expref: 13 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.838572:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000213a6cb7/0xaf35132e630345fd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1957888->35512319] (req 1957888->1966079) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d369c expref: 13 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.838587:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000213a6cb7/0xaf35132e630345fd lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1957888->35512319] (req 1957888->1966079) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d369c expref: 13 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.838597:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(160 177) 00010000:00010000:0.0:1780298643.838600:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(160 177) 00010000:00010000:0.0:1780298643.838607:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e6303466d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3723264->37277695] (req 3723264->3735551) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d370c expref: 7 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.838615:0:6463:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e6303466d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3723264->37277695] (req 3723264->3735551) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d370c expref: 7 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.838642:0:6463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e6303466d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3723264->37277695] (req 3723264->3735551) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d370c expref: 7 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.838652:0:6463:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343966300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6463:1866777604977408:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298643.838691:0:6463:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.839334:0:6463:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343966300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6463:1866777604977408:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298643.839345:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298643.839370:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f746300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609525248:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 831us (922us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.839379:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15124 00000100:00100000:2.0:1780298643.839660:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.839691:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609525376 00000100:00100000:1.0:1780298643.839695:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609525376 00002000:00100000:1.0:1780298643.839715:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (3723264->3727359) req@ffff9cf37f734000 x1866777609525376/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.839721:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f734000 with x1866777609525376 ext(3723264->3727359) 00002000:00010000:1.0:1780298643.839725:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f734000 00000100:00100000:1.0:1780298643.839741:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6209 00000100:00100000:1.0:1780298643.839745:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f734000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609525376:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298643.839868:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f734000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609525376:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 122us (209us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298643.839873:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6209 00002000:00100000:1.0:1780298643.839878:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (3723264->3727359) req@ffff9cf37f734000 x1866777609525376/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.839888:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f734000 with x1866777609525376 ext(3723264->3727359) 00002000:00010000:1.0:1780298643.839890:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f734000 00000100:00100000:3.0:1780298643.840632:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.840669:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609525504 00000100:00100000:2.0:1780298643.840674:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609525504 00002000:00100000:2.0:1780298643.840701:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (3731456->3735551) req@ffff9cf37f745f80 x1866777609525504/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.840711:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f745f80 with x1866777609525504 ext(3731456->3735551) 00002000:00010000:2.0:1780298643.840715:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f745f80 00000100:00100000:2.0:1780298643.840734:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6210 00000100:00100000:2.0:1780298643.840740:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609525504:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298643.840807:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609525504:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 68us (175us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298643.840813:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6210 00002000:00100000:2.0:1780298643.840817:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (3731456->3735551) req@ffff9cf37f745f80 x1866777609525504/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.840823:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f745f80 with x1866777609525504 ext(3731456->3735551) 00002000:00010000:2.0:1780298643.840826:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f745f80 00000100:00100000:0.0:1780298643.841731:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.841764:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609525632 00000100:00100000:1.0:1780298643.841767:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609525632 00000100:00100000:1.0:1780298643.841796:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9575 00000100:00100000:1.0:1780298643.841802:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343966a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609525632:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.841810:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298643.841814:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:1.0:1780298643.841820:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63034674 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3713 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.841849:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548ba300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604977536:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.841859:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609525760 00000100:00100000:0.0:1780298643.841873:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609525760 00000100:00100000:1.0:1780298643.841893:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837ae40 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.841913:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9576 00000100:00100000:0.0:1780298643.841924:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609525760:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.841932:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.841942:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e6303467b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d371a expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.841970:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343966300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604977664:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.842006:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417d80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.842613:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548ba300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604977536:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.842632:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63034674 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3713 expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.842647:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63034674 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3713 expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.842657:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279c00, rc 0) 00000100:00100000:1.0:1780298643.842686:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343966a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609525632:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 884us (956us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.842694:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9575 00002000:00010000:0.0:1780298643.842713:0:19992:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks: 0 -> 1 00000100:00100000:0.0:1780298643.842726:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343966300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604977664:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.842744:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e6303467b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d371a expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.842756:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e6303467b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d371a expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.842767:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baea00, rc 0) 00000100:00100000:0.0:1780298643.842794:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609525760:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 874us (981us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.842801:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9576 00000100:00100000:2.0:1780298643.843574:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.843595:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.843610:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609525888 00000100:00100000:1.0:1780298643.843615:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609525888 00000100:00100000:1.0:1780298643.843642:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609526016 00000100:00100000:1.0:1780298643.843645:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609526016 00000100:00100000:0.0:1780298643.843654:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9577 00000100:00100000:2.0:1780298643.843662:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9578 00000100:00100000:0.0:1780298643.843666:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f734700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609525888:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298643.843673:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f735880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609526016:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.843677:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.843680:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.843687:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e63034682 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3721 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.843689:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000debe9bb3/0xaf35132e63034689 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3728 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.843732:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343966300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604977792:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.843739:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604977920:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.843787:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4540 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.843787:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417d80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.844481:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343966300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604977792:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.844492:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604977920:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.844502:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e63034682 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3721 expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.844509:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000debe9bb3/0xaf35132e63034689 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3728 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.844527:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000debe9bb3/0xaf35132e63034689 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3728 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.844528:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e63034682 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3721 expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.844544:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d09600, rc 0) 00010000:00010000:0.0:1780298643.844549:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae000, rc 0) 00000100:00100000:2.0:1780298643.844568:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f735880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609526016:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 899us (971us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.844578:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9578 00000100:00100000:0.0:1780298643.844579:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f734700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609525888:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 914us (1001us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.844589:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9577 00000100:00100000:3.0:1780298643.845525:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.845555:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609526144 00000100:00100000:2.0:1780298643.845559:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609526144 00000100:00100000:2.0:1780298643.845582:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9579 00000100:00100000:2.0:1780298643.845587:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609526144:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.845594:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.845601:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007ba4a4a6/0xaf35132e63034690 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d372f expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.845621:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604978048:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.845661:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4540 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.845666:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.845696:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609526272 00000100:00100000:1.0:1780298643.845700:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609526272 00000100:00100000:1.0:1780298643.845729:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9580 00000100:00100000:1.0:1780298643.845736:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609526272:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.845744:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.845752:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004fdbd682/0xaf35132e63034697 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3736 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.845779:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548ba300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604978176:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.845812:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837ae40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.846344:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604978048:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.846361:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007ba4a4a6/0xaf35132e63034690 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d372f expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.846371:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007ba4a4a6/0xaf35132e63034690 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d372f expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.846377:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d09c00, rc 0) 00000100:00100000:2.0:1780298643.846405:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609526144:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 818us (882us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.846411:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9579 00000100:00100000:1.0:1780298643.846600:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548ba300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604978176:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.846627:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004fdbd682/0xaf35132e63034697 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3736 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.846648:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004fdbd682/0xaf35132e63034697 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3736 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.846659:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278e00, rc 0) 00000100:00100000:1.0:1780298643.846710:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609526272:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 972us (1044us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.846720:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9580 00000100:00100000:3.0:1780298643.847538:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.847577:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609526400 00000100:00100000:1.0:1780298643.847583:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609526400 00000100:00100000:1.0:1780298643.847618:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6211 00000100:00100000:1.0:1780298643.847626:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255489500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609526400:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298643.847762:0:8363:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:1.0:1780298643.847765:0:8363:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298643.847767:0:8363:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298643.847768:0:8363:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298643.847770:0:8363:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 400 -> 408 00000100:00100000:1.0:1780298643.847796:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255489500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609526400:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 171us (259us total) trans 4294969694 rc 0/0 00000100:00100000:1.0:1780298643.847803:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6211 00000100:00100000:3.0:1780298643.848691:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.848742:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609526528 00000100:00100000:2.0:1780298643.848746:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609526528 00000100:00100000:2.0:1780298643.848777:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9581 00000100:00100000:3.0:1780298643.848778:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.848787:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609526528:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.848794:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298643.848799:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609526656 00010000:00010000:2.0:1780298643.848801:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003e31d8c5/0xaf35132e6303469e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d373d expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.848803:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609526656 00000100:00100000:2.0:1780298643.848829:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604978304:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.848833:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9582 00000100:00100000:1.0:1780298643.848840:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255489f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609526656:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.848846:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.848854:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e630346a5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3744 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.848881:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4540 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.848896:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548ba300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604978432:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.848924:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837ae40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.849689:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604978304:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.849708:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548ba300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604978432:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.849711:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003e31d8c5/0xaf35132e6303469e lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d373d expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.849738:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e630346a5 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3744 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.849740:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003e31d8c5/0xaf35132e6303469e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d373d expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.849768:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d08a00, rc 0) 00010000:00010000:1.0:1780298643.849772:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e630346a5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3744 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.849790:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279200, rc 0) 00000100:00100000:2.0:1780298643.849800:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609526528:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1013us (1108us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.849807:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9581 00000100:00100000:1.0:1780298643.849815:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255489f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609526656:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 973us (1038us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.849824:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9582 00000100:00100000:0.0:1780298643.850867:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.850898:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.850917:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609526784 00000100:00100000:2.0:1780298643.850922:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609526784 00000100:00100000:3.0:1780298643.850928:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609526912 00000100:00100000:3.0:1780298643.850933:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609526912 00000100:00100000:2.0:1780298643.850960:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9583 00000100:00100000:3.0:1780298643.850967:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9584 00000100:00100000:2.0:1780298643.850973:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343966300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609526784:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298643.850980:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343964380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609526912:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.850984:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.850987:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.850995:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fba32a6f/0xaf35132e630346ac lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d374b expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.850996:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e0b8e7b/0xaf35132e630346b3 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3752 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.851038:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255489180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604978560:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.851047:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604978688:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.851074:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.851081:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4540 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.851849:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255489180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604978560:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.851857:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604978688:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.851873:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009e0b8e7b/0xaf35132e630346b3 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3752 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.851878:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000fba32a6f/0xaf35132e630346ac lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d374b expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.851901:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fba32a6f/0xaf35132e630346ac lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d374b expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.851902:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e0b8e7b/0xaf35132e630346b3 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3752 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.851916:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d08600, rc 0) 00010000:00010000:2.0:1780298643.851929:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156c00, rc 0) 00000100:00100000:1.0:1780298643.851944:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343966300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609526784:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 977us (1078us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.851949:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9583 00000100:00100000:2.0:1780298643.851958:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343964380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609526912:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 981us (1057us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.851966:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9584 00000100:00100000:0.0:1780298643.852981:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.853027:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609527040 00000100:00100000:3.0:1780298643.853031:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609527040 00000100:00100000:3.0:1780298643.853058:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9585 00000100:00100000:3.0:1780298643.853064:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343964a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609527040:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.853072:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298643.853078:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:3.0:1780298643.853079:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e630346ba lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3759 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.853097:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609527168 00000100:00100000:2.0:1780298643.853102:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609527168 00000100:00100000:3.0:1780298643.853105:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255488a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604978816:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.853148:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9586 00000100:00100000:2.0:1780298643.853157:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609527168:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.853165:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.853172:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006dfdf24b/0xaf35132e630346c1 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3760 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.853198:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.853203:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604978944:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.853237:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4540 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.853821:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255488a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604978816:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.853836:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e630346ba lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3759 expref: 13 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.853850:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e630346ba lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3759 expref: 13 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.853905:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157200, rc 0) 00000100:00100000:3.0:1780298643.853928:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343964a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609527040:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 865us (950us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.853933:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9585 00000100:00100000:2.0:1780298643.853996:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604978944:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.854015:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000006dfdf24b/0xaf35132e630346c1 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3760 expref: 14 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.854028:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006dfdf24b/0xaf35132e630346c1 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3760 expref: 14 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.854034:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d08200, rc 0) 00000100:00100000:2.0:1780298643.854058:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609527168:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 904us (983us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.854063:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9586 00000100:00100000:1.0:1780298643.854994:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.855026:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.855047:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609527296 00000100:00100000:0.0:1780298643.855053:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609527296 00000100:00100000:2.0:1780298643.855059:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609527424 00000100:00100000:2.0:1780298643.855064:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609527424 00000100:00100000:0.0:1780298643.855089:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9587 00000100:00100000:2.0:1780298643.855098:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9588 00000100:00100000:0.0:1780298643.855102:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548bad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609527296:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298643.855111:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548bb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609527424:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.855113:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.855121:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.855151:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ba4a4a6/0xaf35132e630346cf lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d376e expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.855153:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000058e2d0/0xaf35132e630346c8 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3767 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.855188:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604979072:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.855209:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343966d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604979200:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.855239:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4540 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.855243:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417d80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.855966:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604979072:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.855983:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007ba4a4a6/0xaf35132e630346cf lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d376e expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.855993:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ba4a4a6/0xaf35132e630346cf lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d376e expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.855996:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343966d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604979200:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.856008:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d09c00, rc 0) 00010000:00010000:1.0:1780298643.856019:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000058e2d0/0xaf35132e630346c8 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3767 expref: 13 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.856030:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000058e2d0/0xaf35132e630346c8 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3767 expref: 13 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.856035:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548bb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609527424:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 927us (1007us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.856046:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9588 00010000:00010000:1.0:1780298643.856046:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baea00, rc 0) 00000100:00100000:1.0:1780298643.856068:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548bad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609527296:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 972us (1074us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.856073:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9587 00000100:00100000:1.0:1780298643.857062:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.857107:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609527552 00000100:00100000:0.0:1780298643.857112:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609527552 00000100:00100000:0.0:1780298643.857167:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9589 00000100:00100000:0.0:1780298643.857176:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:29360:x1866777609527552:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.857185:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.857193:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e630346d6 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3775 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.857209:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(160 177) 00010000:00010000:0.0:1780298643.857214:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(160 177) 00010000:00010000:0.0:1780298643.857225:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e630346d6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1581056->3719167] (req 1581056->1589247) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3775 expref: 13 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.857241:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e630346d6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1581056->3719167] (req 1581056->1589247) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3775 expref: 13 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.857250:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae000, rc 0) 00000100:00100000:0.0:1780298643.857277:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:29360:x1866777609527552:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 102us (215us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298643.857294:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9589 00000100:00100000:2.0:1780298643.858054:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.858091:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609527680 00000100:00100000:3.0:1780298643.858098:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609527680 00000100:00100000:3.0:1780298643.858189:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6212 00000100:00100000:3.0:1780298643.858202:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f736a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609527680:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:3.0:1780298643.858413:0:19928:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:3.0:1780298643.858418:0:19928:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:3.0:1780298643.858420:0:19928:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:3.0:1780298643.858423:0:19928:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:3.0:1780298643.858425:0:19928:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 408 -> 408 00000100:00100000:3.0:1780298643.858476:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f736a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2397:x1866777609527680:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 276us (418us total) trans 4294969695 rc 0/0 00000100:00100000:3.0:1780298643.858485:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6212 00000100:00100000:3.0:1780298643.859483:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.859517:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609527808 00000100:00100000:2.0:1780298643.859521:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609527808 00000100:00100000:2.0:1780298643.859548:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9590 00000100:00100000:2.0:1780298643.859555:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255488a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609527808:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.859564:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298643.859571:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:2.0:1780298643.859573:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003e31d8c5/0xaf35132e630346dd lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d377c expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.859606:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f734e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604979328:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.859652:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609527936 00000100:00100000:0.0:1780298643.859657:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609527936 00000100:00100000:2.0:1780298643.859660:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4540 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.859692:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9591 00000100:00100000:0.0:1780298643.859701:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609527936:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.859709:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.859718:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e630346e4 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3783 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.859745:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343965c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604979456:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.859781:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.860454:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f734e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604979328:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.860478:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003e31d8c5/0xaf35132e630346dd lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d377c expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.860488:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343965c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604979456:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.860493:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003e31d8c5/0xaf35132e630346dd lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d377c expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.860501:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d08a00, rc 0) 00010000:00010000:0.0:1780298643.860505:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e630346e4 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3783 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.860521:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e630346e4 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3783 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.860526:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255488a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609527808:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 971us (1046us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.860537:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9590 00010000:00010000:0.0:1780298643.860543:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae400, rc 0) 00000100:00100000:0.0:1780298643.860568:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609527936:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 868us (1000us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.860574:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9591 00000100:00100000:1.0:1780298643.861509:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.861528:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.861542:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609528064 00000100:00100000:0.0:1780298643.861547:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609528064 00000100:00100000:0.0:1780298643.861573:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609528192 00000100:00100000:0.0:1780298643.861575:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609528192 00000100:00100000:3.0:1780298643.861591:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9592 00000100:00100000:0.0:1780298643.861600:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9593 00000100:00100000:3.0:1780298643.861608:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609528064:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298643.861610:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609528192:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.861621:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.861626:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.861628:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e630346eb lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3791 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.861639:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e630346f2 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d378a expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.861651:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343965c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604979584:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.861684:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.861688:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604979712:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.861735:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.862396:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343965c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604979584:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.862409:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e630346eb lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3791 expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.862418:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e630346eb lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3791 expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.862425:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf600, rc 0) 00000100:00100000:0.0:1780298643.862452:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609528192:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 846us (923us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.862456:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9593 00000100:00100000:1.0:1780298643.862484:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604979712:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.862499:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e630346f2 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d378a expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.862510:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e630346f2 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d378a expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.862517:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157200, rc 0) 00000100:00100000:1.0:1780298643.862535:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609528064:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 936us (1026us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.862540:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9592 00000100:00100000:1.0:1780298643.863542:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.863576:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609528320 00000100:00100000:2.0:1780298643.863580:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609528320 00000100:00100000:2.0:1780298643.863608:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9594 00000100:00100000:1.0:1780298643.863616:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.863621:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609528320:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.863635:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298643.863640:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609528448 00010000:00010000:2.0:1780298643.863643:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007ba4a4a6/0xaf35132e630346f9 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3798 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.863649:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609528448 00000100:00100000:2.0:1780298643.863667:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f734e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604979840:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.863702:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4540 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.863766:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9595 00000100:00100000:3.0:1780298643.863780:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556aa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609528448:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.863792:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.863865:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bed20494/0xaf35132e63034700 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d379f expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.863907:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604979968:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.863960:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.864327:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f734e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604979840:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.864345:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007ba4a4a6/0xaf35132e630346f9 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3798 expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.864359:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007ba4a4a6/0xaf35132e630346f9 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3798 expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.864368:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d09c00, rc 0) 00000100:00100000:2.0:1780298643.864403:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609528320:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 788us (862us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.864410:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9594 00000100:00100000:3.0:1780298643.864720:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604979968:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.864744:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bed20494/0xaf35132e63034700 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d379f expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.864764:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bed20494/0xaf35132e63034700 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d379f expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.864774:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157000, rc 0) 00000100:00100000:3.0:1780298643.864822:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556aa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609528448:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1044us (1206us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.864831:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9595 00000100:00100000:1.0:1780298643.865739:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.865769:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609528576 00000100:00100000:0.0:1780298643.865773:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609528576 00000100:00100000:1.0:1780298643.865774:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.865809:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609528704 00000100:00100000:0.0:1780298643.865813:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609528704 00000100:00100000:2.0:1780298643.865818:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9596 00000100:00100000:2.0:1780298643.865826:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609528576:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.865833:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298643.865841:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9597 00010000:00010000:2.0:1780298643.865842:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006dfdf24b/0xaf35132e63034707 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d37a6 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.865853:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556aad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609528704:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.865862:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.865872:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c37b15a2/0xaf35132e6303470e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d37ad expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.865876:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f734e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604980096:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.865910:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343965c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604980224:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.865917:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4540 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.865941:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.866796:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343965c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604980224:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.866810:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f734e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604980096:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.866817:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c37b15a2/0xaf35132e6303470e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d37ad expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.866834:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c37b15a2/0xaf35132e6303470e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d37ad expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.866835:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006dfdf24b/0xaf35132e63034707 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d37a6 expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.866857:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baee00, rc 0) 00010000:00010000:2.0:1780298643.866864:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006dfdf24b/0xaf35132e63034707 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d37a6 expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.866875:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d08200, rc 0) 00000100:00100000:0.0:1780298643.866892:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556aad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609528704:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1042us (1115us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.866902:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609528576:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1074us (1160us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.866902:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9597 00000100:00100000:2.0:1780298643.866913:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9596 00000100:00100000:3.0:1780298643.867991:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.868030:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609528832 00000100:00100000:2.0:1780298643.868035:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609528832 00000100:00100000:2.0:1780298643.868066:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9598 00000100:00100000:2.0:1780298643.868075:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:29360:x1866777609528832:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.868085:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.868094:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fba32a6f/0xaf35132e63034715 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d37b4 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.868112:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000008556a893/0xaf35132e6303462e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [32768->33587199] (req 32768->40959) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d36cd expref: 8 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.868122:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(118 130) 00010000:00010000:2.0:1780298643.868148:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(119 130) 00010000:00010000:2.0:1780298643.868154:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fba32a6f/0xaf35132e63034715 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [442368->450559] (req 442368->450559) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d37b4 expref: 14 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.868183:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000008556a893/0xaf35132e6303462e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [32768->33587199] (req 32768->40959) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d36cd expref: 8 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.868198:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000008556a893/0xaf35132e6303462e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [32768->33587199] (req 32768->40959) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d36cd expref: 8 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.868218:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f734e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604980352:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298643.868259:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4540 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.868916:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f734e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604980352:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298643.868929:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000fba32a6f/0xaf35132e63034715 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [442368->450559] (req 442368->450559) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d37b4 expref: 14 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.868936:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d08600, rc 0) 00000100:00100000:2.0:1780298643.868958:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:29360:x1866777609528832:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 886us (972us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.868963:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9598 00000100:00100000:1.0:1780298643.869107:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.869147:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609528960 00000100:00100000:3.0:1780298643.869151:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609528960 00010000:00010000:3.0:1780298643.869170:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000008556a893/0xaf35132e6303462e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [32768->33587199] (req 32768->40959) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d36cd expref: 9 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.869217:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15125 00000100:00100000:2.0:1780298643.869227:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b5f050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609528960:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298643.869236:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298643.869247:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000008556a893/0xaf35132e6303462e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [32768->33587199] (req 32768->40959) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d36cd expref: 9 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.869258:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000008556a893/0xaf35132e6303462e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [32768->33587199] (req 32768->40959) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d36cd expref: 9 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.869274:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008556a893/0xaf35132e6303462e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [32768->33587199] (req 32768->40959) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d36cd expref: 9 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.869294:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(119 130) 00010000:00010000:2.0:1780298643.869298:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(119 130) 00010000:00010000:2.0:1780298643.869306:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000fba32a6f/0xaf35132e63034715 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [442368->33996799] (req 442368->450559) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d37b4 expref: 13 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.869314:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fba32a6f/0xaf35132e63034715 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [442368->33996799] (req 442368->450559) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d37b4 expref: 13 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.869336:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000fba32a6f/0xaf35132e63034715 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [442368->33996799] (req 442368->450559) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d37b4 expref: 13 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.869346:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f734e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6461:1866777604980480:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298643.869382:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4540 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.870102:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f734e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6461:1866777604980480:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298643.870113:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298643.870165:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b5f050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609528960:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 940us (1059us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.870173:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15125 00000100:00100000:1.0:1780298643.870383:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.870423:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609529088 00000100:00100000:2.0:1780298643.870428:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609529088 00002000:00100000:2.0:1780298643.870452:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (442368->446463) req@ffff9cf242b68700 x1866777609529088/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.870463:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b68700 with x1866777609529088 ext(442368->446463) 00002000:00010000:2.0:1780298643.870467:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf242b68700 00000100:00100000:2.0:1780298643.870489:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6213 00000100:00100000:2.0:1780298643.870495:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b68700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2397:x1866777609529088:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298643.870645:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b68700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2397:x1866777609529088:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 149us (261us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298643.870653:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6213 00002000:00100000:2.0:1780298643.870660:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (442368->446463) req@ffff9cf242b68700 x1866777609529088/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.870668:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b68700 with x1866777609529088 ext(442368->446463) 00002000:00010000:2.0:1780298643.870672:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf242b68700 00000100:00100000:1.0:1780298643.871537:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.871570:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609529216 00000100:00100000:0.0:1780298643.871574:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609529216 00002000:00100000:0.0:1780298643.871594:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x12:0x0] (446464->450559) req@ffff9cf242b6b100 x1866777609529216/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.871610:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b6b100 with x1866777609529216 ext(446464->450559) 00002000:00010000:0.0:1780298643.871615:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf242b6b100 00000100:00100000:0.0:1780298643.871639:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6214 00000100:00100000:0.0:1780298643.871648:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b6b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2398:x1866777609529216:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298643.871727:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b6b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2398:x1866777609529216:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 82us (191us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298643.871732:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6214 00002000:00100000:0.0:1780298643.871737:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x12:0x0] (446464->450559) req@ffff9cf242b6b100 x1866777609529216/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.871743:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b6b100 with x1866777609529216 ext(446464->450559) 00002000:00010000:0.0:1780298643.871746:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf242b6b100 00000100:00100000:2.0:1780298643.872812:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.872841:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.872857:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609529344 00000100:00100000:3.0:1780298643.872864:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609529344 00000100:00100000:1.0:1780298643.872893:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609529472 00000100:00100000:1.0:1780298643.872899:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609529472 00000100:00100000:2.0:1780298643.872932:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9599 00000100:00100000:2.0:1780298643.872946:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f736680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2398:x1866777609529344:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298643.872946:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9600 00010000:00010000:2.0:1780298643.872958:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298643.872962:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f734e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2398:x1866777609529472:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.872971:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003e31d8c5/0xaf35132e6303471c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d37bb expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.872972:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.872984:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e63034723 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d37c2 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.873013:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f735c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604980608:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.873023:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b6a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604980736:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.873071:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4b40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.873072:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837ae40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.873935:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f735c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604980608:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.873936:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b6a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604980736:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.873971:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003e31d8c5/0xaf35132e6303471c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d37bb expref: 15 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.873971:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e63034723 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d37c2 expref: 15 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.874002:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003e31d8c5/0xaf35132e6303471c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d37bb expref: 15 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.874002:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e63034723 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d37c2 expref: 15 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.874025:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279e00, rc 0) 00010000:00010000:2.0:1780298643.874026:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d08a00, rc 0) 00000100:00100000:0.0:1780298643.874057:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f734e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2398:x1866777609529472:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1101us (1216us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.874062:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9600 00000100:00100000:2.0:1780298643.874067:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f736680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2398:x1866777609529344:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1125us (1254us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.874075:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9599 00000100:00100000:2.0:1780298643.875060:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.875096:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609529600 00000100:00100000:0.0:1780298643.875102:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609529600 00000100:00100000:0.0:1780298643.875177:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9601 00000100:00100000:0.0:1780298643.875188:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f735c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2397:x1866777609529600:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298643.875196:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:0.0:1780298643.875202:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.875214:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e6303472a lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d37c9 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.875216:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609529728 00000100:00100000:2.0:1780298643.875222:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609529728 00000100:00100000:2.0:1780298643.875251:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9602 00000100:00100000:0.0:1780298643.875253:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343965c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604980864:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.875262:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b68380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2397:x1866777609529728:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.875271:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.875289:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006dfdf24b/0xaf35132e63034731 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d37d0 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.875315:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.875321:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f735f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604980992:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.875345:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4b40 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.875953:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343965c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604980864:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.875971:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e6303472a lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d37c9 expref: 15 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.875986:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e6303472a lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d37c9 expref: 15 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.875996:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baee00, rc 0) 00000100:00100000:0.0:1780298643.876029:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f735c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2397:x1866777609529600:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 844us (968us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.876037:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9601 00000100:00100000:2.0:1780298643.876065:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f735f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604980992:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.876084:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000006dfdf24b/0xaf35132e63034731 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d37d0 expref: 15 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.876099:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006dfdf24b/0xaf35132e63034731 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d37d0 expref: 15 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.876109:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d08200, rc 0) 00000100:00100000:2.0:1780298643.876162:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b68380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2397:x1866777609529728:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 903us (971us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.876167:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9602 00000100:00100000:3.0:1780298643.876919:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.876957:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609529856 00000100:00100000:1.0:1780298643.876961:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609529856 00000100:00100000:1.0:1780298643.876984:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9603 00000100:00100000:1.0:1780298643.876988:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:29360:x1866777609529856:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.876994:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.877001:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f4922155/0xaf35132e63034738 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d37d7 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.877013:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(160 177) 00010000:00010000:1.0:1780298643.877016:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(160 177) 00010000:00010000:1.0:1780298643.877023:0:20252:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f4922155/0xaf35132e63034738 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1110016->1581055] (req 1110016->1114111) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d37d7 expref: 15 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.877033:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f4922155/0xaf35132e63034738 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1110016->1581055] (req 1110016->1114111) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d37d7 expref: 15 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.877043:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278600, rc 0) 00000100:00100000:1.0:1780298643.877063:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:29360:x1866777609529856:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 74us (147us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.877067:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9603 00000100:00100000:1.0:1780298643.877676:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.877716:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609529984 00000100:00100000:2.0:1780298643.877721:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609529984 00000100:00100000:2.0:1780298643.877756:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6215 00000100:00100000:2.0:1780298643.877763:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b6a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2398:x1866777609529984:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298643.877948:0:16972:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:2.0:1780298643.877951:0:16972:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298643.877953:0:16972:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298643.877956:0:16972:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298643.877958:0:16972:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 408 -> 416 00000100:00100000:2.0:1780298643.877987:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b6a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2398:x1866777609529984:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 224us (309us total) trans 4294969696 rc 0/0 00000100:00100000:2.0:1780298643.877994:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6215 00000100:00100000:2.0:1780298643.878803:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.878856:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609530112 00000100:00100000:0.0:1780298643.878861:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609530112 00000100:00100000:0.0:1780298643.878893:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9604 00000100:00100000:0.0:1780298643.878902:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f735f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2398:x1866777609530112:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298643.878913:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:0.0:1780298643.878916:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.878925:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e6303473f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d37de expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.878940:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609530240 00000100:00100000:2.0:1780298643.878944:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609530240 00000100:00100000:0.0:1780298643.878960:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343965c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604981120:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.878977:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9605 00000100:00100000:2.0:1780298643.878985:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255091180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2397:x1866777609530240:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.878994:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.879003:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000debe9bb3/0xaf35132e63034746 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d37e5 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.879010:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.879041:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604981248:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.879066:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4b40 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.879713:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343965c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604981120:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.879731:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e6303473f lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d37de expref: 17 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.879746:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e6303473f lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d37de expref: 17 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.879757:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf600, rc 0) 00000100:00100000:0.0:1780298643.879788:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f735f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2398:x1866777609530112:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 888us (985us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.879796:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9604 00000100:00100000:2.0:1780298643.879873:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604981248:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.879890:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000debe9bb3/0xaf35132e63034746 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d37e5 expref: 15 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.879903:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000debe9bb3/0xaf35132e63034746 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d37e5 expref: 15 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.879912:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d09600, rc 0) 00000100:00100000:2.0:1780298643.879938:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255091180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2397:x1866777609530240:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 954us (1029us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.879945:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9605 00000100:00100000:1.0:1780298643.880763:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.880793:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609530368 00000100:00100000:2.0:1780298643.880799:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609530368 00000100:00100000:1.0:1780298643.880799:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.880824:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609530496 00000100:00100000:2.0:1780298643.880827:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609530496 00000100:00100000:0.0:1780298643.880835:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9606 00000100:00100000:0.0:1780298643.880846:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255090700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609530368:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.880854:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298643.880862:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9607 00010000:00010000:0.0:1780298643.880864:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e6303474d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d37ec expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.880875:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609530496:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.880883:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.880891:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008d64202a/0xaf35132e63034754 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d37f3 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.880902:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343965c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604981376:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.880916:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604981504:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.880945:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4b40 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.880945:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.881581:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343965c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604981376:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.881598:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e6303474d lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d37ec expref: 17 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.881609:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e6303474d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d37ec expref: 17 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.881617:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae400, rc 0) 00000100:00100000:0.0:1780298643.881641:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255090700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609530368:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 800us (880us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.881646:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9606 00000100:00100000:2.0:1780298643.881660:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604981504:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.881680:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008d64202a/0xaf35132e63034754 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d37f3 expref: 15 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.881701:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008d64202a/0xaf35132e63034754 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d37f3 expref: 15 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.881707:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d09e00, rc 0) 00000100:00100000:2.0:1780298643.881727:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609530496:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 858us (927us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.881734:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9607 00000100:00100000:1.0:1780298643.882641:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.882677:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609530624 00000100:00100000:0.0:1780298643.882682:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609530624 00000100:00100000:0.0:1780298643.882721:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9608 00000100:00100000:0.0:1780298643.882730:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254919500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609530624:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298643.882738:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:0.0:1780298643.882740:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.882750:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e6303475b lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d37fa expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.882761:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609530752 00000100:00100000:2.0:1780298643.882766:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609530752 00000100:00100000:0.0:1780298643.882785:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343965c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604981632:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.882800:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9609 00000100:00100000:2.0:1780298643.882810:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254918700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609530752:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.882819:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.882829:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042649f54/0xaf35132e63034762 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3801 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.882830:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.882855:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604981760:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.882884:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4b40 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.883418:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343965c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604981632:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.883430:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e6303475b lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d37fa expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.883444:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e6303475b lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d37fa expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.883451:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf000, rc 0) 00000100:00100000:0.0:1780298643.883474:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254919500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609530624:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 747us (839us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.883479:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9608 00000100:00100000:2.0:1780298643.883598:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604981760:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.883616:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000042649f54/0xaf35132e63034762 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3801 expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.883633:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042649f54/0xaf35132e63034762 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3801 expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.883639:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d08400, rc 0) 00000100:00100000:2.0:1780298643.883669:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254918700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609530752:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 863us (934us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.883673:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9609 00000100:00100000:1.0:1780298643.884476:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.884497:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.884512:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609530880 00000100:00100000:2.0:1780298643.884517:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609530880 00000100:00100000:0.0:1780298643.884524:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609531008 00000100:00100000:0.0:1780298643.884529:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609531008 00000100:00100000:2.0:1780298643.884545:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9610 00000100:00100000:2.0:1780298643.884553:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254919c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609530880:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.884560:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298643.884561:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9611 00010000:00010000:2.0:1780298643.884569:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007ba4a4a6/0xaf35132e63034769 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3808 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.884574:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609531008:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.884587:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.884595:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63034770 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d380f expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.884607:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604981888:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.884621:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254919880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604982016:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.884657:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4b40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.884657:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837ae40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.885400:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604981888:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.885412:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254919880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604982016:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.885427:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007ba4a4a6/0xaf35132e63034769 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3808 expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.885434:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63034770 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d380f expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.885451:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63034770 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d380f expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.885458:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007ba4a4a6/0xaf35132e63034769 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3808 expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.885459:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279200, rc 0) 00010000:00010000:2.0:1780298643.885477:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d09c00, rc 0) 00000100:00100000:3.0:1780298643.885487:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609531008:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 918us (988us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.885492:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9611 00000100:00100000:2.0:1780298643.885502:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254919c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609530880:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 949us (1023us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.885509:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9610 00000100:00100000:3.0:1780298643.886387:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.886423:0:7411:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609531264 00000100:00100000:1.0:1780298643.886427:0:7411:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609531264 00000100:00100000:1.0:1780298643.886454:0:7411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61998 00000100:00100000:1.0:1780298643.886458:0:7411:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255489180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+152:29360:x1866777609531264:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:1.0:1780298643.886494:0:7411:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298643.886509:0:7411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841c000/0xaf35132e6303459b lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 80 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3641 expref: 152 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.886518:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298643.886521:0:7411:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298643.886641:0:7411:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278e00/0xaf35132e63034777 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 79 type: IBT flags: 0x40000000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.886659:0:7411:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bec00/0xaf35132e630345a9 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 79 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3648 expref: 114 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.886674:0:7411:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278e00/0xaf35132e63034777 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 79 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.886690:0:7411:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bec00/0xaf35132e630345a9 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 79 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3648 expref: 114 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.886700:0:7411:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bec00/0xaf35132e630345a9 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 79 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3648 expref: 114 pid: 6477 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.886708:0:7411:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254919180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7411:1866777604982144:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298643.886737:0:7411:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.887261:0:7411:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254919180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7411:1866777604982144:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298643.887294:0:7411:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278e00/0xaf35132e63034777 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 79 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.887484:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.887517:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609531392 00000100:00100000:0.0:1780298643.887522:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609531392 00010000:00010000:0.0:1780298643.887543:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bec00/0xaf35132e630345a9 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 79 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3648 expref: 115 pid: 6477 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.887566:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15126 00000100:00100000:0.0:1780298643.887572:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254888700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+115:2400:x1866777609531392:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298643.887584:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298643.887588:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bec00/0xaf35132e630345a9 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 80 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3648 expref: 115 pid: 6477 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.887596:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bec00/0xaf35132e630345a9 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 80 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3648 expref: 115 pid: 6477 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.887606:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bec00/0xaf35132e630345a9 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 80 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3648 expref: 115 pid: 6477 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.887619:0:7400:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298643.887622:0:7400:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278e00/0xaf35132e63034777 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 79 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.887630:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278e00/0xaf35132e63034777 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 79 type: IBT flags: 0x40290000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.887635:0:7400:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278e00/0xaf35132e63034777 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 79 type: IBT flags: 0x40290000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.887651:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298643.887668:0:7411:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278e00/0xaf35132e63034777 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 78 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.887672:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254888700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+114:2400:x1866777609531392:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 100us (187us total) trans 0 rc 0/0 00010000:00010000:1.0:1780298643.887680:0:7411:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278e00/0xaf35132e63034777 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 78 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.887684:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15126 00000001:00080000:1.0:1780298643.887734:0:7411:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991060 00000004:00010000:1.0:1780298643.887755:0:7411:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000d071a581 reply state 000000006f6a007f transno 4294991061 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278e00/0xaf35132e63034777 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 78 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.887764:0:7411:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278e00/0xaf35132e63034777 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 78 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.887768:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298643.887769:0:7411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1780298643.887771:0:7411:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1780298643.887816:0:7411:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255489180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+154:29360:x1866777609531264:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1356us (1429us total) trans 4294991061 rc 0/0 00000100:00100000:1.0:1780298643.887820:0:7411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61998 00000020:00080000:1.0:1780298643.887824:0:7411:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1780298643.888400:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.888437:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609531520 00000100:00100000:1.0:1780298643.888441:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609531520 00000100:00100000:1.0:1780298643.888467:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9612 00000100:00100000:1.0:1780298643.888474:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254889c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:29360:x1866777609531520:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.888484:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.888493:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e6303477e lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3816 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.888511:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f4922155/0xaf35132e63034738 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1110016->1581055] (req 1110016->1114111) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d37d7 expref: 17 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.888524:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e630346d6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1581056->3719167] (req 1581056->1589247) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3775 expref: 17 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.888532:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e6303466d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3723264->37277695] (req 3723264->3735551) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d370c expref: 8 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.888538:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 3 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(160 177) 00010000:00010000:1.0:1780298643.888542:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(163 177) 00010000:00010000:1.0:1780298643.888546:0:20252:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e6303477e lrc: 6/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3816 expref: 17 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.888565:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e6303466d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3723264->37277695] (req 3723264->3735551) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d370c expref: 8 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.888573:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e6303466d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3723264->37277695] (req 3723264->3735551) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d370c expref: 8 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.888598:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254919180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604982272:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298643.888644:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e630346d6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1581056->3719167] (req 1581056->1589247) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3775 expref: 17 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.888653:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e630346d6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1581056->3719167] (req 1581056->1589247) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3775 expref: 17 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.888664:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25491bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604982400:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298643.888692:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f4922155/0xaf35132e63034738 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1110016->1581055] (req 1110016->1114111) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d37d7 expref: 17 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.888702:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f4922155/0xaf35132e63034738 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1110016->1581055] (req 1110016->1114111) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d37d7 expref: 17 pid: 20252 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.888713:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254919880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604982528:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298643.888744:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.889355:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254919180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604982272:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298643.889370:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25491bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604982400:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298643.889384:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254919880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604982528:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298643.889417:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e6303477e lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3816 expref: 17 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.889430:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279c00, rc 0) 00000100:00100000:1.0:1780298643.889465:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254889c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:29360:x1866777609531520:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 992us (1064us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.889473:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9612 00000100:00100000:3.0:1780298643.889551:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.889594:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609531776 00000100:00100000:0.0:1780298643.889600:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609531776 00000100:00100000:2.0:1780298643.889604:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:0.0:1780298643.889622:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000f4922155/0xaf35132e63034738 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1110016->1581055] (req 1110016->1114111) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d37d7 expref: 17 pid: 20252 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.889631:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609531648 00000100:00100000:1.0:1780298643.889637:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609531648 00000100:00100000:0.0:1780298643.889654:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15127 00002000:00100000:1.0:1780298643.889665:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (3723264->3735551) req@ffff9cf37f736d80 x1866777609531648/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 12776/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00000100:00100000:0.0:1780298643.889666:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254889180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2398:x1866777609531776:12345-192.168.203.25@tcp:103:ldlm_bl.0 00002000:00100000:1.0:1780298643.889678:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f736d80 with x1866777609531648 ext(3723264->3735551) 00010000:00010000:0.0:1780298643.889681:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298643.889684:0:8363:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e6303466d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3723264->37277695] (req 3723264->3735551) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d370c expref: 9 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.889694:0:8363:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e6303466d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [3723264->37277695] (req 3723264->3735551) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d370c expref: 9 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.889699:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f4922155/0xaf35132e63034738 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [1110016->1581055] (req 1110016->1114111) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d37d7 expref: 17 pid: 20252 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.889705:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00002000:00010000:1.0:1780298643.889711:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf37f736d80 00010000:00010000:0.0:1780298643.889722:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f4922155/0xaf35132e63034738 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [1110016->1581055] (req 1110016->1114111) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d37d7 expref: 17 pid: 20252 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.889732:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 199 00000100:00100000:2.0:1780298643.889741:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609531904 00000100:00100000:1.0:1780298643.889742:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f736d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609531648:12345-192.168.203.25@tcp:4:fsx.0 00010000:00010000:0.0:1780298643.889743:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f4922155/0xaf35132e63034738 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [1110016->1581055] (req 1110016->1114111) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d37d7 expref: 17 pid: 20252 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.889746:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609531904 00010000:00010000:0.0:1780298643.889758:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298643.889767:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e630346d6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1581056->3719167] (req 1581056->1589247) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3775 expref: 17 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.889793:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254889180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2398:x1866777609531776:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 130us (241us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.889794:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15128 00000100:00100000:0.0:1780298643.889809:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15127 00000100:00100000:2.0:1780298643.889810:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254889f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609531904:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298643.889821:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298643.889830:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e630346d6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1581056->3719167] (req 1581056->1589247) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3775 expref: 16 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.889842:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e630346d6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1581056->3719167] (req 1581056->1589247) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3775 expref: 16 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.889856:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e630346d6 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1581056->3719167] (req 1581056->1589247) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3775 expref: 16 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.889866:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298643.889894:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254889f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2400:x1866777609531904:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 90us (193us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.889901:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15128 00000100:00100000:1.0:1780298643.890223:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f736d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609531648:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 484us (622us total) trans 4294969697 rc 0/0 00000100:00100000:1.0:1780298643.890229:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 199 00002000:00100000:1.0:1780298643.890234:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (3723264->3735551) req@ffff9cf37f736d80 x1866777609531648/t4294969697(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 12776/448 e 0 to 0 dl 1780298654 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.890240:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f736d80 with x1866777609531648 ext(3723264->3735551) 00010000:00010000:1.0:1780298643.890245:0:8363:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e6303466d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3723264->37277695] (req 3723264->3735551) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d370c expref: 11 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.890258:0:8363:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e6303466d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3723264->37277695] (req 3723264->3735551) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d370c expref: 11 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1780298643.890266:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf37f736d80 00000100:00100000:3.0:1780298643.890890:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.890935:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609532032 00000100:00100000:2.0:1780298643.890940:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609532032 00010000:00010000:2.0:1780298643.890958:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e6303466d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3723264->37277695] (req 3723264->3735551) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d370c expref: 11 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.890988:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15129 00000100:00100000:2.0:1780298643.890996:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254889500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609532032:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298643.891005:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:2.0:1780298643.891012:0:8374:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 416 -> 440 00010000:00010000:2.0:1780298643.891018:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e6303466d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3723264->37277695] (req 3723264->3735551) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d370c expref: 11 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.891029:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e6303466d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3723264->37277695] (req 3723264->3735551) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d370c expref: 11 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.891042:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e6303466d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3723264->37277695] (req 3723264->3735551) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d370c expref: 11 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.891053:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(163 177) 00010000:00010000:2.0:1780298643.891056:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(163 177) 00010000:00010000:2.0:1780298643.891062:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e6303477e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3816 expref: 14 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.891073:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e6303477e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3816 expref: 14 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.891095:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e6303477e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3816 expref: 14 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.891105:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f737b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777604982656:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298643.891171:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4b40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.891847:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f737b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777604982656:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298643.891857:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298643.891878:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254889500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609532032:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 884us (988us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.891883:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15129 00000100:00100000:2.0:1780298643.891983:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.892021:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609532160 00000100:00100000:1.0:1780298643.892026:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609532160 00000100:00100000:1.0:1780298643.892055:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9613 00000100:00100000:1.0:1780298643.892062:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f737b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:29360:x1866777609532160:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.892071:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.892079:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007fddc766/0xaf35132e63034785 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d381d expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.892092:0:19992:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000fba32a6f/0xaf35132e63034715 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [442368->33996799] (req 442368->450559) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d37b4 expref: 15 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.892097:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(119 130) 00010000:00010000:1.0:1780298643.892099:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(120 130) 00010000:00010000:1.0:1780298643.892102:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007fddc766/0xaf35132e63034785 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d381d expref: 15 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.892115:0:19992:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000fba32a6f/0xaf35132e63034715 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [442368->33996799] (req 442368->450559) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d37b4 expref: 15 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.892138:0:19992:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000fba32a6f/0xaf35132e63034715 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [442368->33996799] (req 442368->450559) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d37b4 expref: 15 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.892149:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254919880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604982784:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298643.892184:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.892783:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254919880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604982784:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298643.892800:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007fddc766/0xaf35132e63034785 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d381d expref: 15 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.892812:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278000, rc 0) 00000100:00100000:1.0:1780298643.892843:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f737b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:29360:x1866777609532160:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 781us (858us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.892850:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9613 00000100:00100000:3.0:1780298643.893044:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.893090:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609532288 00000100:00100000:2.0:1780298643.893095:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609532288 00002000:00100000:2.0:1780298643.893112:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x12:0x0] (442368->450559) req@ffff9cf254888000 x1866777609532288/t0(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 8680/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.893119:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf254888000 with x1866777609532288 ext(442368->450559) 00010000:00010000:2.0:1780298643.893135:0:19928:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000fba32a6f/0xaf35132e63034715 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [442368->33996799] (req 442368->450559) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d37b4 expref: 15 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.893149:0:19928:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000fba32a6f/0xaf35132e63034715 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [442368->33996799] (req 442368->450559) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d37b4 expref: 15 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1780298643.893154:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9cf254888000 00000100:00100000:2.0:1780298643.893173:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 200 00000100:00100000:2.0:1780298643.893181:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254888000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2399:x1866777609532288:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:2.0:1780298643.893594:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254888000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2399:x1866777609532288:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 412us (549us total) trans 4294969246 rc 0/0 00000100:00100000:2.0:1780298643.893602:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 200 00002000:00100000:2.0:1780298643.893608:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x12:0x0] (442368->450559) req@ffff9cf254888000 x1866777609532288/t4294969246(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 8680/448 e 0 to 0 dl 1780298654 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.893617:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf254888000 with x1866777609532288 ext(442368->450559) 00010000:00010000:2.0:1780298643.893622:0:19928:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000fba32a6f/0xaf35132e63034715 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [442368->33996799] (req 442368->450559) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d37b4 expref: 17 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.893634:0:19928:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000fba32a6f/0xaf35132e63034715 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [442368->33996799] (req 442368->450559) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d37b4 expref: 17 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1780298643.893641:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9cf254888000 00000100:00100000:2.0:1780298643.894429:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.894468:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609532416 00000100:00100000:0.0:1780298643.894473:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609532416 00010000:00010000:0.0:1780298643.894494:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000fba32a6f/0xaf35132e63034715 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [442368->33996799] (req 442368->450559) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d37b4 expref: 17 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.894524:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15130 00000100:00100000:0.0:1780298643.894532:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f737800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609532416:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298643.894542:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:0.0:1780298643.894551:0:6463:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 1 -> 16 00010000:00010000:0.0:1780298643.894557:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000fba32a6f/0xaf35132e63034715 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [442368->33996799] (req 442368->450559) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d37b4 expref: 17 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.894568:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000fba32a6f/0xaf35132e63034715 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [442368->33996799] (req 442368->450559) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d37b4 expref: 17 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.894592:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fba32a6f/0xaf35132e63034715 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [442368->33996799] (req 442368->450559) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d37b4 expref: 17 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.894608:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(120 130) 00010000:00010000:0.0:1780298643.894612:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(120 130) 00010000:00010000:0.0:1780298643.894618:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000007fddc766/0xaf35132e63034785 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d381d expref: 16 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.894628:0:6463:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007fddc766/0xaf35132e63034785 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d381d expref: 16 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.894653:0:6463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000007fddc766/0xaf35132e63034785 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d381d expref: 16 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.894663:0:6463:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343965c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777604982912:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298643.894708:0:6463:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.895319:0:6463:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343965c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777604982912:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298643.895333:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298643.895363:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f737800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609532416:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 832us (934us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.895371:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15130 00000100:00100000:1.0:1780298643.895426:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.895462:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609532672 00000100:00100000:2.0:1780298643.895466:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609532672 00002000:00010000:2.0:1780298643.895483:0:19951:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (201564->18446744073709551615) 00002000:00100000:2.0:1780298643.895487:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf254919880 with x1866777609532672 ext(201564->18446744073709551615) 00002000:00010000:2.0:1780298643.895495:0:19951:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf254919880. 00000100:00100000:2.0:1780298643.895526:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6216 00000100:00100000:2.0:1780298643.895538:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254919880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609532672:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:1.0:1780298643.895552:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.895577:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609532544 00000100:00100000:0.0:1780298643.895581:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609532544 00002000:00010000:0.0:1780298643.895593:0:19943:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:0.0:1780298643.895596:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25491bb80 with x1866777609532544 ext(4194304->18446744073709551615) 00002000:00010000:0.0:1780298643.895601:0:19943:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf25491bb80. 00000100:00100000:0.0:1780298643.895618:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6217 00000100:00100000:0.0:1780298643.895624:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2400:x1866777609532544:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:0.0:1780298643.895682:0:19943:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:0.0:1780298643.895684:0:19943:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298643.895686:0:19943:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298643.895688:0:19943:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298643.895690:0:19943:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 440 -> 440 00000100:00100000:0.0:1780298643.895715:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609532544:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 92us (165us total) trans 4294969698 rc 0/0 00000100:00100000:0.0:1780298643.895719:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6217 00002000:00010000:0.0:1780298643.895722:0:19943:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:0.0:1780298643.895724:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25491bb80 with x1866777609532544 ext(4194304->18446744073709551615) 00002000:00010000:0.0:1780298643.895728:0:19943:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf25491bb80. 00002000:00010000:2.0:1780298643.895745:0:19951:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 2111883 -> 201564 00002000:00010000:2.0:1780298643.895750:0:19951:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298643.895752:0:19951:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298643.895754:0:19951:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298643.895757:0:19951:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 16 -> 0 00000100:00100000:2.0:1780298643.895781:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254919880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609532672:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 246us (357us total) trans 4294969247 rc 0/0 00000100:00100000:2.0:1780298643.895788:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6216 00002000:00010000:2.0:1780298643.895792:0:19951:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (201564->18446744073709551615) 00002000:00100000:2.0:1780298643.895795:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf254919880 with x1866777609532672 ext(201564->18446744073709551615) 00002000:00010000:2.0:1780298643.895800:0:19951:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf254919880. 00000100:00100000:3.0:1780298643.896556:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.896594:0:6476:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609532800 00000100:00100000:2.0:1780298643.896598:0:6476:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609532800 00000100:00100000:2.0:1780298643.896633:0:6476:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 61999 00000100:00100000:2.0:1780298643.896645:0:6476:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25488bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+114:29360:x1866777609532800:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.896655:0:6476:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.896668:0:6476:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08400/0xaf35132e6303478c lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 79 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3824 expref: -99 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.896743:0:6476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09e00/0xaf35132e63034793 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 80 type: IBT flags: 0x40000000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.896758:0:6476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09e00/0xaf35132e63034793 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 80 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.896772:0:6476:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09e00/0xaf35132e63034793 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 80 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298643.896796:0:6476:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 4395868 00000004:00200000:2.0:1780298643.896800:0:6476:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298643.896815:0:6476:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09e00/0xaf35132e63034793 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 80 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.896836:0:6476:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08400/0xaf35132e6303478c lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 80 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3824 expref: 116 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.896851:0:6476:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09e00/0xaf35132e63034793 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 79 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3824 expref: 115 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.896859:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298643.896862:0:6476:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298643.896865:0:6476:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d09e00, rc 0) 00000100:00100000:2.0:1780298643.896895:0:6476:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25488bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+115:29360:x1866777609532800:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 253us (340us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.896903:0:6476:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 61999 00000100:00100000:1.0:1780298643.897908:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.897929:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.897945:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609532928 00000100:00100000:0.0:1780298643.897950:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609532928 00000100:00100000:3.0:1780298643.897968:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609533056 00000100:00100000:3.0:1780298643.897974:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609533056 00000100:00100000:0.0:1780298643.897978:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9614 00000100:00100000:0.0:1780298643.897985:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254919180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609532928:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.897991:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.898000:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e6303479a lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d382b expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.898011:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9615 00000100:00100000:3.0:1780298643.898020:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243844000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609533056:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.898030:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298643.898037:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343965c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604983040:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.898040:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bed20494/0xaf35132e630347a1 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3832 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.898078:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417d80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.898081:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254888a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604983168:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.898107:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.898832:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254888a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604983168:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.898843:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343965c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604983040:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.898852:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bed20494/0xaf35132e630347a1 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3832 expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.898860:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e6303479a lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d382b expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.898872:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e6303479a lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d382b expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.898875:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bed20494/0xaf35132e630347a1 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3832 expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.898903:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157000, rc 0) 00010000:00010000:0.0:1780298643.898903:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf600, rc 0) 00000100:00100000:0.0:1780298643.898933:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254919180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609532928:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 950us (1026us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.898939:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243844000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609533056:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 919us (1006us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.898943:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9614 00000100:00100000:3.0:1780298643.898951:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9615 00000100:00100000:2.0:1780298643.899812:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.899838:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.899852:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609533184 00000100:00100000:1.0:1780298643.899857:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609533184 00000100:00100000:3.0:1780298643.899872:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609533312 00000100:00100000:3.0:1780298643.899877:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609533312 00000100:00100000:1.0:1780298643.899887:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9616 00000100:00100000:1.0:1780298643.899895:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f737480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609533184:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.899902:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.899910:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e630347a8 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3839 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.899919:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9617 00000100:00100000:2.0:1780298643.899930:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e4f1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609533312:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.899940:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298643.899940:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243846d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604983296:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.899951:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e31d8c5/0xaf35132e630347af lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3840 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.899983:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604983424:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.899988:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.900010:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.900561:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243846d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604983296:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.900595:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e630347a8 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3839 expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.900605:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e630347a8 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3839 expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.900613:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279800, rc 0) 00000100:00100000:3.0:1780298643.900643:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f737480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609533184:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 749us (830us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.900650:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9616 00000100:00100000:1.0:1780298643.900789:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604983424:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.900804:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003e31d8c5/0xaf35132e630347af lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3840 expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.900813:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e31d8c5/0xaf35132e630347af lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3840 expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.900829:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d08a00, rc 0) 00000100:00100000:1.0:1780298643.900850:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e4f1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609533312:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 925us (1012us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.900857:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9617 00000100:00100000:2.0:1780298643.901755:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.901794:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609533440 00000100:00100000:0.0:1780298643.901798:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609533440 00000100:00100000:0.0:1780298643.901829:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6218 00000100:00100000:0.0:1780298643.901837:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f734a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609533440:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298643.902009:0:16972:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:0.0:1780298643.902012:0:16972:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298643.902013:0:16972:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298643.902015:0:16972:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298643.902016:0:16972:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 440 -> 456 00000100:00100000:0.0:1780298643.902040:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f734a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2397:x1866777609533440:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 206us (286us total) trans 4294969699 rc 0/0 00000100:00100000:0.0:1780298643.902048:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6218 00000100:00100000:0.0:1780298643.902823:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.902862:0:17578:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609533568 00000100:00100000:1.0:1780298643.902867:0:17578:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609533568 00000100:00100000:1.0:1780298643.902902:0:17578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62000 00000100:00100000:1.0:1780298643.902911:0:17578:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343965c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+154:29360:x1866777609533568:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.902925:0:17578:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.902938:0:17578:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278400/0xaf35132e630347b6 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 80 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3847 expref: -99 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.903008:0:17578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278600/0xaf35132e630347bd lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 81 type: IBT flags: 0x40000000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.903022:0:17578:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278600/0xaf35132e630347bd lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 81 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.903033:0:17578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278600/0xaf35132e630347bd lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 81 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1780298643.903057:0:17578:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 4395868 00000004:00200000:1.0:1780298643.903061:0:17578:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1780298643.903075:0:17578:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278600/0xaf35132e630347bd lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 81 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.903095:0:17578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278400/0xaf35132e630347b6 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 81 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3847 expref: 156 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.903110:0:17578:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278600/0xaf35132e630347bd lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 80 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3847 expref: 155 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.903118:0:17578:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298643.903121:0:17578:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1780298643.903146:0:17578:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278600, rc 0) 00000100:00100000:1.0:1780298643.903174:0:17578:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343965c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+155:29360:x1866777609533568:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 265us (353us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.903182:0:17578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62000 00000100:00100000:1.0:1780298643.904509:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.904545:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609533696 00000100:00100000:0.0:1780298643.904549:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609533696 00000100:00100000:0.0:1780298643.904576:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9618 00000100:00100000:0.0:1780298643.904584:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243844000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:29360:x1866777609533696:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.904593:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.904603:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e630347c4 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d384e expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.904630:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e6303477e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3816 expref: 16 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.904636:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(163 177) 00010000:00010000:0.0:1780298643.904640:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(164 177) 00010000:00010000:0.0:1780298643.904643:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e630347c4 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3178496->3186687] (req 3178496->3186687) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d384e expref: 11 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.904663:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e6303477e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3816 expref: 16 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.904672:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e6303477e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3816 expref: 16 pid: 20252 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.904680:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343965880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604983552:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298643.904724:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417d80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.906359:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343965880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604983552:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298643.906385:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e630347c4 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3178496->3186687] (req 3178496->3186687) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d384e expref: 11 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.906403:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baee00, rc 0) 00000100:00100000:0.0:1780298643.906447:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243844000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:29360:x1866777609533696:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1863us (1939us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298643.906459:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9618 00000100:00100000:3.0:1780298643.906794:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.906838:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609533824 00000100:00100000:0.0:1780298643.906843:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609533824 00010000:00010000:0.0:1780298643.906866:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e6303477e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3816 expref: 17 pid: 20252 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.906897:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15131 00000100:00100000:0.0:1780298643.906904:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254888a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609533824:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298643.906914:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298643.906930:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e6303477e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3816 expref: 17 pid: 20252 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.906951:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e6303477e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3816 expref: 17 pid: 20252 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.906969:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e6303477e lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3816 expref: 17 pid: 20252 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.906978:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(164 177) 00010000:00010000:0.0:1780298643.906981:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(164 177) 00010000:00010000:0.0:1780298643.906987:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e630347c4 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3178496->3186687) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d384e expref: 10 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.906993:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e630347c4 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3178496->3186687) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d384e expref: 10 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.907026:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e630347c4 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3178496->3186687) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d384e expref: 10 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.907041:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343965880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777604983680:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298643.907096:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417d80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.908299:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343965880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777604983680:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298643.908324:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298643.908379:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254888a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609533824:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1475us (1585us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298643.908391:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15131 00000100:00100000:3.0:1780298643.908604:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.908658:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609533952 00000100:00100000:1.0:1780298643.908666:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609533952 00002000:00100000:1.0:1780298643.908712:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (3178496->3186687) req@ffff9cf254888380 x1866777609533952/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.908723:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf254888380 with x1866777609533952 ext(3178496->3186687) 00002000:00010000:1.0:1780298643.908728:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf254888380 00000100:00100000:1.0:1780298643.908755:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6219 00000100:00100000:1.0:1780298643.908765:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254888380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609533952:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298643.908994:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254888380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609533952:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 230us (390us total) trans 0 rc 8192/8192 00000100:00100000:1.0:1780298643.909001:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6219 00002000:00100000:1.0:1780298643.909007:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (3178496->3186687) req@ffff9cf254888380 x1866777609533952/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/8632 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 8192/8192 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.909022:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf254888380 with x1866777609533952 ext(3178496->3186687) 00002000:00010000:1.0:1780298643.909025:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf254888380 00000100:00100000:2.0:1780298643.910173:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.910229:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609534080 00000100:00100000:3.0:1780298643.910236:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609534080 00000100:00100000:3.0:1780298643.910278:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9619 00000100:00100000:3.0:1780298643.910299:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f735880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2399:x1866777609534080:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.910311:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.910328:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e630347cb lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3855 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.910353:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(164 177) 00010000:00010000:3.0:1780298643.910357:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(164 177) 00010000:00010000:3.0:1780298643.910366:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e630347cb lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3855 expref: 17 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.910385:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e630347cb lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3855 expref: 17 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.910396:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157000, rc 0) 00000100:00100000:3.0:1780298643.910433:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f735880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2399:x1866777609534080:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 138us (307us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298643.910440:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9619 00000100:00100000:0.0:1780298643.911773:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.911826:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609534208 00000100:00100000:2.0:1780298643.911831:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609534208 00000100:00100000:2.0:1780298643.911863:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9620 00000100:00100000:2.0:1780298643.911871:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343965880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:29360:x1866777609534208:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.911883:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.911894:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000debe9bb3/0xaf35132e630347d2 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d385c expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.911917:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e630347c4 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3178496->3186687) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d384e expref: 10 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.911926:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e630347cb lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3855 expref: 18 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.911934:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(164 177) 00010000:00010000:2.0:1780298643.911937:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(164 177) 00010000:00010000:2.0:1780298643.911942:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000debe9bb3/0xaf35132e630347d2 lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [266240->274431] (req 266240->274431) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d385c expref: 18 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.911972:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e630347cb lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3855 expref: 18 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.911986:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e630347cb lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3855 expref: 18 pid: 9128 timeout: 933 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.912012:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f734700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604983808:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298643.912090:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e630347c4 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3178496->3186687) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d384e expref: 10 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.912101:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e630347c4 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3178496->3186687) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d384e expref: 10 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.912112:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f734e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604983936:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298643.912172:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4600 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.912833:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f734700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604983808:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298643.912843:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f734e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604983936:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298643.912868:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000debe9bb3/0xaf35132e630347d2 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [266240->274431] (req 266240->274431) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d385c expref: 18 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.912882:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d09600, rc 0) 00000100:00100000:2.0:1780298643.912914:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343965880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:29360:x1866777609534208:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1044us (1141us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.912921:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9620 00000100:00100000:3.0:1780298643.913200:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.913244:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609534336 00000100:00100000:0.0:1780298643.913251:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609534336 00010000:00010000:0.0:1780298643.913275:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e630347cb lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3855 expref: 18 pid: 9128 timeout: 933 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.913321:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15132 00000100:00100000:2.0:1780298643.913329:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.913341:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b5d850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609534336:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298643.913352:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1780298643.913363:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609534464 00010000:00010000:0.0:1780298643.913375:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e630347cb lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3855 expref: 18 pid: 9128 timeout: 933 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.913391:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e630347cb lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3855 expref: 18 pid: 9128 timeout: 933 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.913407:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609534464 00010000:00010000:0.0:1780298643.913413:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e630347cb lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3855 expref: 18 pid: 9128 timeout: 933 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.913432:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298643.913467:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b5d850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609534336:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 133us (275us total) trans 0 rc 0/0 00010000:00010000:2.0:1780298643.913477:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e630347c4 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3178496->3186687) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d384e expref: 11 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.913480:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15132 00000100:00100000:2.0:1780298643.913523:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15133 00000100:00100000:2.0:1780298643.913533:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f734e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609534464:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298643.913546:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298643.913560:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e630347c4 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3178496->3186687) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d384e expref: 11 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.913573:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e630347c4 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3178496->3186687) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d384e expref: 11 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.913592:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e630347c4 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3178496->3186687) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d384e expref: 11 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.913606:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(164 177) 00010000:00010000:2.0:1780298643.913610:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(164 177) 00010000:00010000:2.0:1780298643.913618:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000debe9bb3/0xaf35132e630347d2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [266240->33820671] (req 266240->274431) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d385c expref: 16 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.913627:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000debe9bb3/0xaf35132e630347d2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [266240->33820671] (req 266240->274431) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d385c expref: 16 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.913652:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000debe9bb3/0xaf35132e630347d2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [266240->33820671] (req 266240->274431) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d385c expref: 16 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.913668:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f734700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777604984064:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298643.913708:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.914360:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f734700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777604984064:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298643.914373:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298643.914400:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f734e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609534464:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 870us (1074us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298643.914408:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15133 00000100:00100000:0.0:1780298643.914550:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.914598:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609534592 00000100:00100000:2.0:1780298643.914604:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609534592 00000100:00100000:2.0:1780298643.914648:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6220 00000100:00100000:2.0:1780298643.914658:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343966d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609534592:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298643.914868:0:19928:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:2.0:1780298643.914872:0:19928:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298643.914874:0:19928:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298643.914877:0:19928:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298643.914880:0:19928:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 456 -> 472 00000100:00100000:2.0:1780298643.914913:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343966d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609534592:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 258us (362us total) trans 4294969700 rc 0/0 00000100:00100000:2.0:1780298643.914920:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6220 00000100:00100000:0.0:1780298643.915797:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.915843:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609534720 00000100:00100000:1.0:1780298643.915849:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609534720 00000100:00100000:1.0:1780298643.915885:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9621 00000100:00100000:1.0:1780298643.915895:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343965f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2397:x1866777609534720:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.915903:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.915915:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e630347d9 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3863 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.915956:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243844a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604984192:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.916020:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.916692:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243844a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604984192:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.916716:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e630347d9 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3863 expref: 19 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.916734:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e630347d9 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3863 expref: 19 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.916746:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278400, rc 0) 00000100:00100000:1.0:1780298643.916780:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343965f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2397:x1866777609534720:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 888us (982us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.916787:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9621 00000100:00100000:3.0:1780298643.917640:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.917685:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609534848 00000100:00100000:0.0:1780298643.917691:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609534848 00000100:00100000:0.0:1780298643.917727:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9622 00000100:00100000:0.0:1780298643.917734:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25488a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2398:x1866777609534848:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.917745:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.917754:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000058e2d0/0xaf35132e630347e0 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d386a expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.917784:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343964000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604984320:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.917833:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.918521:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343964000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604984320:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.918545:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000058e2d0/0xaf35132e630347e0 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d386a expref: 19 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.918555:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000058e2d0/0xaf35132e630347e0 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d386a expref: 19 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.918563:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baea00, rc 0) 00000100:00100000:1.0:1780298643.918588:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25488a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2398:x1866777609534848:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 855us (955us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.918593:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9622 00000100:00100000:2.0:1780298643.919573:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.919628:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609534976 00000100:00100000:3.0:1780298643.919637:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609534976 00000100:00100000:3.0:1780298643.919688:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9623 00000100:00100000:3.0:1780298643.919702:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f736680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2398:x1866777609534976:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.919719:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.919734:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000eae09c9b/0xaf35132e630347e7 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3871 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.919787:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25488ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604984448:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.919860:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.920587:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25488ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604984448:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.920613:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000eae09c9b/0xaf35132e630347e7 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3871 expref: 19 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.920632:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000eae09c9b/0xaf35132e630347e7 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3871 expref: 19 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.920644:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157a00, rc 0) 00000100:00100000:1.0:1780298643.920687:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f736680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2398:x1866777609534976:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 990us (1115us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.920695:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9623 00000100:00100000:0.0:1780298643.921636:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.921671:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609535104 00000100:00100000:1.0:1780298643.921677:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609535104 00000100:00100000:1.0:1780298643.921720:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6221 00000100:00100000:1.0:1780298643.921725:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343966a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2397:x1866777609535104:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298643.921928:0:8364:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 201564 -> 201564 00002000:00010000:1.0:1780298643.921931:0:8364:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298643.921933:0:8364:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298643.921934:0:8364:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298643.921936:0:8364:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 0 -> 16 00000100:00100000:1.0:1780298643.921978:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343966a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2397:x1866777609535104:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 247us (338us total) trans 4294969248 rc 0/0 00000100:00100000:1.0:1780298643.921986:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6221 00000100:00100000:2.0:1780298643.922898:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.922955:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609535232 00000100:00100000:0.0:1780298643.922961:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609535232 00000100:00100000:0.0:1780298643.922993:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9624 00000100:00100000:0.0:1780298643.923001:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f735c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2397:x1866777609535232:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.923013:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.923025:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005d10d3aa/0xaf35132e630347ee lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3878 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.923073:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343966300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604984576:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.923167:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417e40 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.923915:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343966300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604984576:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.923932:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005d10d3aa/0xaf35132e630347ee lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3878 expref: 19 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.923944:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005d10d3aa/0xaf35132e630347ee lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3878 expref: 19 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.923951:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baec00, rc 0) 00000100:00100000:0.0:1780298643.923984:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f735c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2397:x1866777609535232:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 984us (1090us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.923991:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9624 00000100:00100000:1.0:1780298643.924859:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.924910:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609535360 00000100:00100000:2.0:1780298643.924915:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609535360 00000100:00100000:2.0:1780298643.924953:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9625 00000100:00100000:2.0:1780298643.924964:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243844a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2398:x1866777609535360:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.924974:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.924983:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007ba4a4a6/0xaf35132e630347f5 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d387f expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.925019:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f735f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604984704:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.925067:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4b40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.925834:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f735f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604984704:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.925877:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007ba4a4a6/0xaf35132e630347f5 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d387f expref: 19 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.925895:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007ba4a4a6/0xaf35132e630347f5 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d387f expref: 19 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.925904:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d09c00, rc 0) 00000100:00100000:1.0:1780298643.925953:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243844a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2398:x1866777609535360:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 991us (1092us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.925962:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9625 00000100:00100000:3.0:1780298643.927070:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.927211:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.927270:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609535488 00000100:00100000:2.0:1780298643.927277:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609535488 00000100:00100000:1.0:1780298643.927337:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609535616 00000100:00100000:2.0:1780298643.927342:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9626 00000100:00100000:1.0:1780298643.927342:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609535616 00000100:00100000:2.0:1780298643.927354:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25488aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609535488:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.927370:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298643.927382:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9627 00010000:00010000:2.0:1780298643.927386:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000042649f54/0xaf35132e630347fc lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3886 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.927394:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25488a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609535616:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.927406:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.927419:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63034803 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d388d expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.927441:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f737b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604984832:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.927454:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243846300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604984960:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.927517:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4b40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.927518:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.928359:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243846300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604984960:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.928378:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f737b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604984832:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.928389:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63034803 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d388d expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.928398:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000042649f54/0xaf35132e630347fc lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3886 expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.928413:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63034803 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d388d expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.928414:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000042649f54/0xaf35132e630347fc lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3886 expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.928433:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278a00, rc 0) 00010000:00010000:2.0:1780298643.928440:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d08400, rc 0) 00000100:00100000:1.0:1780298643.928468:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25488a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609535616:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1079us (1258us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.928476:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25488aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609535488:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1123us (1408us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.928476:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9627 00000100:00100000:2.0:1780298643.928488:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9626 00000100:00100000:0.0:1780298643.929543:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.929593:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609535744 00000100:00100000:2.0:1780298643.929599:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609535744 00000100:00100000:2.0:1780298643.929632:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9628 00000100:00100000:2.0:1780298643.929640:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343966300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609535744:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.929651:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.929662:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003e31d8c5/0xaf35132e6303480a lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3894 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.929701:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f737b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604985088:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.929715:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.929752:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4b40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.929765:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609535872 00000100:00100000:1.0:1780298643.929770:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609535872 00000100:00100000:1.0:1780298643.929804:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9629 00000100:00100000:1.0:1780298643.929814:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343964380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609535872:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.929822:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.929831:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63034811 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d389b expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.929859:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243846300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604985216:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.929906:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a6c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.930573:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f737b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604985088:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.930581:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243846300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604985216:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.930597:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003e31d8c5/0xaf35132e6303480a lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3894 expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.930603:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63034811 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d389b expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.930618:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003e31d8c5/0xaf35132e6303480a lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3894 expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.930619:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63034811 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d389b expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.930640:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d08a00, rc 0) 00010000:00010000:1.0:1780298643.930640:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279200, rc 0) 00000100:00100000:2.0:1780298643.930675:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343966300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609535744:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1034us (1133us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.930676:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343964380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609535872:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 863us (959us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.930688:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9628 00000100:00100000:1.0:1780298643.930694:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9629 00000100:00100000:0.0:1780298643.931904:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.931951:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609536000 00000100:00100000:1.0:1780298643.931957:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609536000 00000100:00100000:1.0:1780298643.931984:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9630 00000100:00100000:1.0:1780298643.931998:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343964a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:29360:x1866777609536000:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.932009:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.932021:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63034818 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d38a2 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.932043:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000debe9bb3/0xaf35132e630347d2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [266240->33820671] (req 266240->274431) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d385c expref: 17 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.932053:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(164 177) 00010000:00010000:1.0:1780298643.932057:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(165 177) 00010000:00010000:1.0:1780298643.932064:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63034818 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3031040->3039231] (req 3031040->3039231) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38a2 expref: 11 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.932087:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000debe9bb3/0xaf35132e630347d2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [266240->33820671] (req 266240->274431) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d385c expref: 17 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.932102:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000debe9bb3/0xaf35132e630347d2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [266240->33820671] (req 266240->274431) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d385c expref: 17 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.932114:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243846300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604985344:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298643.932192:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.932916:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243846300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604985344:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298643.932934:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63034818 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3031040->3039231] (req 3031040->3039231) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38a2 expref: 11 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.932946:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278200, rc 0) 00000100:00100000:1.0:1780298643.932977:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343964a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:29360:x1866777609536000:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 987us (1078us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.932983:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9630 00000100:00100000:0.0:1780298643.933073:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.933119:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609536128 00000100:00100000:2.0:1780298643.933140:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609536128 00010000:00010000:2.0:1780298643.933161:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000debe9bb3/0xaf35132e630347d2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [266240->33820671] (req 266240->274431) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d385c expref: 18 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.933205:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15134 00000100:00100000:2.0:1780298643.933215:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343964000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609536128:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298643.933226:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298643.933239:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000debe9bb3/0xaf35132e630347d2 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [266240->33820671] (req 266240->274431) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d385c expref: 18 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.933249:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000debe9bb3/0xaf35132e630347d2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [266240->33820671] (req 266240->274431) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d385c expref: 18 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.933262:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000debe9bb3/0xaf35132e630347d2 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [266240->33820671] (req 266240->274431) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d385c expref: 18 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.933269:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(165 177) 00010000:00010000:2.0:1780298643.933272:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(165 177) 00010000:00010000:2.0:1780298643.933277:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63034818 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3031040->36585471] (req 3031040->3039231) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38a2 expref: 10 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.933297:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63034818 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3031040->36585471] (req 3031040->3039231) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38a2 expref: 10 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.933316:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63034818 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3031040->36585471] (req 3031040->3039231) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38a2 expref: 10 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.933323:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f737b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777604985472:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298643.933355:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4b40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.934147:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f737b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777604985472:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298643.934164:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298643.934190:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343964000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609536128:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 978us (1118us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.934199:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15134 00000100:00100000:0.0:1780298643.934242:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.934279:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609536256 00000100:00100000:2.0:1780298643.934293:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609536256 00000100:00100000:2.0:1780298643.934337:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6222 00000100:00100000:2.0:1780298643.934347:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609536256:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298643.934513:0:19951:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:2.0:1780298643.934516:0:19951:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298643.934517:0:19951:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298643.934519:0:19951:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298643.934521:0:19951:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 472 -> 472 00000100:00100000:2.0:1780298643.934543:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609536256:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 200us (304us total) trans 4294969701 rc 0/0 00000100:00100000:2.0:1780298643.934547:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6222 00000100:00100000:3.0:1780298643.935723:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.935779:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609536384 00000100:00100000:2.0:1780298643.935787:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609536384 00000100:00100000:3.0:1780298643.935789:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.935823:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609536512 00000100:00100000:2.0:1780298643.935825:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609536512 00000100:00100000:2.0:1780298643.935861:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9631 00000100:00100000:0.0:1780298643.935878:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9632 00000100:00100000:2.0:1780298643.935879:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25488b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609536384:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298643.935899:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25488ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609536512:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.935900:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.935916:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.935924:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fba32a6f/0xaf35132e6303481f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d38a9 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.935929:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d4d8e3d7/0xaf35132e63034826 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d38b0 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.935997:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481ff800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604985600:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.936016:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f737b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604985728:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.936051:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417e40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.936081:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4b40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.936926:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f737b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604985728:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.936938:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481ff800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604985600:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.936968:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000fba32a6f/0xaf35132e6303481f lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38a9 expref: 13 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.936976:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d4d8e3d7/0xaf35132e63034826 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38b0 expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.936989:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fba32a6f/0xaf35132e6303481f lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38a9 expref: 13 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.936998:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d4d8e3d7/0xaf35132e63034826 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38b0 expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.937001:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d08600, rc 0) 00010000:00010000:0.0:1780298643.937021:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf000, rc 0) 00000100:00100000:2.0:1780298643.937044:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25488b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609536384:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1174us (1330us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.937050:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9631 00000100:00100000:0.0:1780298643.937059:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25488ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609536512:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1168us (1268us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298643.937068:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9632 00000100:00100000:1.0:1780298643.938360:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.938400:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.938417:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609536640 00000100:00100000:2.0:1780298643.938424:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609536640 00000100:00100000:0.0:1780298643.938452:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609536768 00000100:00100000:0.0:1780298643.938457:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609536768 00000100:00100000:2.0:1780298643.938473:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9633 00000100:00100000:2.0:1780298643.938482:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243846300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609536640:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.938493:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298643.938508:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9634 00010000:00010000:2.0:1780298643.938516:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000042649f54/0xaf35132e6303482d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d38b7 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.938524:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243847480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609536768:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.938541:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.938551:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63034834 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d38be expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.938569:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f737b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604985856:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.938581:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481ff800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604985984:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.938624:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417e40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.938637:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4b40 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.939646:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481ff800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604985984:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.939673:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63034834 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38be expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.939686:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f737b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604985856:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298643.939697:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63034834 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38be expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.939712:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae400, rc 0) 00010000:00010000:2.0:1780298643.939721:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000042649f54/0xaf35132e6303482d lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38b7 expref: 13 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.939741:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000042649f54/0xaf35132e6303482d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38b7 expref: 13 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.939753:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243847480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609536768:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1234us (1354us total) trans 0 rc 301/301 00010000:00010000:2.0:1780298643.939755:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d08400, rc 0) 00000100:00100000:0.0:1780298643.939768:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9634 00000100:00100000:2.0:1780298643.939800:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243846300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609536640:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1317us (1438us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.939809:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9633 00000100:00100000:1.0:1780298643.941114:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.941190:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609536896 00000100:00100000:2.0:1780298643.941197:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609536896 00000100:00100000:2.0:1780298643.941242:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9635 00000100:00100000:2.0:1780298643.941253:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243845c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:29360:x1866777609536896:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.941265:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.941277:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003e31d8c5/0xaf35132e6303483b lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d38c5 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.941309:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(165 177) 00010000:00010000:2.0:1780298643.941313:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(165 177) 00010000:00010000:2.0:1780298643.941324:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003e31d8c5/0xaf35132e6303483b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2641920->3031039] (req 2641920->2650111) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38c5 expref: 13 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.941339:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003e31d8c5/0xaf35132e6303483b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2641920->3031039] (req 2641920->2650111) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38c5 expref: 13 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.941352:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d08a00, rc 0) 00000100:00100000:2.0:1780298643.941394:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243845c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:29360:x1866777609536896:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 144us (280us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.941401:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9635 00000100:00100000:1.0:1780298643.942181:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.942214:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609537024 00000100:00100000:0.0:1780298643.942221:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609537024 00000100:00100000:0.0:1780298643.942261:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6223 00000100:00100000:0.0:1780298643.942267:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243845180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2399:x1866777609537024:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298643.942563:0:16972:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:0.0:1780298643.942566:0:16972:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298643.942568:0:16972:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298643.942569:0:16972:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298643.942571:0:16972:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 472 -> 488 00000100:00100000:0.0:1780298643.942601:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243845180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2399:x1866777609537024:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 334us (422us total) trans 4294969702 rc 0/0 00000100:00100000:0.0:1780298643.942607:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6223 00000100:00100000:0.0:1780298643.943776:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.943812:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609537152 00000100:00100000:3.0:1780298643.943819:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609537152 00000100:00100000:3.0:1780298643.943847:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9636 00000100:00100000:3.0:1780298643.943853:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481ff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2399:x1866777609537152:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.943862:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.943872:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000700ba1da/0xaf35132e63034842 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d38cc expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.943916:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254888700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604986112:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.943973:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.944735:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254888700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604986112:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.944756:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000700ba1da/0xaf35132e63034842 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38cc expref: 18 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.944770:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000700ba1da/0xaf35132e63034842 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38cc expref: 18 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.944778:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156000, rc 0) 00000100:00100000:3.0:1780298643.944803:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481ff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2399:x1866777609537152:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 951us (1029us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.944808:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9636 00000100:00100000:1.0:1780298643.945905:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.945949:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609537280 00000100:00100000:0.0:1780298643.945955:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609537280 00000100:00100000:0.0:1780298643.945990:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9637 00000100:00100000:0.0:1780298643.946000:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243846a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609537280:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.946012:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.946021:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e63034849 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d38d3 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.946056:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604986240:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.946107:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417e40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.946910:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604986240:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.946933:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e63034849 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38d3 expref: 18 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.946949:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e63034849 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38d3 expref: 18 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.946961:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf600, rc 0) 00000100:00100000:1.0:1780298643.946992:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243846a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609537280:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 994us (1097us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.947000:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9637 00000100:00100000:1.0:1780298643.948057:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.948114:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609537408 00000100:00100000:2.0:1780298643.948141:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609537408 00000100:00100000:2.0:1780298643.948207:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9638 00000100:00100000:2.0:1780298643.948218:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243844700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2397:x1866777609537408:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.948236:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298643.948237:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:2.0:1780298643.948258:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fba32a6f/0xaf35132e63034850 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d38da expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.948289:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609537536 00000100:00100000:1.0:1780298643.948296:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609537536 00000100:00100000:2.0:1780298643.948322:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f737b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604986368:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.948331:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9639 00000100:00100000:1.0:1780298643.948337:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609537536:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.948347:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.948360:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002ff2dd4d/0xaf35132e63034857 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d38e1 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.948395:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243845500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604986496:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.948429:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4b40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.948435:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a6c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.949206:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f737b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604986368:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.949239:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000fba32a6f/0xaf35132e63034850 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38da expref: 15 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.949261:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fba32a6f/0xaf35132e63034850 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38da expref: 15 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.949272:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d08600, rc 0) 00000100:00100000:1.0:1780298643.949310:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243845500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604986496:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298643.949334:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243844700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2397:x1866777609537408:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1114us (1274us total) trans 0 rc 301/301 00010000:00010000:1.0:1780298643.949343:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002ff2dd4d/0xaf35132e63034857 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38e1 expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.949351:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9638 00010000:00010000:1.0:1780298643.949388:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002ff2dd4d/0xaf35132e63034857 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38e1 expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.949401:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278800, rc 0) 00000100:00100000:1.0:1780298643.949439:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609537536:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1101us (1204us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.949447:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9639 00000100:00100000:0.0:1780298643.950709:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.950735:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609537664 00000100:00100000:3.0:1780298643.950741:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609537664 00000100:00100000:3.0:1780298643.950768:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9640 00000100:00100000:3.0:1780298643.950774:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:29360:x1866777609537664:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.950783:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.950793:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000042c16ab0/0xaf35132e6303485e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d38e8 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.950808:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(165 177) 00010000:00010000:3.0:1780298643.950811:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(165 177) 00010000:00010000:3.0:1780298643.950817:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000042c16ab0/0xaf35132e6303485e lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1183744->2641919] (req 1183744->1187839) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38e8 expref: 15 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.950828:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000042c16ab0/0xaf35132e6303485e lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1183744->2641919] (req 1183744->1187839) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38e8 expref: 15 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.950839:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156800, rc 0) 00000100:00100000:3.0:1780298643.950862:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:29360:x1866777609537664:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 90us (157us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298643.950866:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9640 00000100:00100000:0.0:1780298643.951853:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.951908:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609537792 00000100:00100000:1.0:1780298643.951913:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609537792 00002000:00100000:1.0:1780298643.951948:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (1183744->1187839) req@ffff9cf3481fd500 x1866777609537792/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.951958:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fd500 with x1866777609537792 ext(1183744->1187839) 00002000:00010000:1.0:1780298643.951962:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481fd500 00000100:00100000:1.0:1780298643.951984:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6224 00000100:00100000:1.0:1780298643.951993:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2399:x1866777609537792:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298643.952220:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2399:x1866777609537792:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 228us (367us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298643.952229:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6224 00002000:00100000:1.0:1780298643.952235:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (1183744->1187839) req@ffff9cf3481fd500 x1866777609537792/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298643.952244:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fd500 with x1866777609537792 ext(1183744->1187839) 00002000:00010000:1.0:1780298643.952248:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481fd500 00000100:00100000:0.0:1780298643.953439:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.953497:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609537920 00000100:00100000:2.0:1780298643.953510:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609537920 00002000:00100000:2.0:1780298643.953553:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (1187840->1191935) req@ffff9cf3481ff480 x1866777609537920/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.953564:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481ff480 with x1866777609537920 ext(1187840->1191935) 00002000:00010000:2.0:1780298643.953574:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481ff480 00000100:00100000:2.0:1780298643.953607:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6225 00000100:00100000:2.0:1780298643.953621:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481ff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2400:x1866777609537920:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298643.953788:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481ff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2400:x1866777609537920:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 172us (351us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298643.953798:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6225 00002000:00100000:2.0:1780298643.953806:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (1187840->1191935) req@ffff9cf3481ff480 x1866777609537920/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:349/0 lens 488/4536 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298643.953815:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481ff480 with x1866777609537920 ext(1187840->1191935) 00002000:00010000:2.0:1780298643.953818:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481ff480 00000100:00100000:0.0:1780298643.954894:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.954946:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609538048 00000100:00100000:1.0:1780298643.954953:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609538048 00000100:00100000:1.0:1780298643.955015:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9641 00000100:00100000:1.0:1780298643.955026:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2399:x1866777609538048:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.955041:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.955057:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000488d1c2a/0xaf35132e63034865 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d38ef expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.955112:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243845500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604986624:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.955235:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.956106:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243845500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604986624:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.956191:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000488d1c2a/0xaf35132e63034865 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38ef expref: 18 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.956211:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000488d1c2a/0xaf35132e63034865 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38ef expref: 18 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.956224:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279400, rc 0) 00000100:00100000:1.0:1780298643.956259:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2399:x1866777609538048:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1235us (1367us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.956267:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9641 00000100:00100000:0.0:1780298643.957300:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.957339:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609538176 00000100:00100000:3.0:1780298643.957344:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609538176 00000100:00100000:3.0:1780298643.957373:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9642 00000100:00100000:3.0:1780298643.957380:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481ffb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2398:x1866777609538176:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.957389:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.957398:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000700ba1da/0xaf35132e6303486c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d38f6 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.957430:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254888700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604986752:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.957480:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.958235:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254888700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604986752:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.958251:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000700ba1da/0xaf35132e6303486c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38f6 expref: 18 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.958261:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000700ba1da/0xaf35132e6303486c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38f6 expref: 18 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.958268:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156000, rc 0) 00000100:00100000:3.0:1780298643.958303:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481ffb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2398:x1866777609538176:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 925us (1006us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.958308:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9642 00000100:00100000:0.0:1780298643.961295:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.961346:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609538432 00000100:00100000:2.0:1780298643.961358:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609538432 00000100:00100000:2.0:1780298643.961408:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62001 00000100:00100000:2.0:1780298643.961418:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+115:29360:x1866777609538432:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:2.0:1780298643.961470:0:6477:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298643.961493:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09e00/0xaf35132e63034793 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 81 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3824 expref: 115 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.961514:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298643.961518:0:6477:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298643.961662:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08400/0xaf35132e63034873 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 80 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.961674:0:6477:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278600/0xaf35132e630347bd lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 80 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3847 expref: 154 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.961691:0:6477:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08400/0xaf35132e63034873 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 80 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.961719:0:6477:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278600/0xaf35132e630347bd lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 80 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3847 expref: 154 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.961730:0:6477:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278600/0xaf35132e630347bd lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 80 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3847 expref: 154 pid: 17578 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.961741:0:6477:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f737b80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6477:1866777604986880:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298643.961810:0:6477:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4c00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.962757:0:6477:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f737b80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6477:1866777604986880:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298643.962782:0:6477:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08400/0xaf35132e63034873 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 80 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.962962:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.962989:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609538560 00000100:00100000:3.0:1780298643.962996:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609538560 00010000:00010000:3.0:1780298643.963030:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278600/0xaf35132e630347bd lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 80 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3847 expref: 155 pid: 17578 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.963058:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15135 00000100:00100000:3.0:1780298643.963068:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+155:2400:x1866777609538560:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298643.963079:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298643.963084:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278600/0xaf35132e630347bd lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 81 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3847 expref: 155 pid: 17578 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.963096:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278600/0xaf35132e630347bd lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 81 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3847 expref: 155 pid: 17578 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.963110:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278600/0xaf35132e630347bd lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 81 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3847 expref: 155 pid: 17578 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.963151:0:8374:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298643.963157:0:8374:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08400/0xaf35132e63034873 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 80 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.963171:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08400/0xaf35132e63034873 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 80 type: IBT flags: 0x40290000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.963179:0:8374:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08400/0xaf35132e63034873 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 80 type: IBT flags: 0x40290000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.963205:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298643.963225:0:6477:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08400/0xaf35132e63034873 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 79 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.963234:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08400/0xaf35132e63034873 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 79 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.963241:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+154:2400:x1866777609538560:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 174us (278us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298643.963258:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15135 00000001:00080000:2.0:1780298643.963317:0:6477:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991062 00000004:00010000:2.0:1780298643.963345:0:6477:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000506af9da reply state 0000000068d6125e transno 4294991063 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08400/0xaf35132e63034873 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 79 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.963359:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08400/0xaf35132e63034873 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 79 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.963366:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298643.963369:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1780298643.963372:0:6477:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1780298643.963436:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+117:29360:x1866777609538432:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 2020us (2157us total) trans 4294991063 rc 0/0 00000100:00100000:2.0:1780298643.963443:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62001 00000020:00080000:2.0:1780298643.963447:0:6477:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1780298643.964238:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.964293:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609538688 00000100:00100000:1.0:1780298643.964298:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609538688 00000100:00100000:1.0:1780298643.964330:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9643 00000100:00100000:1.0:1780298643.964336:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:29360:x1866777609538688:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.964348:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.964359:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e6303487a lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d38fd expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.964376:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000003e31d8c5/0xaf35132e6303483b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2641920->3031039] (req 2641920->2650111) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38c5 expref: 16 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.964383:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63034818 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3031040->36585471] (req 3031040->3039231) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38a2 expref: 16 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.964390:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000042c16ab0/0xaf35132e6303485e lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1183744->2641919] (req 1183744->1187839) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38e8 expref: 16 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.964396:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 3 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(165 177) 00010000:00010000:1.0:1780298643.964400:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(168 177) 00010000:00010000:1.0:1780298643.964405:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e6303487a lrc: 6/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38fd expref: 16 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.964428:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000042c16ab0/0xaf35132e6303485e lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1183744->2641919] (req 1183744->1187839) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38e8 expref: 16 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.964441:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000042c16ab0/0xaf35132e6303485e lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1183744->2641919] (req 1183744->1187839) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38e8 expref: 16 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.964449:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243845500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604987008:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298643.964500:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63034818 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3031040->36585471] (req 3031040->3039231) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38a2 expref: 16 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.964507:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63034818 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3031040->36585471] (req 3031040->3039231) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38a2 expref: 16 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.964514:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243845f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604987136:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298643.964540:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000003e31d8c5/0xaf35132e6303483b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2641920->3031039] (req 2641920->2650111) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38c5 expref: 16 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.964551:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000003e31d8c5/0xaf35132e6303483b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2641920->3031039] (req 2641920->2650111) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38c5 expref: 16 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.964562:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243846d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604987264:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298643.964591:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.965416:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243845500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604987008:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298643.965428:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243845f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604987136:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298643.965620:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.965641:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243846d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604987264:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298643.965668:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609538816 00000100:00100000:2.0:1780298643.965675:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609538816 00010000:00010000:1.0:1780298643.965676:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e6303487a lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38fd expref: 16 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.965693:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278800, rc 0) 00010000:00010000:2.0:1780298643.965697:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63034818 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3031040->36585471] (req 3031040->3039231) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38a2 expref: 17 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.965733:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15136 00000100:00100000:1.0:1780298643.965733:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:29360:x1866777609538688:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1392us (1491us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.965744:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2399:x1866777609538816:12345-192.168.203.25@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1780298643.965745:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9643 00010000:00010000:2.0:1780298643.965759:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298643.965777:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63034818 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [3031040->36585471] (req 3031040->3039231) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38a2 expref: 16 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.965790:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63034818 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [3031040->36585471] (req 3031040->3039231) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38a2 expref: 16 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.965814:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63034818 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [3031040->36585471] (req 3031040->3039231) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38a2 expref: 16 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.965826:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298643.965849:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2399:x1866777609538816:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 110us (232us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.965855:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15136 00000100:00100000:3.0:1780298643.965944:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.965971:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609538944 00000100:00100000:0.0:1780298643.965975:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609538944 00010000:00010000:0.0:1780298643.965994:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000042c16ab0/0xaf35132e6303485e lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1183744->2641919] (req 1183744->1187839) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38e8 expref: 15 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.966009:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.966034:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609539072 00000100:00100000:0.0:1780298643.966039:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609539072 00000100:00100000:2.0:1780298643.966046:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15137 00010000:00010000:0.0:1780298643.966054:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000003e31d8c5/0xaf35132e6303483b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2641920->3031039] (req 2641920->2650111) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38c5 expref: 16 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.966058:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254888700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609538944:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298643.966073:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:0.0:1780298643.966079:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15138 00010000:00010000:2.0:1780298643.966084:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000042c16ab0/0xaf35132e6303485e lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1183744->2641919] (req 1183744->1187839) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38e8 expref: 16 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.966092:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254889180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2398:x1866777609539072:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298643.966102:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000042c16ab0/0xaf35132e6303485e lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1183744->2641919] (req 1183744->1187839) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38e8 expref: 16 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.966103:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298643.966118:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000042c16ab0/0xaf35132e6303485e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [1183744->2641919] (req 1183744->1187839) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38e8 expref: 16 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.966145:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000003e31d8c5/0xaf35132e6303483b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [2641920->3031039] (req 2641920->2650111) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38c5 expref: 16 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.966151:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1780298643.966162:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000003e31d8c5/0xaf35132e6303483b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2641920->3031039] (req 2641920->2650111) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38c5 expref: 15 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.966179:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003e31d8c5/0xaf35132e6303483b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2641920->3031039] (req 2641920->2650111) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38c5 expref: 15 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.966184:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254888700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2400:x1866777609538944:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 129us (239us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.966198:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15137 00010000:00010000:0.0:1780298643.966201:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(168 177) 00010000:00010000:0.0:1780298643.966205:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(168 177) 00010000:00010000:0.0:1780298643.966213:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e6303487a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38fd expref: 13 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.966223:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e6303487a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38fd expref: 13 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298643.966256:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e6303487a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38fd expref: 13 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.966267:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fea00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777604987392:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298643.966319:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417e40 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.966994:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fea00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777604987392:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298643.967005:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298643.967030:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254889180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2398:x1866777609539072:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 944us (1023us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298643.967039:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15138 00000100:00100000:2.0:1780298643.967226:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.967249:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609539200 00000100:00100000:3.0:1780298643.967254:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609539200 00000100:00100000:3.0:1780298643.967301:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9644 00000100:00100000:3.0:1780298643.967312:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f737b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:29360:x1866777609539200:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.967324:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.967335:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004ba675d2/0xaf35132e63034881 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3904 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.967353:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000007fddc766/0xaf35132e63034785 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d381d expref: 17 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.967360:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(120 130) 00010000:00010000:3.0:1780298643.967364:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(121 130) 00010000:00010000:3.0:1780298643.967368:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004ba675d2/0xaf35132e63034881 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3904 expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.967387:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000007fddc766/0xaf35132e63034785 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d381d expref: 17 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.967405:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000007fddc766/0xaf35132e63034785 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d381d expref: 17 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.967417:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254889f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604987520:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298643.967455:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.968103:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254889f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604987520:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298643.968118:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004ba675d2/0xaf35132e63034881 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3904 expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.968146:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156e00, rc 0) 00000100:00100000:3.0:1780298643.968174:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f737b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:29360:x1866777609539200:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 867us (949us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298643.968182:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9644 00000100:00100000:0.0:1780298643.968422:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.968469:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609539328 00000100:00100000:2.0:1780298643.968474:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609539328 00010000:00010000:2.0:1780298643.968491:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000007fddc766/0xaf35132e63034785 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d381d expref: 18 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.968518:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15139 00000100:00100000:2.0:1780298643.968525:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b5e050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2399:x1866777609539328:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298643.968533:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298643.968544:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000007fddc766/0xaf35132e63034785 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d381d expref: 18 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.968555:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000007fddc766/0xaf35132e63034785 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d381d expref: 18 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.968571:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007fddc766/0xaf35132e63034785 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d381d expref: 18 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.968583:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(121 130) 00010000:00010000:2.0:1780298643.968587:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(121 130) 00010000:00010000:2.0:1780298643.968592:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000004ba675d2/0xaf35132e63034881 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3904 expref: 8 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.968602:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004ba675d2/0xaf35132e63034881 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3904 expref: 8 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.968622:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000004ba675d2/0xaf35132e63034881 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3904 expref: 8 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.968631:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f737480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6461:1866777604987648:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298643.968669:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.969485:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f737480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6461:1866777604987648:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298643.969499:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298643.969528:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b5e050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2399:x1866777609539328:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1003us (1106us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298643.969537:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15139 00000100:00100000:0.0:1780298643.969691:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.969738:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.969741:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609539456 00000100:00100000:1.0:1780298643.969748:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609539456 00002000:00010000:1.0:1780298643.969771:0:8364:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (5099861->18446744073709551615) 00000100:00100000:2.0:1780298643.969776:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609539584 00002000:00100000:1.0:1780298643.969776:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fce00 with x1866777609539456 ext(5099861->18446744073709551615) 00000100:00100000:2.0:1780298643.969783:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609539584 00002000:00010000:1.0:1780298643.969787:0:8364:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481fce00. 00002000:00010000:2.0:1780298643.969802:0:19950:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4194304->18446744073709551615) 00002000:00100000:2.0:1780298643.969807:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fca80 with x1866777609539584 ext(4194304->18446744073709551615) 00002000:00010000:2.0:1780298643.969814:0:19950:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf3481fca80. 00000100:00100000:1.0:1780298643.969818:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6226 00000100:00100000:1.0:1780298643.969829:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609539456:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:3.0:1780298643.969835:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6227 00000100:00100000:3.0:1780298643.969847:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609539584:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:3.0:1780298643.969978:0:19951:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 201564 -> 4194304 00002000:00010000:3.0:1780298643.969982:0:19951:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:3.0:1780298643.969985:0:19951:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:3.0:1780298643.969987:0:19951:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:3.0:1780298643.969990:0:19951:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 16 -> 16 00002000:00010000:1.0:1780298643.969999:0:8364:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 5099861 00002000:00010000:1.0:1780298643.970004:0:8364:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298643.970006:0:8364:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298643.970009:0:8364:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298643.970011:0:8364:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 488 -> 488 00000100:00100000:3.0:1780298643.970024:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609539584:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 179us (286us total) trans 4294969249 rc 0/0 00000100:00100000:3.0:1780298643.970036:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6227 00000100:00100000:1.0:1780298643.970041:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609539456:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 213us (348us total) trans 4294969703 rc 0/0 00002000:00010000:3.0:1780298643.970042:0:19951:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4194304->18446744073709551615) 00002000:00100000:3.0:1780298643.970048:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fca80 with x1866777609539584 ext(4194304->18446744073709551615) 00000100:00100000:1.0:1780298643.970054:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6226 00002000:00010000:3.0:1780298643.970056:0:19951:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf3481fca80. 00002000:00010000:1.0:1780298643.970059:0:8364:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (5099861->18446744073709551615) 00002000:00100000:1.0:1780298643.970063:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fce00 with x1866777609539456 ext(5099861->18446744073709551615) 00002000:00010000:1.0:1780298643.970069:0:8364:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481fce00. 00000100:00100000:3.0:1780298643.971016:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.971055:0:7411:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609539712 00000100:00100000:1.0:1780298643.971060:0:7411:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609539712 00000100:00100000:1.0:1780298643.971092:0:7411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62002 00000100:00100000:1.0:1780298643.971099:0:7411:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254889f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+154:29360:x1866777609539712:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.971109:0:7411:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.971143:0:7411:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279400/0xaf35132e63034888 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 80 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d390b expref: -99 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.971228:0:7411:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279a00/0xaf35132e6303488f lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 81 type: IBT flags: 0x40000000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.971240:0:7411:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279a00/0xaf35132e6303488f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 81 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.971250:0:7411:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279a00/0xaf35132e6303488f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 81 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1780298643.971271:0:7411:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 9294165 00000004:00200000:1.0:1780298643.971275:0:7411:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1780298643.971302:0:7411:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279a00/0xaf35132e6303488f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 81 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.971325:0:7411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279400/0xaf35132e63034888 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 81 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d390b expref: 156 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.971337:0:7411:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279a00/0xaf35132e6303488f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 80 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d390b expref: 155 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.971344:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298643.971347:0:7411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1780298643.971351:0:7411:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279a00, rc 0) 00000100:00100000:1.0:1780298643.971381:0:7411:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254889f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+155:29360:x1866777609539712:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 283us (366us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.971388:0:7411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62002 00000100:00100000:2.0:1780298643.972437:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.972469:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609539840 00000100:00100000:1.0:1780298643.972473:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609539840 00000100:00100000:2.0:1780298643.972492:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.972513:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609539968 00000100:00100000:1.0:1780298643.972515:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609539968 00000100:00100000:3.0:1780298643.972533:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9645 00000100:00100000:1.0:1780298643.972575:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9646 00000100:00100000:3.0:1780298643.972581:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f736300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609539840:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298643.972585:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f734e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2399:x1866777609539968:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.972596:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.972596:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.972604:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f4922155/0xaf35132e63034896 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3919 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.972607:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e6303489d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3912 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.972650:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243846d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604987776:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.972652:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254889500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604987904:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.972704:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.972708:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.973466:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243846d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604987776:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.973483:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254889500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604987904:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.973486:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f4922155/0xaf35132e63034896 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3919 expref: 18 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.973503:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f4922155/0xaf35132e63034896 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3919 expref: 18 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.973510:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278600, rc 0) 00010000:00010000:3.0:1780298643.973511:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e6303489d lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3912 expref: 18 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.973526:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e6303489d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3912 expref: 18 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.973536:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f734e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2399:x1866777609539968:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 954us (1043us total) trans 0 rc 301/301 00010000:00010000:3.0:1780298643.973540:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156a00, rc 0) 00000100:00100000:1.0:1780298643.973541:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9646 00000100:00100000:3.0:1780298643.973568:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f736300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609539840:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 992us (1132us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.973576:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9645 00000100:00100000:0.0:1780298643.974660:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.974688:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609540096 00000100:00100000:1.0:1780298643.974692:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609540096 00000100:00100000:1.0:1780298643.974719:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9647 00000100:00100000:1.0:1780298643.974723:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2397:x1866777609540096:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.974729:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.974736:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e630348a4 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3920 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.974756:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243846d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604988032:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.974791:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.974807:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.974872:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609540224 00000100:00100000:3.0:1780298643.974877:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609540224 00000100:00100000:3.0:1780298643.974938:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9648 00000100:00100000:3.0:1780298643.974945:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2398:x1866777609540224:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.974954:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.974963:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000700ba1da/0xaf35132e630348ab lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3927 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.974990:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254889500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604988160:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.975024:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.975433:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243846d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604988032:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.975447:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e630348a4 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3920 expref: 18 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.975457:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e630348a4 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3920 expref: 18 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.975463:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279800, rc 0) 00000100:00100000:1.0:1780298643.975485:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2397:x1866777609540096:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 762us (828us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298643.975491:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9647 00000100:00100000:3.0:1780298643.975641:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254889500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604988160:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.975656:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000700ba1da/0xaf35132e630348ab lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3927 expref: 18 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.975670:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000700ba1da/0xaf35132e630348ab lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3927 expref: 18 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.975679:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156000, rc 0) 00000100:00100000:3.0:1780298643.975708:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2398:x1866777609540224:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 763us (903us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.975713:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9648 00000100:00100000:0.0:1780298643.976768:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.976794:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609540352 00000100:00100000:3.0:1780298643.976800:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609540352 00000100:00100000:3.0:1780298643.976827:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9649 00000100:00100000:3.0:1780298643.976834:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343964000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:29360:x1866777609540352:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.976844:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.976852:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000042c16ab0/0xaf35132e630348b2 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d392e expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.976869:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e6303487a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38fd expref: 13 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.976877:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(168 177) 00010000:00010000:3.0:1780298643.976881:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(169 177) 00010000:00010000:3.0:1780298643.976885:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000042c16ab0/0xaf35132e630348b2 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [4673536->4681727] (req 4673536->4681727) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d392e expref: 18 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.976904:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e6303487a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38fd expref: 13 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.976917:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e6303487a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38fd expref: 13 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.976927:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254889500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604988288:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298643.976962:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.977649:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254889500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604988288:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298643.977664:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000042c16ab0/0xaf35132e630348b2 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [4673536->4681727] (req 4673536->4681727) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d392e expref: 18 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.977674:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156800, rc 0) 00000100:00100000:3.0:1780298643.977701:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343964000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:29360:x1866777609540352:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 868us (935us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298643.977708:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9649 00000100:00100000:0.0:1780298643.977914:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.977942:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609540480 00000100:00100000:1.0:1780298643.977945:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609540480 00010000:00010000:1.0:1780298643.977962:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e6303487a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38fd expref: 14 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.977990:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15140 00000100:00100000:1.0:1780298643.977998:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56dc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609540480:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298643.978008:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298643.978026:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e6303487a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38fd expref: 14 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.978036:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e6303487a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38fd expref: 14 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.978049:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e6303487a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d38fd expref: 14 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.978056:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(169 177) 00010000:00010000:1.0:1780298643.978058:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(169 177) 00010000:00010000:1.0:1780298643.978065:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000042c16ab0/0xaf35132e630348b2 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4673536->4681727) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d392e expref: 17 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.978074:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000042c16ab0/0xaf35132e630348b2 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4673536->4681727) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d392e expref: 17 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.978098:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000042c16ab0/0xaf35132e630348b2 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4673536->4681727) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d392e expref: 17 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.978108:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243846d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777604988416:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298643.978174:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.978974:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243846d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777604988416:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298643.978985:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298643.979011:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56dc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609540480:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1015us (1097us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.979023:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15140 00000100:00100000:2.0:1780298643.979331:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.979364:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609540608 00000100:00100000:0.0:1780298643.979369:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609540608 00002000:00100000:0.0:1780298643.979396:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (4673536->4681727) req@ffff9cf37f737480 x1866777609540608/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/0 e 0 to 0 dl 1780298654 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.979406:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f737480 with x1866777609540608 ext(4673536->4681727) 00002000:00010000:0.0:1780298643.979410:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f737480 00000100:00100000:0.0:1780298643.979428:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6228 00000100:00100000:0.0:1780298643.979435:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f737480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609540608:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298643.979603:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f737480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609540608:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 168us (273us total) trans 0 rc 8192/8192 00000100:00100000:0.0:1780298643.979611:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6228 00002000:00100000:0.0:1780298643.979617:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (4673536->4681727) req@ffff9cf37f737480 x1866777609540608/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:349/0 lens 488/8632 e 0 to 0 dl 1780298654 ref 1 fl Complete:/600/0 rc 8192/8192 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298643.979626:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f737480 with x1866777609540608 ext(4673536->4681727) 00002000:00010000:0.0:1780298643.979630:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f737480 00000100:00100000:0.0:1780298643.980722:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.980753:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609540736 00000100:00100000:3.0:1780298643.980758:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609540736 00000100:00100000:3.0:1780298643.980828:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9650 00000100:00100000:3.0:1780298643.980836:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609540736:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.980845:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.980856:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bed20494/0xaf35132e630348b9 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3935 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.980894:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254889500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604988544:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298643.980941:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.981731:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254889500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604988544:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.981750:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bed20494/0xaf35132e630348b9 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3935 expref: 18 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.981766:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bed20494/0xaf35132e630348b9 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3935 expref: 18 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.981777:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157000, rc 0) 00000100:00100000:2.0:1780298643.981809:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609540736:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 974us (1087us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.981817:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9650 00000100:00100000:2.0:1780298643.982755:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.982792:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609540864 00000100:00100000:0.0:1780298643.982797:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609540864 00000100:00100000:0.0:1780298643.982824:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9651 00000100:00100000:0.0:1780298643.982832:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f735f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609540864:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.982840:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.982849:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e630348c0 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d393c expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.982876:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e56fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604988672:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.982914:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.983769:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e56fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604988672:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298643.983787:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e630348c0 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d393c expref: 18 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.983799:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e630348c0 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d393c expref: 18 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.983806:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf600, rc 0) 00000100:00100000:2.0:1780298643.983830:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f735f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609540864:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1000us (1077us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298643.983835:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9651 00000100:00100000:3.0:1780298643.984886:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.984932:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609540992 00000100:00100000:0.0:1780298643.984937:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609540992 00000100:00100000:0.0:1780298643.984972:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9652 00000100:00100000:0.0:1780298643.984981:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254888a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609540992:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.984990:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.984999:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e630348c7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3943 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.985039:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e56f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604988800:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.985084:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.985931:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e56f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604988800:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.985948:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e630348c7 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3943 expref: 18 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.985963:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e630348c7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3943 expref: 18 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.985973:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baea00, rc 0) 00000100:00100000:3.0:1780298643.986001:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254888a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609540992:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1022us (1117us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.986007:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9652 00000100:00100000:3.0:1780298643.986833:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.986876:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609541120 00000100:00100000:0.0:1780298643.986881:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609541120 00000100:00100000:0.0:1780298643.986910:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9653 00000100:00100000:0.0:1780298643.986918:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254889c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609541120:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.986924:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.986932:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e630348ce lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d394a expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.986959:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e56d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604988928:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298643.986995:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.987760:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e56d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604988928:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298643.987777:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e630348ce lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d394a expref: 18 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.987790:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e630348ce lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d394a expref: 18 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.987799:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf400, rc 0) 00000100:00100000:3.0:1780298643.987827:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254889c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609541120:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 911us (995us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298643.987834:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9653 00000100:00100000:2.0:1780298643.988550:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.988590:0:6476:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609541248 00000100:00100000:1.0:1780298643.988595:0:6476:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609541248 00000100:00100000:1.0:1780298643.988631:0:6476:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62003 00000100:00100000:1.0:1780298643.988640:0:6476:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f734700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+117:29360:x1866777609541248:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.988654:0:6476:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298643.988667:0:6476:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279800/0xaf35132e630348d5 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 81 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3951 expref: -99 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.988744:0:6476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278600/0xaf35132e630348dc lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 82 type: IBT flags: 0x40000000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.988760:0:6476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278600/0xaf35132e630348dc lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 82 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.988771:0:6476:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278600/0xaf35132e630348dc lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 82 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1780298643.988792:0:6476:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 9294165 00000004:00200000:1.0:1780298643.988795:0:6476:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1780298643.988808:0:6476:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278600/0xaf35132e630348dc lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 82 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.988825:0:6476:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279800/0xaf35132e630348d5 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 82 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3951 expref: 119 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.988834:0:6476:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278600/0xaf35132e630348dc lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 81 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3951 expref: 118 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.988842:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298643.988844:0:6476:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1780298643.988846:0:6476:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278600, rc 0) 00000100:00100000:1.0:1780298643.988869:0:6476:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f734700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+118:29360:x1866777609541248:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 233us (321us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298643.988874:0:6476:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62003 00000100:00100000:0.0:1780298643.989605:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.989634:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609541376 00000100:00100000:2.0:1780298643.989638:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609541376 00000100:00100000:2.0:1780298643.989662:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9654 00000100:00100000:2.0:1780298643.989667:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609541376:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.989674:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.989682:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007ba4a4a6/0xaf35132e630348e3 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3958 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.989695:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(169 177) 00010000:00010000:2.0:1780298643.989698:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(169 177) 00010000:00010000:2.0:1780298643.989703:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007ba4a4a6/0xaf35132e630348e3 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3958 expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.989711:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007ba4a4a6/0xaf35132e630348e3 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3958 expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.989721:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d09c00, rc 0) 00000100:00100000:2.0:1780298643.989742:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609541376:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 75us (138us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.989746:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9654 00000100:00100000:0.0:1780298643.990600:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.990636:0:17578:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609541632 00000100:00100000:1.0:1780298643.990642:0:17578:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609541632 00000100:00100000:1.0:1780298643.990674:0:17578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62004 00000100:00100000:1.0:1780298643.990682:0:17578:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+155:29360:x1866777609541632:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:1.0:1780298643.990709:0:17578:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298643.990723:0:17578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279a00/0xaf35132e6303488f lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 82 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d390b expref: 155 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.990735:0:17578:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298643.990739:0:17578:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298643.990872:0:17578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e630348ea lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 81 type: IBT flags: 0x40000000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.990883:0:17578:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278600/0xaf35132e630348dc lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 81 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3951 expref: 117 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.990902:0:17578:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e630348ea lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 81 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.990926:0:17578:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278600/0xaf35132e630348dc lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 81 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3951 expref: 117 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.990941:0:17578:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278600/0xaf35132e630348dc lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 81 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3951 expref: 117 pid: 6476 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.990951:0:17578:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243846d80 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000_UUID:17578:1866777604989056:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298643.990996:0:17578:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837af00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.991679:0:17578:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243846d80 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000_UUID:17578:1866777604989056:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298643.991696:0:17578:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e630348ea lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 81 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.991868:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.991897:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609541760 00000100:00100000:2.0:1780298643.991901:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609541760 00010000:00010000:2.0:1780298643.991915:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278600/0xaf35132e630348dc lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 81 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3951 expref: 118 pid: 6476 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.991935:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15141 00000100:00100000:2.0:1780298643.991939:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+118:2400:x1866777609541760:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298643.991947:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298643.991951:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278600/0xaf35132e630348dc lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 82 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3951 expref: 118 pid: 6476 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.991959:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278600/0xaf35132e630348dc lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 82 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3951 expref: 118 pid: 6476 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.991968:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278600/0xaf35132e630348dc lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 82 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3951 expref: 118 pid: 6476 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.991975:0:20253:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298643.991978:0:20253:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e630348ea lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 81 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.991989:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e630348ea lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 81 type: IBT flags: 0x40290000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.991993:0:20253:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e630348ea lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 81 type: IBT flags: 0x40290000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.992008:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298643.992025:0:17578:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e630348ea lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 80 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.992032:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+117:2400:x1866777609541760:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 91us (162us total) trans 0 rc 0/0 00010000:00010000:1.0:1780298643.992039:0:17578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e630348ea lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 80 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.992042:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15141 00000001:00080000:1.0:1780298643.992094:0:17578:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991064 00000004:00010000:1.0:1780298643.992114:0:17578:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 000000002c870c1f reply state 00000000d959c7d9 transno 4294991065 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e630348ea lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 80 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.992147:0:17578:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e630348ea lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 80 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298643.992154:0:17578:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298643.992158:0:17578:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1780298643.992160:0:17578:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1780298643.992212:0:17578:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+157:29360:x1866777609541632:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1530us (1610us total) trans 4294991065 rc 0/0 00000100:00100000:1.0:1780298643.992220:0:17578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62004 00000020:00080000:1.0:1780298643.992224:0:17578:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1780298643.992948:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.992977:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609541888 00000100:00100000:2.0:1780298643.992981:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609541888 00000100:00100000:2.0:1780298643.993005:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9655 00000100:00100000:2.0:1780298643.993010:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:29360:x1866777609541888:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.993016:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.993022:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fba32a6f/0xaf35132e630348f1 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d395f expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.993037:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000042c16ab0/0xaf35132e630348b2 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4673536->4681727) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d392e expref: 19 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.993043:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000007ba4a4a6/0xaf35132e630348e3 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3958 expref: 13 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.993048:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(169 177) 00010000:00010000:2.0:1780298643.993050:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(169 177) 00010000:00010000:2.0:1780298643.993053:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fba32a6f/0xaf35132e630348f1 lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d395f expref: 19 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.993067:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000007ba4a4a6/0xaf35132e630348e3 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3958 expref: 13 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.993078:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000007ba4a4a6/0xaf35132e630348e3 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3958 expref: 13 pid: 19992 timeout: 933 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.993089:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f737800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604989184:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298643.993169:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000042c16ab0/0xaf35132e630348b2 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4673536->4681727) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d392e expref: 19 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.993179:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000042c16ab0/0xaf35132e630348b2 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4673536->4681727) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d392e expref: 19 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.993190:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f747100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604989312:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298643.993221:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4600 going to sleep for 11 seconds 00000100:00100000:2.0:1780298643.993756:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f737800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604989184:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298643.993891:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f747100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604989312:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298643.993911:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000fba32a6f/0xaf35132e630348f1 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d395f expref: 19 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.993925:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d08600, rc 0) 00000100:00100000:2.0:1780298643.993957:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:29360:x1866777609541888:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 946us (1008us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.993965:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9655 00000100:00100000:0.0:1780298643.994167:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298643.994197:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609542016 00000100:00100000:1.0:1780298643.994201:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609542016 00000100:00100000:0.0:1780298643.994210:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:1.0:1780298643.994223:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000007ba4a4a6/0xaf35132e630348e3 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3958 expref: 14 pid: 19992 timeout: 933 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.994254:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609542144 00000100:00100000:1.0:1780298643.994257:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609542144 00010000:00010000:1.0:1780298643.994274:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000042c16ab0/0xaf35132e630348b2 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4673536->4681727) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d392e expref: 19 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.994308:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15142 00000100:00100000:3.0:1780298643.994321:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609542016:12345-192.168.203.25@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1780298643.994326:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15143 00010000:00010000:3.0:1780298643.994335:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1780298643.994340:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2400:x1866777609542144:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298643.994350:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298643.994355:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000007ba4a4a6/0xaf35132e630348e3 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3958 expref: 14 pid: 19992 timeout: 933 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.994362:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000042c16ab0/0xaf35132e630348b2 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 4673536->4681727) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d392e expref: 19 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.994373:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000007ba4a4a6/0xaf35132e630348e3 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3958 expref: 14 pid: 19992 timeout: 933 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.994386:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000042c16ab0/0xaf35132e630348b2 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 4673536->4681727) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d392e expref: 19 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.994409:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007ba4a4a6/0xaf35132e630348e3 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3958 expref: 14 pid: 19992 timeout: 933 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.994415:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000042c16ab0/0xaf35132e630348b2 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 4673536->4681727) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d392e expref: 19 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.994427:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(169 177) 00010000:00010000:3.0:1780298643.994433:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(169 177) 00010000:00010000:3.0:1780298643.994439:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000fba32a6f/0xaf35132e630348f1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d395f expref: 18 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.994447:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fba32a6f/0xaf35132e630348f1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d395f expref: 18 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.994456:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298643.994484:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609542144:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 149us (273us total) trans 0 rc 0/0 00010000:00010000:3.0:1780298643.994486:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000fba32a6f/0xaf35132e630348f1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d395f expref: 18 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.994497:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15143 00000100:00100000:3.0:1780298643.994506:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254889500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777604989440:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298643.994557:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.995146:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254889500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777604989440:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298643.995207:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298643.995238:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609542016:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 921us (1071us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298643.995246:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15142 00000100:00100000:0.0:1780298643.995265:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.995298:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609542272 00000100:00100000:3.0:1780298643.995303:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609542272 00000100:00100000:3.0:1780298643.995387:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9656 00000100:00100000:3.0:1780298643.995395:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:29360:x1866777609542272:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298643.995404:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298643.995413:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000700ba1da/0xaf35132e630348f8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3966 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.995430:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000004ba675d2/0xaf35132e63034881 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3904 expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.995438:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(121 130) 00010000:00010000:3.0:1780298643.995441:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(122 130) 00010000:00010000:3.0:1780298643.995445:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000700ba1da/0xaf35132e630348f8 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3966 expref: 18 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.995461:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000004ba675d2/0xaf35132e63034881 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3904 expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.995472:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000004ba675d2/0xaf35132e63034881 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3904 expref: 9 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298643.995481:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254889500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604989568:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298643.995514:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298643.996089:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254889500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604989568:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298643.996101:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000700ba1da/0xaf35132e630348f8 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3966 expref: 18 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298643.996111:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156000, rc 0) 00000100:00100000:3.0:1780298643.996154:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:29360:x1866777609542272:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 759us (892us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298643.996161:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9656 00000100:00100000:0.0:1780298643.996321:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.996349:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609542400 00000100:00100000:2.0:1780298643.996353:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609542400 00010000:00010000:2.0:1780298643.996367:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000004ba675d2/0xaf35132e63034881 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3904 expref: 10 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.996388:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15144 00000100:00100000:2.0:1780298643.996394:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609542400:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298643.996403:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298643.996413:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000004ba675d2/0xaf35132e63034881 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3904 expref: 10 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.996420:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000004ba675d2/0xaf35132e63034881 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3904 expref: 10 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.996430:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004ba675d2/0xaf35132e63034881 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3904 expref: 10 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.996437:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(122 130) 00010000:00010000:2.0:1780298643.996440:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(122 130) 00010000:00010000:2.0:1780298643.996443:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000700ba1da/0xaf35132e630348f8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3966 expref: 17 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.996448:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000700ba1da/0xaf35132e630348f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3966 expref: 17 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.996461:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000700ba1da/0xaf35132e630348f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3966 expref: 17 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298643.996467:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f747100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777604989696:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298643.996497:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.997149:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f747100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777604989696:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298643.997163:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298643.997197:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609542400:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 803us (876us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298643.997203:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15144 00000100:00100000:1.0:1780298643.997251:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.997292:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609542528 00000100:00100000:2.0:1780298643.997296:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609542528 00000100:00100000:1.0:1780298643.997301:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00002000:00010000:2.0:1780298643.997316:0:8363:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:2.0:1780298643.997319:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243846d80 with x1866777609542528 ext(4194304->18446744073709551615) 00002000:00010000:2.0:1780298643.997325:0:8363:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf243846d80. 00000100:00100000:0.0:1780298643.997329:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609542656 00000100:00100000:0.0:1780298643.997333:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609542656 00000100:00100000:2.0:1780298643.997348:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6229 00002000:00010000:0.0:1780298643.997349:0:19943:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4155114->18446744073709551615) 00002000:00100000:0.0:1780298643.997353:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243845f80 with x1866777609542656 ext(4155114->18446744073709551615) 00000100:00100000:2.0:1780298643.997358:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243846d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609542528:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:0.0:1780298643.997358:0:19943:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf243845f80. 00000100:00100000:0.0:1780298643.997383:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6230 00000100:00100000:0.0:1780298643.997391:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243845f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609542656:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:2.0:1780298643.997501:0:8363:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 5099861 -> 4194304 00002000:00010000:2.0:1780298643.997505:0:8363:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298643.997508:0:8363:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298643.997510:0:19943:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4194304 -> 4155114 00002000:00010000:2.0:1780298643.997511:0:8363:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298643.997514:0:19943:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298643.997517:0:8363:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 488 -> 488 00002000:00010000:0.0:1780298643.997517:0:19943:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298643.997521:0:19943:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298643.997524:0:19943:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 16 -> 16 00000100:00100000:0.0:1780298643.997560:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243845f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2400:x1866777609542656:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 166us (253us total) trans 4294969250 rc 0/0 00000100:00100000:2.0:1780298643.997561:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243846d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2400:x1866777609542528:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 199us (304us total) trans 4294969704 rc 0/0 00000100:00100000:0.0:1780298643.997572:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6230 00002000:00010000:0.0:1780298643.997575:0:19943:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4155114->18446744073709551615) 00002000:00100000:0.0:1780298643.997578:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243845f80 with x1866777609542656 ext(4155114->18446744073709551615) 00000100:00100000:2.0:1780298643.997579:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6229 00002000:00010000:0.0:1780298643.997582:0:19943:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf243845f80. 00002000:00010000:2.0:1780298643.997584:0:8363:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:2.0:1780298643.997586:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243846d80 with x1866777609542528 ext(4194304->18446744073709551615) 00002000:00010000:2.0:1780298643.997591:0:8363:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf243846d80. 00000100:00100000:1.0:1780298643.998229:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298643.998259:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609542784 00000100:00100000:2.0:1780298643.998263:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609542784 00000100:00100000:2.0:1780298643.998302:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62005 00000100:00100000:2.0:1780298643.998309:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243845500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+117:29360:x1866777609542784:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298643.998317:0:6477:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298643.998326:0:6477:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09e00/0xaf35132e630348ff lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 81 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d396d expref: -99 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.998387:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09600/0xaf35132e63034906 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 82 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.998402:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09600/0xaf35132e63034906 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 82 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.998409:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09600/0xaf35132e63034906 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 82 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298643.998427:0:6477:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 8349418 00000004:00200000:2.0:1780298643.998430:0:6477:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298643.998441:0:6477:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09600/0xaf35132e63034906 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 82 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.998456:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09e00/0xaf35132e630348ff lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 82 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d396d expref: 119 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.998465:0:6477:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09600/0xaf35132e63034906 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 81 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d396d expref: 118 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298643.998471:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298643.998474:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298643.998477:0:6477:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d09600, rc 0) 00000100:00100000:2.0:1780298643.998504:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243845500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+118:29360:x1866777609542784:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 196us (275us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298643.998511:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62005 00000100:00100000:3.0:1780298643.999232:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298643.999260:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298643.999276:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609542912 00000100:00100000:0.0:1780298643.999291:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609542912 00000100:00100000:1.0:1780298643.999306:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609543040 00000100:00100000:1.0:1780298643.999311:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609543040 00000100:00100000:0.0:1780298643.999321:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9657 00000100:00100000:0.0:1780298643.999329:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254889500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609542912:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298643.999335:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298643.999343:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e6303490d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3974 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298643.999352:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9658 00000100:00100000:1.0:1780298643.999363:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254888700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609543040:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298643.999373:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298643.999375:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604989824:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298643.999383:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017dab243/0xaf35132e63034914 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d397b expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298643.999414:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298643.999419:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548ba300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604989952:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298643.999452:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298643.999987:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604989824:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.000004:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e6303490d lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3974 expref: 14 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.000015:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e6303490d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3974 expref: 14 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.000022:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae400, rc 0) 00000100:00100000:0.0:1780298644.000043:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254889500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609542912:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 718us (814us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.000048:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9657 00000100:00100000:1.0:1780298644.000197:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548ba300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604989952:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.000211:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000017dab243/0xaf35132e63034914 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d397b expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.000225:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017dab243/0xaf35132e63034914 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d397b expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.000231:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279800, rc 0) 00000100:00100000:1.0:1780298644.000251:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254888700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609543040:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 893us (992us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.000256:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9658 00000100:00100000:3.0:1780298644.001074:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.001102:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.001104:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609543168 00000100:00100000:2.0:1780298644.001109:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609543168 00000100:00100000:2.0:1780298644.001146:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609543296 00000100:00100000:2.0:1780298644.001148:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609543296 00000100:00100000:3.0:1780298644.001163:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9659 00000100:00100000:3.0:1780298644.001177:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254889180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2397:x1866777609543168:12345-192.168.203.25@tcp:101:fsx.0 00000100:00080000:1.0:1780298644.001179:0:6473:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1780298644.001184:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9660 00010000:00010000:3.0:1780298644.001188:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298644.001194:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609543296:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.001199:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e6303491b lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3982 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.001201:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.001210:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006dfdf24b/0xaf35132e63034922 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3989 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.001233:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433dd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604990080:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.001238:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f744700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604990208:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.001295:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856600 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.001295:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4b40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.001995:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f744700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604990208:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.002012:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000006dfdf24b/0xaf35132e63034922 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3989 expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.002018:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433dd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604990080:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.002025:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006dfdf24b/0xaf35132e63034922 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3989 expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.002033:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d08200, rc 0) 00010000:00010000:0.0:1780298644.002037:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e6303491b lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3982 expref: 14 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.002048:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e6303491b lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3982 expref: 14 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.002056:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156a00, rc 0) 00000100:00100000:2.0:1780298644.002060:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609543296:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 868us (957us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.002065:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9660 00000100:00100000:0.0:1780298644.002080:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254889180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2397:x1866777609543168:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 908us (1006us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.002086:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9659 00000100:00100000:1.0:1780298644.002901:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.002937:0:7411:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609543424 00000100:00100000:2.0:1780298644.002942:0:7411:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609543424 00000100:00080000:1.0:1780298644.002970:0:6473:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1780298644.002981:0:7411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62006 00000100:00100000:2.0:1780298644.002989:0:7411:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548ba300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+157:29360:x1866777609543424:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.003001:0:7411:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.003012:0:7411:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08a00/0xaf35132e63034929 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 82 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3990 expref: -99 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.003076:0:7411:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bec00/0xaf35132e63034930 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 83 type: IBT flags: 0x40000000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.003085:0:7411:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bec00/0xaf35132e63034930 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 83 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.003091:0:7411:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bec00/0xaf35132e63034930 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 83 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298644.003105:0:7411:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 8349418 00000004:00200000:2.0:1780298644.003107:0:7411:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298644.003116:0:7411:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bec00/0xaf35132e63034930 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 83 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.003152:0:7411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08a00/0xaf35132e63034929 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 83 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3990 expref: 159 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.003162:0:7411:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bec00/0xaf35132e63034930 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 82 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3990 expref: 158 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.003168:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298644.003170:0:7411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298644.003172:0:7411:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf2445bec00, rc 0) 00000100:00100000:2.0:1780298644.003192:0:7411:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548ba300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+158:29360:x1866777609543424:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 205us (291us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.003196:0:7411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62006 00000100:00100000:3.0:1780298644.004258:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.004297:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609543552 00000100:00100000:0.0:1780298644.004301:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609543552 00000100:00080000:1.0:1780298644.004329:0:6473:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1780298644.004341:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6231 00000100:00100000:0.0:1780298644.004347:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2400:x1866777609543552:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298644.004516:0:19949:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:0.0:1780298644.004520:0:19949:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298644.004522:0:19949:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298644.004523:0:19949:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298644.004525:0:19949:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 488 -> 512 00000100:00100000:0.0:1780298644.004553:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2400:x1866777609543552:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 206us (296us total) trans 4294969705 rc 0/0 00000100:00100000:0.0:1780298644.004561:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6231 00000100:00100000:3.0:1780298644.005806:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.005843:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609543680 00000100:00100000:2.0:1780298644.005848:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609543680 00000100:00100000:2.0:1780298644.005877:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9661 00000100:00100000:2.0:1780298644.005885:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:29360:x1866777609543680:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.005894:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.005904:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63034937 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3997 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.005924:0:19992:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000700ba1da/0xaf35132e630348f8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3966 expref: 18 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.005932:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298643 v=(122 130) 00010000:00010000:2.0:1780298644.005937:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(1 122) 00010000:00010000:2.0:1780298644.005941:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63034937 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2641920->2654207] (req 2641920->2654207) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3997 expref: 10 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.005963:0:19992:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000700ba1da/0xaf35132e630348f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3966 expref: 18 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.005978:0:19992:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000700ba1da/0xaf35132e630348f8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3966 expref: 18 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.005990:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f744700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604990336:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298644.006032:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4b40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.006672:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f744700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604990336:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298644.006688:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63034937 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2641920->2654207] (req 2641920->2654207) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3997 expref: 10 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.006701:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf2445be600, rc 0) 00000100:00100000:2.0:1780298644.006726:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:29360:x1866777609543680:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 843us (922us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.006730:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9661 00000100:00100000:3.0:1780298644.006845:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.006909:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609543808 00000100:00100000:1.0:1780298644.006912:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609543808 00010000:00010000:1.0:1780298644.006938:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000700ba1da/0xaf35132e630348f8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3966 expref: 19 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1780298644.006939:0:6473:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1780298644.006960:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15145 00000100:00100000:1.0:1780298644.006967:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433ded80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2400:x1866777609543808:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298644.006977:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298644.006988:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000700ba1da/0xaf35132e630348f8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3966 expref: 19 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.006998:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000700ba1da/0xaf35132e630348f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3966 expref: 19 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.007015:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000700ba1da/0xaf35132e630348f8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3966 expref: 19 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.007022:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(1 122) 00010000:00010000:1.0:1780298644.007025:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(1 122) 00010000:00010000:1.0:1780298644.007030:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63034937 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2641920->36196351] (req 2641920->2654207) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3997 expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.007036:0:6463:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63034937 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2641920->36196351] (req 2641920->2654207) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3997 expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.007052:0:6463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63034937 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2641920->36196351] (req 2641920->2654207) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3997 expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.007059:0:6463:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548bb480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777604990464:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298644.007087:0:6463:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.007694:0:6463:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548bb480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777604990464:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298644.007703:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298644.007723:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433ded80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609543808:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 758us (880us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.007729:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15145 00000100:00100000:2.0:1780298644.007967:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.007992:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609543936 00000100:00100000:1.0:1780298644.007995:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609543936 00002000:00100000:1.0:1780298644.008012:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (2641920->2646015) req@ffff9cf37f744700 x1866777609543936/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.008019:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f744700 with x1866777609543936 ext(2641920->2646015) 00002000:00010000:1.0:1780298644.008022:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f744700 00000100:00100000:1.0:1780298644.008036:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6232 00000100:00100000:1.0:1780298644.008040:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f744700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609543936:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298644.008209:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f744700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609543936:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 168us (242us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298644.008215:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6232 00002000:00100000:1.0:1780298644.008220:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (2641920->2646015) req@ffff9cf37f744700 x1866777609543936/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/4536 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.008225:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f744700 with x1866777609543936 ext(2641920->2646015) 00002000:00010000:1.0:1780298644.008228:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f744700 00000100:00100000:3.0:1780298644.009008:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.009042:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609544064 00000100:00100000:0.0:1780298644.009046:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609544064 00002000:00100000:0.0:1780298644.009062:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x12:0x0] (2650112->2654207) req@ffff9cf2433df480 x1866777609544064/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298644.009069:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2433df480 with x1866777609544064 ext(2650112->2654207) 00002000:00010000:0.0:1780298644.009071:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2433df480 00000100:00100000:0.0:1780298644.009087:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6233 00000100:00100000:0.0:1780298644.009094:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609544064:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298644.009190:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609544064:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 96us (183us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298644.009195:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6233 00002000:00100000:0.0:1780298644.009199:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x12:0x0] (2650112->2654207) req@ffff9cf2433df480 x1866777609544064/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/4536 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298644.009204:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2433df480 with x1866777609544064 ext(2650112->2654207) 00002000:00010000:0.0:1780298644.009206:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2433df480 00000100:00100000:2.0:1780298644.010010:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.010042:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609544192 00000100:00100000:1.0:1780298644.010047:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609544192 00000100:00100000:1.0:1780298644.010082:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9662 00000100:00100000:1.0:1780298644.010091:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2398:x1866777609544192:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.010104:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.010113:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e6303493e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d399e expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.010165:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.010166:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548bb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604990592:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.010188:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609544320 00000100:00100000:2.0:1780298644.010193:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609544320 00000100:00100000:1.0:1780298644.010218:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.010223:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9663 00000100:00100000:2.0:1780298644.010231:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609544320:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.010238:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.010247:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fbbd4b6c/0xaf35132e63034945 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d39a5 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.010270:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f746300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604990720:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.010307:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4b40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.010751:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548bb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604990592:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.010765:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e6303493e lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d399e expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.010777:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e6303493e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d399e expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.010784:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279800, rc 0) 00000100:00100000:1.0:1780298644.010811:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2398:x1866777609544192:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 721us (801us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.010818:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9662 00000100:00100000:2.0:1780298644.010994:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f746300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604990720:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.011006:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000fbbd4b6c/0xaf35132e63034945 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d39a5 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.011016:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fbbd4b6c/0xaf35132e63034945 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d39a5 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.011026:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf36d473e00, rc 0) 00000100:00100000:2.0:1780298644.011050:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609544320:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 821us (957us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.011054:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9663 00000100:00100000:2.0:1780298644.011865:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.011904:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609544448 00000100:00100000:0.0:1780298644.011910:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609544448 00000100:00100000:0.0:1780298644.011949:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9664 00000100:00100000:0.0:1780298644.011960:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e4f0450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609544448:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.011973:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.011983:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e6303494c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d39ac expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.012000:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.012016:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604990848:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.012028:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609544576 00000100:00100000:2.0:1780298644.012032:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609544576 00000100:00100000:2.0:1780298644.012054:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9665 00000100:00100000:2.0:1780298644.012061:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609544576:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.012067:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298644.012071:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00010000:00010000:2.0:1780298644.012074:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000b8a2561/0xaf35132e63034953 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d39b3 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.012094:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f746300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604990976:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.012137:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4b40 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.012665:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604990848:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.012678:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e6303494c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d39ac expref: 14 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.012696:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e6303494c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d39ac expref: 14 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.012713:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf200, rc 0) 00000100:00100000:0.0:1780298644.012744:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e4f0450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609544448:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 788us (878us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.012749:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9664 00000100:00100000:2.0:1780298644.012831:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f746300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604990976:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.012857:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000b8a2561/0xaf35132e63034953 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d39b3 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.012872:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000b8a2561/0xaf35132e63034953 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d39b3 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.012883:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf3582cb000, rc 0) 00000100:00100000:2.0:1780298644.012913:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609544576:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 852us (950us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.012920:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9665 00000100:00100000:3.0:1780298644.013762:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.013792:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609544704 00000100:00100000:1.0:1780298644.013796:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609544704 00000100:00100000:1.0:1780298644.013824:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6234 00000100:00100000:1.0:1780298644.013829:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609544704:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298644.013974:0:19950:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4155114 -> 4155114 00002000:00010000:1.0:1780298644.013977:0:19950:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298644.013978:0:19950:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298644.013980:0:19950:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298644.013981:0:19950:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 16 -> 32 00000100:00100000:1.0:1780298644.014005:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609544704:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 176us (245us total) trans 4294969251 rc 0/0 00000100:00100000:1.0:1780298644.014009:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6234 00000100:00100000:3.0:1780298644.014771:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.014800:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609544832 00000100:00100000:1.0:1780298644.014804:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609544832 00000100:00100000:1.0:1780298644.014826:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9666 00000100:00100000:1.0:1780298644.014831:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2399:x1866777609544832:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.014840:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.014848:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e6303495a lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d39ba expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.014857:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.014881:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609544960 00000100:00100000:1.0:1780298644.014881:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548bb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604991104:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.014888:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609544960 00000100:00100000:2.0:1780298644.014929:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9667 00000100:00100000:1.0:1780298644.014931:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.014940:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609544960:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.014948:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.014956:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003bced65c/0xaf35132e63034961 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d39c1 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.014981:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f746300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604991232:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.015015:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4b40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.015626:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548bb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604991104:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.015645:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f746300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604991232:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.015651:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e6303495a lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d39ba expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.015664:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e6303495a lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d39ba expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.015668:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003bced65c/0xaf35132e63034961 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d39c1 expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.015679:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279200, rc 0) 00010000:00010000:2.0:1780298644.015685:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003bced65c/0xaf35132e63034961 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d39c1 expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.015697:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf3582cbe00, rc 0) 00000100:00100000:1.0:1780298644.015704:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2399:x1866777609544832:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 872us (934us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.015710:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9666 00000100:00100000:2.0:1780298644.015725:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609544960:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 787us (872us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.015733:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9667 00000100:00100000:3.0:1780298644.016748:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.016778:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609545088 00000100:00100000:1.0:1780298644.016782:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609545088 00000100:00100000:1.0:1780298644.016804:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9668 00000100:00100000:1.0:1780298644.016810:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2398:x1866777609545088:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.016817:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298644.016818:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:1.0:1780298644.016825:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63034968 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d39c8 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.016850:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548bb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604991360:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.016862:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609545216 00000100:00100000:2.0:1780298644.016868:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609545216 00000100:00100000:1.0:1780298644.016885:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.016908:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9669 00000100:00100000:2.0:1780298644.016918:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609545216:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.016926:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.016936:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007544ea9c/0xaf35132e6303496f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d39cf expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.016962:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f746300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604991488:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.016996:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4b40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.017487:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548bb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604991360:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.017499:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63034968 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d39c8 expref: 14 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.017514:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63034968 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d39c8 expref: 14 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.017521:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279c00, rc 0) 00000100:00100000:1.0:1780298644.017541:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2398:x1866777609545088:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 732us (795us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.017546:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9668 00000100:00100000:0.0:1780298644.017666:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f746300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604991488:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.017679:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007544ea9c/0xaf35132e6303496f lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d39cf expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.017689:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007544ea9c/0xaf35132e6303496f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d39cf expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.017696:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf3582cba00, rc 0) 00000100:00100000:0.0:1780298644.017717:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609545216:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 802us (901us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.017724:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9669 00000100:00100000:3.0:1780298644.018516:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.018551:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609545344 00000100:00100000:0.0:1780298644.018556:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609545344 00000100:00100000:0.0:1780298644.018586:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9670 00000100:00100000:0.0:1780298644.018594:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609545344:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.018602:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.018610:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c37b15a2/0xaf35132e63034976 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d39d6 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.018638:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604991616:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.018676:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.019214:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604991616:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.019224:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c37b15a2/0xaf35132e63034976 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d39d6 expref: 19 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.019233:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c37b15a2/0xaf35132e63034976 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d39d6 expref: 19 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.019239:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baee00, rc 0) 00000100:00100000:0.0:1780298644.019259:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609545344:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 668us (746us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.019264:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9670 00000100:00100000:2.0:1780298644.020014:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.020057:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609545472 00000100:00100000:3.0:1780298644.020062:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609545472 00000100:00100000:3.0:1780298644.020089:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9671 00000100:00100000:3.0:1780298644.020096:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2399:x1866777609545472:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.020104:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.020112:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008746f5b4/0xaf35132e6303497d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d39dd expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.020166:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433dd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604991744:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.020206:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856600 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.020764:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433dd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604991744:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.020779:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008746f5b4/0xaf35132e6303497d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d39dd expref: 19 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.020792:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008746f5b4/0xaf35132e6303497d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d39dd expref: 19 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.020800:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157200, rc 0) 00000100:00100000:2.0:1780298644.020827:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2399:x1866777609545472:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 732us (815us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.020833:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9671 00000100:00100000:1.0:1780298644.021801:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.021840:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609545600 00000100:00100000:0.0:1780298644.021844:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609545600 00000100:00100000:0.0:1780298644.021868:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9672 00000100:00100000:0.0:1780298644.021873:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548bb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2398:x1866777609545600:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.021879:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.021886:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63034984 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d39e4 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.021907:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604991872:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.021936:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.022636:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604991872:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.022649:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63034984 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d39e4 expref: 19 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.022662:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63034984 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d39e4 expref: 19 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.022669:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae400, rc 0) 00000100:00100000:0.0:1780298644.022690:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548bb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2398:x1866777609545600:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 818us (891us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.022700:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9672 00000100:00100000:3.0:1780298644.023402:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.023431:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609545728 00000100:00100000:0.0:1780298644.023435:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609545728 00000100:00100000:0.0:1780298644.023456:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9673 00000100:00100000:0.0:1780298644.023460:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:29360:x1866777609545728:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.023466:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.023473:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e6303498b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d39eb expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.023485:0:19992:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63034937 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2641920->36196351] (req 2641920->2654207) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3997 expref: 10 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.023491:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(1 122) 00010000:00010000:0.0:1780298644.023494:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(2 122) 00010000:00010000:0.0:1780298644.023497:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e6303498b lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3702784->3710975] (req 3702784->3710975) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d39eb expref: 19 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.023511:0:19992:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63034937 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2641920->36196351] (req 2641920->2654207) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3997 expref: 10 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.023521:0:19992:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63034937 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2641920->36196351] (req 2641920->2654207) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3997 expref: 10 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.023528:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604992000:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298644.023555:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.024101:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604992000:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298644.024112:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e6303498b lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3702784->3710975] (req 3702784->3710975) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d39eb expref: 19 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.024119:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf600, rc 0) 00000100:00100000:0.0:1780298644.024158:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:29360:x1866777609545728:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 697us (757us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.024164:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9673 00000100:00100000:3.0:1780298644.024472:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.024511:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609545856 00000100:00100000:2.0:1780298644.024516:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609545856 00002000:00100000:2.0:1780298644.024539:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x12:0x0] (2641920->2654207) req@ffff9cf348318e00 x1866777609545856/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 12776/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.024550:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf348318e00 with x1866777609545856 ext(2641920->2654207) 00010000:00010000:2.0:1780298644.024555:0:8363:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63034937 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2641920->36196351] (req 2641920->2654207) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3997 expref: 11 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.024564:0:8363:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63034937 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2641920->36196351] (req 2641920->2654207) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3997 expref: 11 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1780298644.024569:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9cf348318e00 00000100:00100000:2.0:1780298644.024584:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 201 00000100:00100000:2.0:1780298644.024589:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348318e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609545856:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:2.0:1780298644.025075:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348318e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609545856:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 485us (603us total) trans 4294969252 rc 0/0 00000100:00100000:2.0:1780298644.025080:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 201 00002000:00100000:2.0:1780298644.025084:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x12:0x0] (2641920->2654207) req@ffff9cf348318e00 x1866777609545856/t4294969252(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 12776/448 e 0 to 0 dl 1780298655 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.025090:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf348318e00 with x1866777609545856 ext(2641920->2654207) 00010000:00010000:2.0:1780298644.025094:0:8363:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63034937 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2641920->36196351] (req 2641920->2654207) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3997 expref: 13 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.025102:0:8363:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63034937 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2641920->36196351] (req 2641920->2654207) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3997 expref: 13 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1780298644.025107:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9cf348318e00 00000100:00100000:3.0:1780298644.025809:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.025850:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609545984 00000100:00100000:2.0:1780298644.025854:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609545984 00010000:00010000:2.0:1780298644.025872:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63034937 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2641920->36196351] (req 2641920->2654207) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3997 expref: 13 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.025900:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15146 00000100:00100000:2.0:1780298644.025908:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b5e450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2397:x1866777609545984:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298644.025920:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:2.0:1780298644.025928:0:6461:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 32 -> 56 00010000:00010000:2.0:1780298644.025933:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63034937 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2641920->36196351] (req 2641920->2654207) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3997 expref: 13 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.025944:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63034937 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2641920->36196351] (req 2641920->2654207) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3997 expref: 13 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.025958:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63034937 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2641920->36196351] (req 2641920->2654207) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3997 expref: 13 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.025969:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(2 122) 00010000:00010000:2.0:1780298644.025972:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(2 122) 00010000:00010000:2.0:1780298644.025980:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e6303498b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3702784->37257215] (req 3702784->3710975) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d39eb expref: 18 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.025990:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e6303498b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3702784->37257215] (req 3702784->3710975) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d39eb expref: 18 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.026024:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e6303498b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3702784->37257215] (req 3702784->3710975) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d39eb expref: 18 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.026031:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f745c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6461:1866777604992128:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298644.026064:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4b40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.026745:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f745c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6461:1866777604992128:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298644.026756:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298644.026778:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b5e450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609545984:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 872us (970us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.026783:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15146 00000100:00100000:3.0:1780298644.026957:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.026989:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609546112 00000100:00100000:0.0:1780298644.026993:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609546112 00000100:00100000:0.0:1780298644.027020:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6235 00000100:00100000:0.0:1780298644.027026:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348318000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2400:x1866777609546112:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298644.027121:0:19949:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4155114 -> 4155114 00002000:00010000:0.0:1780298644.027147:0:19949:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298644.027150:0:19949:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298644.027152:0:19949:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298644.027155:0:19949:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 56 -> 72 00000100:00100000:0.0:1780298644.027183:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348318000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2400:x1866777609546112:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 157us (226us total) trans 4294969253 rc 0/0 00000100:00100000:0.0:1780298644.027192:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6235 00000100:00100000:2.0:1780298644.028066:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.028104:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609546240 00000100:00100000:0.0:1780298644.028110:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609546240 00000100:00100000:0.0:1780298644.028171:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9674 00000100:00100000:0.0:1780298644.028180:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2400:x1866777609546240:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.028190:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.028200:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e63034992 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d39f2 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.028232:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604992256:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.028270:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.028913:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604992256:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.028927:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e63034992 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d39f2 expref: 21 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.028938:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e63034992 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d39f2 expref: 21 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.028944:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf200, rc 0) 00000100:00100000:0.0:1780298644.028967:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2400:x1866777609546240:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 789us (901us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.028971:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9674 00000100:00100000:3.0:1780298644.029813:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.029850:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609546368 00000100:00100000:2.0:1780298644.029856:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609546368 00000100:00100000:2.0:1780298644.029888:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9675 00000100:00100000:2.0:1780298644.029897:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348318380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2400:x1866777609546368:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.029906:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.029914:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003bced65c/0xaf35132e63034999 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d39f9 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.029936:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f747100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604992384:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.029969:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.030564:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f747100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604992384:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.030581:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003bced65c/0xaf35132e63034999 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d39f9 expref: 21 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.030594:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003bced65c/0xaf35132e63034999 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d39f9 expref: 21 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.030604:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf3582cbe00, rc 0) 00000100:00100000:3.0:1780298644.030634:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348318380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2400:x1866777609546368:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 738us (822us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.030641:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9675 00000100:00100000:1.0:1780298644.031405:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.031434:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609546496 00000100:00100000:0.0:1780298644.031438:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609546496 00000100:00100000:1.0:1780298644.031444:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.031470:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609546624 00000100:00100000:0.0:1780298644.031472:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609546624 00000100:00100000:2.0:1780298644.031483:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9676 00000100:00100000:2.0:1780298644.031494:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548bad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2399:x1866777609546496:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298644.031500:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9677 00010000:00010000:2.0:1780298644.031511:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298644.031511:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609546624:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.031520:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.031524:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001b8cdae1/0xaf35132e630349a0 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3a00 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.031528:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e630349a7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3a07 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.031571:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f747100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604992512:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.031573:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604992640:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.031614:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.031615:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.032215:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f747100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604992512:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.032230:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001b8cdae1/0xaf35132e630349a0 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a00 expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.032235:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604992640:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.032255:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001b8cdae1/0xaf35132e630349a0 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a00 expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.032257:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e630349a7 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a07 expref: 13 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.032267:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf3582cae00, rc 0) 00010000:00010000:1.0:1780298644.032290:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e630349a7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a07 expref: 13 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.032307:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548bad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2399:x1866777609546496:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 816us (901us total) trans 0 rc 301/301 00010000:00010000:1.0:1780298644.032308:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baea00, rc 0) 00000100:00100000:2.0:1780298644.032315:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9676 00000100:00100000:1.0:1780298644.032339:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609546624:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 829us (895us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.032345:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9677 00000100:00100000:3.0:1780298644.033214:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.033244:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609546752 00000100:00100000:0.0:1780298644.033249:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609546752 00000100:00100000:0.0:1780298644.033278:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9678 00000100:00100000:0.0:1780298644.033299:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34831a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:29360:x1866777609546752:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.033307:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.033316:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e630349ae lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3a0e expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.033334:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(2 122) 00010000:00010000:0.0:1780298644.033344:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(2 122) 00010000:00010000:0.0:1780298644.033353:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e630349ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2412544->3702783] (req 2412544->2420735) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a0e expref: 13 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.033364:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e630349ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2412544->3702783] (req 2412544->2420735) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a0e expref: 13 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.033370:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae400, rc 0) 00000100:00100000:0.0:1780298644.033393:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34831a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:29360:x1866777609546752:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 98us (182us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.033398:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9678 00000100:00100000:3.0:1780298644.033958:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.033990:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609546880 00000100:00100000:1.0:1780298644.033994:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609546880 00000100:00100000:1.0:1780298644.034022:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6236 00000100:00100000:1.0:1780298644.034027:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34831b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609546880:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298644.034196:0:8364:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4155114 -> 4155114 00002000:00010000:1.0:1780298644.034199:0:8364:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298644.034200:0:8364:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298644.034202:0:8364:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298644.034203:0:8364:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 72 -> 96 00000100:00100000:1.0:1780298644.034224:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34831b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2400:x1866777609546880:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 197us (267us total) trans 4294969254 rc 0/0 00000100:00100000:1.0:1780298644.034229:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6236 00000100:00100000:3.0:1780298644.035055:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.035088:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609547008 00000100:00100000:2.0:1780298644.035092:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609547008 00000100:00100000:2.0:1780298644.035113:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9679 00000100:00100000:2.0:1780298644.035118:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348319180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2399:x1866777609547008:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.035140:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.035147:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007dbcc772/0xaf35132e630349b5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3a15 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.035180:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f747100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604992768:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.035223:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4600 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.035842:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f747100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604992768:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.035854:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007dbcc772/0xaf35132e630349b5 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a15 expref: 21 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.035863:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007dbcc772/0xaf35132e630349b5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a15 expref: 21 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.035870:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf3582cb400, rc 0) 00000100:00100000:2.0:1780298644.035891:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348319180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2399:x1866777609547008:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 773us (837us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.035895:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9679 00000100:00100000:3.0:1780298644.036672:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.036728:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609547136 00000100:00100000:0.0:1780298644.036733:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609547136 00000100:00100000:0.0:1780298644.036763:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9680 00000100:00100000:0.0:1780298644.036772:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348318a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2397:x1866777609547136:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.036781:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.036790:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e630349bc lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3a1c expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.036819:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604992896:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.036860:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.037475:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604992896:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.037491:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e630349bc lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a1c expref: 21 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.037505:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e630349bc lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a1c expref: 21 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.037516:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf200, rc 0) 00000100:00100000:0.0:1780298644.037546:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348318a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2397:x1866777609547136:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 776us (874us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.037554:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9680 00000100:00100000:3.0:1780298644.038496:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.038598:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609547264 00000100:00100000:1.0:1780298644.038603:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609547264 00000100:00100000:3.0:1780298644.038633:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.038690:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609547392 00000100:00100000:1.0:1780298644.038694:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609547392 00000100:00100000:2.0:1780298644.038704:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9681 00000100:00100000:2.0:1780298644.038714:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34831aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2398:x1866777609547264:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.038722:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298644.038722:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9682 00010000:00010000:2.0:1780298644.038733:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004f507fef/0xaf35132e630349c3 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3a23 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.038740:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348318700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2398:x1866777609547392:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.038752:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.038762:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c37b15a2/0xaf35132e630349ca lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3a2a expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.038765:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f747100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604993024:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.038803:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604993152:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.038806:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.038834:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.039550:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f747100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604993024:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.039566:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604993152:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.039570:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004f507fef/0xaf35132e630349c3 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a23 expref: 14 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.039584:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004f507fef/0xaf35132e630349c3 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a23 expref: 14 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.039588:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c37b15a2/0xaf35132e630349ca lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a2a expref: 16 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.039605:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf3582ca800, rc 0) 00010000:00010000:0.0:1780298644.039612:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c37b15a2/0xaf35132e630349ca lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a2a expref: 16 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.039624:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baee00, rc 0) 00000100:00100000:2.0:1780298644.039640:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34831aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2398:x1866777609547264:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 929us (1144us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.039648:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348318700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2398:x1866777609547392:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 914us (1014us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.039651:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9681 00000100:00100000:0.0:1780298644.039660:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9682 00000100:00100000:3.0:1780298644.040698:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.040732:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609547520 00000100:00100000:2.0:1780298644.040736:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609547520 00000100:00100000:2.0:1780298644.040762:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9683 00000100:00100000:2.0:1780298644.040766:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34831bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:29360:x1866777609547520:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.040772:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.040779:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001b8cdae1/0xaf35132e630349d1 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3a31 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.040794:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000fba32a6f/0xaf35132e630348f1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d395f expref: 19 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.040800:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298643 v=(169 177) 00010000:00010000:2.0:1780298644.040803:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(1 169) 00010000:00010000:2.0:1780298644.040806:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001b8cdae1/0xaf35132e630349d1 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3047424->3051519] (req 3047424->3051519) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a31 expref: 14 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.040825:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000fba32a6f/0xaf35132e630348f1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d395f expref: 19 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.040835:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000fba32a6f/0xaf35132e630348f1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d395f expref: 19 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.040842:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f747100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604993280:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298644.040872:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4600 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.041454:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f747100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604993280:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298644.041469:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001b8cdae1/0xaf35132e630349d1 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3047424->3051519] (req 3047424->3051519) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a31 expref: 14 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.041481:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf3582cae00, rc 0) 00000100:00100000:2.0:1780298644.041511:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34831bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:29360:x1866777609547520:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 743us (813us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.041519:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9683 00000100:00100000:3.0:1780298644.041657:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.041686:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609547648 00000100:00100000:0.0:1780298644.041690:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609547648 00010000:00010000:0.0:1780298644.041716:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000fba32a6f/0xaf35132e630348f1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d395f expref: 20 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.041747:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15147 00000100:00100000:0.0:1780298644.041752:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2399:x1866777609547648:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298644.041761:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298644.041769:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000fba32a6f/0xaf35132e630348f1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d395f expref: 20 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.041778:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000fba32a6f/0xaf35132e630348f1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d395f expref: 20 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.041795:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fba32a6f/0xaf35132e630348f1 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d395f expref: 20 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.041805:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(1 169) 00010000:00010000:0.0:1780298644.041809:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(1 169) 00010000:00010000:0.0:1780298644.041816:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000001b8cdae1/0xaf35132e630349d1 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3047424->3051519) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a31 expref: 13 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.041826:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001b8cdae1/0xaf35132e630349d1 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3047424->3051519) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a31 expref: 13 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.041849:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000001b8cdae1/0xaf35132e630349d1 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3047424->3051519) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a31 expref: 13 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.041857:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519f480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777604993408:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298644.041886:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3484176c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.042458:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519f480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777604993408:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298644.042465:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298644.042489:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2399:x1866777609547648:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 736us (831us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.042497:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15147 00000100:00100000:3.0:1780298644.042653:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.042682:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609547776 00000100:00100000:0.0:1780298644.042685:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609547776 00002000:00100000:0.0:1780298644.042708:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (3047424->3051519) req@ffff9cf2558f1c00 x1866777609547776/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298644.042719:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2558f1c00 with x1866777609547776 ext(3047424->3051519) 00002000:00010000:0.0:1780298644.042724:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2558f1c00 00000100:00100000:0.0:1780298644.042745:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6237 00000100:00100000:0.0:1780298644.042753:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609547776:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298644.042879:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609547776:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 127us (227us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298644.042884:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6237 00002000:00100000:0.0:1780298644.042889:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (3047424->3051519) req@ffff9cf2558f1c00 x1866777609547776/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/4536 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298644.042895:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2558f1c00 with x1866777609547776 ext(3047424->3051519) 00002000:00010000:0.0:1780298644.042897:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2558f1c00 00000100:00100000:3.0:1780298644.043798:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.043844:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609547904 00000100:00100000:0.0:1780298644.043849:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609547904 00000100:00100000:0.0:1780298644.043881:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9684 00000100:00100000:0.0:1780298644.043889:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2397:x1866777609547904:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.043898:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.043907:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e630349d8 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3a38 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.043941:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604993536:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.043975:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.044595:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604993536:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.044610:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e630349d8 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a38 expref: 16 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.044624:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e630349d8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a38 expref: 16 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.044634:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baec00, rc 0) 00000100:00100000:0.0:1780298644.044657:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2397:x1866777609547904:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 770us (860us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.044661:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9684 00000100:00100000:3.0:1780298644.045327:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.045364:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609548032 00000100:00100000:1.0:1780298644.045368:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609548032 00000100:00100000:1.0:1780298644.045392:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9685 00000100:00100000:1.0:1780298644.045398:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2397:x1866777609548032:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.045405:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.045412:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e630349df lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3a3f expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.045439:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548b9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604993664:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.045471:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.046034:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548b9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604993664:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.046047:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e630349df lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a3f expref: 16 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.046055:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e630349df lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a3f expref: 16 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.046062:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279e00, rc 0) 00000100:00100000:1.0:1780298644.046082:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2397:x1866777609548032:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 685us (757us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.046087:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9685 00000100:00100000:3.0:1780298644.046856:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.046885:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609548160 00000100:00100000:0.0:1780298644.046889:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609548160 00000100:00100000:0.0:1780298644.046910:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9686 00000100:00100000:0.0:1780298644.046915:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2397:x1866777609548160:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.046921:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.046927:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e630349e6 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3a46 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.046950:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604993792:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.046977:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.047567:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604993792:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.047578:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e630349e6 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a46 expref: 16 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.047587:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e630349e6 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a46 expref: 16 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.047593:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf200, rc 0) 00000100:00100000:0.0:1780298644.047613:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2397:x1866777609548160:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 699us (760us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.047618:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9686 00000100:00100000:3.0:1780298644.048477:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.048507:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609548288 00000100:00100000:1.0:1780298644.048511:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609548288 00002000:00100000:1.0:1780298644.048529:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (2166784->2170879) req@ffff9cf2558f2d80 x1866777609548288/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.048535:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2558f2d80 with x1866777609548288 ext(2166784->2170879) 00002000:00010000:1.0:1780298644.048538:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2558f2d80 00000100:00100000:1.0:1780298644.048552:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6238 00000100:00100000:1.0:1780298644.048557:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2398:x1866777609548288:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298644.048674:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2398:x1866777609548288:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 116us (198us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298644.048679:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6238 00002000:00100000:1.0:1780298644.048683:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (2166784->2170879) req@ffff9cf2558f2d80 x1866777609548288/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/4536 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.048689:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2558f2d80 with x1866777609548288 ext(2166784->2170879) 00002000:00010000:1.0:1780298644.048692:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2558f2d80 00000100:00100000:3.0:1780298644.049639:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.049729:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609548416 00000100:00100000:2.0:1780298644.049735:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609548416 00002000:00100000:2.0:1780298644.049758:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (2142208->2207743) req@ffff9cf2558f1500 x1866777609548416/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 504/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.049768:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2558f1500 with x1866777609548416 ext(2142208->2207743) 00002000:00010000:2.0:1780298644.049772:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2558f1500 00000100:00100000:2.0:1780298644.049793:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6239 00000100:00100000:2.0:1780298644.049802:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2398:x1866777609548416:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298644.050711:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2398:x1866777609548416:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 910us (1072us total) trans 0 rc 61440/61440 00000100:00100000:2.0:1780298644.050720:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6239 00002000:00100000:2.0:1780298644.050727:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (2142208->2207743) req@ffff9cf2558f1500 x1866777609548416/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 504/440 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 61440/61440 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.050736:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2558f1500 with x1866777609548416 ext(2142208->2207743) 00002000:00010000:2.0:1780298644.050739:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2558f1500 00000100:00100000:3.0:1780298644.051596:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.051633:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609548544 00000100:00100000:2.0:1780298644.051636:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609548544 00000100:00100000:2.0:1780298644.051662:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9687 00000100:00100000:3.0:1780298644.051667:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.051671:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2400:x1866777609548544:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.051678:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.051689:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004f507fef/0xaf35132e630349ed lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3a4d expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.051695:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609548672 00000100:00100000:1.0:1780298644.051701:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609548672 00010000:00010000:2.0:1780298644.051706:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(1 169) 00010000:00010000:2.0:1780298644.051710:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(1 169) 00010000:00010000:2.0:1780298644.051716:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004f507fef/0xaf35132e630349ed lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a4d expref: 20 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.051727:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004f507fef/0xaf35132e630349ed lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a4d expref: 20 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.051734:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf3582ca800, rc 0) 00000100:00100000:1.0:1780298644.051740:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9688 00000100:00100000:1.0:1780298644.051750:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2400:x1866777609548672:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.051760:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298644.051762:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2400:x1866777609548544:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 94us (166us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.051767:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9687 00010000:00010000:1.0:1780298644.051772:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005b92bd2b/0xaf35132e630349f4 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3a54 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.051811:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548b9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604993920:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.051860:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.052483:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548b9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604993920:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.052498:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005b92bd2b/0xaf35132e630349f4 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a54 expref: 21 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.052516:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005b92bd2b/0xaf35132e630349f4 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a54 expref: 21 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.052523:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279c00, rc 0) 00000100:00100000:1.0:1780298644.052544:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2400:x1866777609548672:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 798us (880us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.052549:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9688 00000100:00100000:3.0:1780298644.053354:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.053382:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609548800 00000100:00100000:1.0:1780298644.053385:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609548800 00000100:00100000:1.0:1780298644.053407:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9689 00000100:00100000:1.0:1780298644.053412:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2399:x1866777609548800:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.053417:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.053424:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e630349fb lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3a5b expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.053443:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548b9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604994048:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.053471:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.054089:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548b9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604994048:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.054107:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e630349fb lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a5b expref: 21 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.054122:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e630349fb lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a5b expref: 21 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.054158:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279e00, rc 0) 00000100:00100000:1.0:1780298644.054189:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2399:x1866777609548800:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 776us (837us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.054196:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9689 00000100:00100000:3.0:1780298644.054935:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.054967:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609548928 00000100:00100000:0.0:1780298644.054972:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609548928 00000100:00100000:0.0:1780298644.055003:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9690 00000100:00100000:0.0:1780298644.055010:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2399:x1866777609548928:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.055017:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.055023:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e63034a02 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3a62 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.055044:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604994176:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.055073:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.055682:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604994176:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.055697:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e63034a02 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a62 expref: 16 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.055710:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e63034a02 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a62 expref: 16 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.055717:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baec00, rc 0) 00000100:00100000:0.0:1780298644.055737:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2399:x1866777609548928:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 729us (805us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.055742:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9690 00000100:00100000:1.0:1780298644.056595:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.056623:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609549056 00000100:00100000:0.0:1780298644.056626:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609549056 00002000:00100000:0.0:1780298644.056644:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (4030464->4034559) req@ffff9cf2548b9f80 x1866777609549056/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298644.056650:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2548b9f80 with x1866777609549056 ext(4030464->4034559) 00002000:00010000:0.0:1780298644.056653:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2548b9f80 00000100:00100000:0.0:1780298644.056669:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6240 00000100:00100000:0.0:1780298644.056673:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2399:x1866777609549056:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298644.056783:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2399:x1866777609549056:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 109us (188us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298644.056788:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6240 00002000:00100000:0.0:1780298644.056792:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (4030464->4034559) req@ffff9cf2548b9f80 x1866777609549056/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/4536 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298644.056798:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2548b9f80 with x1866777609549056 ext(4030464->4034559) 00002000:00010000:0.0:1780298644.056800:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2548b9f80 00000100:00100000:3.0:1780298644.057706:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.057748:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609549184 00000100:00100000:1.0:1780298644.057752:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609549184 00002000:00100000:1.0:1780298644.057770:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (4034560->4038655) req@ffff9cf2558f2a00 x1866777609549184/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.057776:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2558f2a00 with x1866777609549184 ext(4034560->4038655) 00002000:00010000:1.0:1780298644.057779:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2558f2a00 00000100:00100000:1.0:1780298644.057794:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6241 00000100:00100000:1.0:1780298644.057799:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2399:x1866777609549184:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298644.057865:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2399:x1866777609549184:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 66us (160us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298644.057871:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6241 00002000:00100000:1.0:1780298644.057875:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (4034560->4038655) req@ffff9cf2558f2a00 x1866777609549184/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/4536 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.057881:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2558f2a00 with x1866777609549184 ext(4034560->4038655) 00002000:00010000:1.0:1780298644.057883:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2558f2a00 00000100:00100000:3.0:1780298644.058736:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.058772:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609549312 00000100:00100000:2.0:1780298644.058776:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609549312 00000100:00100000:2.0:1780298644.058800:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9691 00000100:00100000:2.0:1780298644.058806:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2399:x1866777609549312:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.058813:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.058821:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007dbcc772/0xaf35132e63034a09 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3a69 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.058855:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f747100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604994304:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.058900:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4600 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.059563:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f747100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604994304:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.059582:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007dbcc772/0xaf35132e63034a09 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a69 expref: 16 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.059598:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007dbcc772/0xaf35132e63034a09 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a69 expref: 16 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.059607:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf3582cb400, rc 0) 00000100:00100000:2.0:1780298644.059636:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2399:x1866777609549312:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 830us (901us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.059642:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9691 00000100:00100000:3.0:1780298644.060547:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.060587:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609549440 00000100:00100000:1.0:1780298644.060592:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609549440 00000100:00100000:1.0:1780298644.060625:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9692 00000100:00100000:1.0:1780298644.060631:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2400:x1866777609549440:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.060639:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.060651:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63034a10 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3a70 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.060672:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548b9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604994432:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.060703:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.061314:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548b9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604994432:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.061334:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63034a10 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a70 expref: 16 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.061348:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63034a10 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a70 expref: 16 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.061359:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278a00, rc 0) 00000100:00100000:1.0:1780298644.061386:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2400:x1866777609549440:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 755us (840us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.061393:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9692 00000100:00100000:0.0:1780298644.062191:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.062232:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609549568 00000100:00100000:1.0:1780298644.062237:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609549568 00000100:00100000:1.0:1780298644.062267:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9693 00000100:00100000:1.0:1780298644.062276:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2399:x1866777609549568:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.062297:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.062306:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005b92bd2b/0xaf35132e63034a17 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3a77 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.062334:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548b9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604994560:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.062373:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.062967:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548b9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604994560:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.062981:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005b92bd2b/0xaf35132e63034a17 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a77 expref: 21 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.062991:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005b92bd2b/0xaf35132e63034a17 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a77 expref: 21 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.062997:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279c00, rc 0) 00000100:00100000:1.0:1780298644.063020:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2399:x1866777609549568:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 747us (830us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.063024:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9693 00000100:00100000:2.0:1780298644.063941:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.063975:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609549696 00000100:00100000:0.0:1780298644.063979:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609549696 00002000:00100000:0.0:1780298644.064002:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (1413120->1417215) req@ffff9cf37f747100 x1866777609549696/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298644.064009:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f747100 with x1866777609549696 ext(1413120->1417215) 00002000:00010000:0.0:1780298644.064013:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f747100 00000100:00100000:0.0:1780298644.064029:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6242 00000100:00100000:0.0:1780298644.064035:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f747100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2400:x1866777609549696:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298644.064174:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f747100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2400:x1866777609549696:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 138us (234us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298644.064179:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6242 00002000:00100000:0.0:1780298644.064184:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (1413120->1417215) req@ffff9cf37f747100 x1866777609549696/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/4536 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298644.064190:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f747100 with x1866777609549696 ext(1413120->1417215) 00002000:00010000:0.0:1780298644.064193:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f747100 00000100:00100000:0.0:1780298644.065105:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.065162:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609549824 00000100:00100000:1.0:1780298644.065167:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609549824 00000100:00100000:1.0:1780298644.065197:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9694 00000100:00100000:1.0:1780298644.065206:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2397:x1866777609549824:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.065215:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.065224:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e63034a1e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3a7e expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.065255:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548b9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604994688:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.065310:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.065936:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548b9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604994688:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.065957:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e63034a1e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a7e expref: 21 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.065973:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e63034a1e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a7e expref: 21 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.065984:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278400, rc 0) 00000100:00100000:1.0:1780298644.066013:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2397:x1866777609549824:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 809us (909us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.066020:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9694 00000100:00100000:2.0:1780298644.066803:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.066844:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609549952 00000100:00100000:3.0:1780298644.066849:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609549952 00000100:00100000:3.0:1780298644.066884:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9695 00000100:00100000:3.0:1780298644.066893:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2398:x1866777609549952:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.066902:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.066911:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006d4295ea/0xaf35132e63034a25 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3a85 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.066942:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604994816:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.066984:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.067675:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604994816:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.067693:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000006d4295ea/0xaf35132e63034a25 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a85 expref: 21 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.067707:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006d4295ea/0xaf35132e63034a25 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a85 expref: 21 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.067716:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157400, rc 0) 00000100:00100000:3.0:1780298644.067744:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2398:x1866777609549952:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 853us (941us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.067751:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9695 00000100:00100000:0.0:1780298644.068677:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.068723:0:6476:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609550208 00000100:00100000:1.0:1780298644.068728:0:6476:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609550208 00000100:00100000:1.0:1780298644.068781:0:6476:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62007 00000100:00080000:2.0:1780298644.068782:0:6473:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1780298644.068793:0:6476:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+118:29360:x1866777609550208:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:1.0:1780298644.068837:0:6476:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298644.068856:0:6476:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09600/0xaf35132e63034906 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 83 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d396d expref: 118 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.068874:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.068878:0:6476:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298644.069040:0:6476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279e00/0xaf35132e63034a2c lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 82 type: IBT flags: 0x40000000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.069055:0:6476:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bec00/0xaf35132e63034930 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 82 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3990 expref: 157 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.069076:0:6476:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279e00/0xaf35132e63034a2c lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 82 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.069098:0:6476:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bec00/0xaf35132e63034930 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 82 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3990 expref: 157 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.069112:0:6476:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bec00/0xaf35132e63034930 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 82 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3990 expref: 157 pid: 7411 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.069137:0:6476:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548b9180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6476:1866777604994944:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298644.069181:0:6476:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a540 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.069766:0:6476:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548b9180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6476:1866777604994944:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298644.069777:0:6476:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279e00/0xaf35132e63034a2c lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 82 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.069953:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.069999:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609550336 00000100:00100000:1.0:1780298644.070004:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609550336 00010000:00010000:1.0:1780298644.070022:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bec00/0xaf35132e63034930 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 82 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3990 expref: 158 pid: 7411 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.070046:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15148 00000100:00100000:1.0:1780298644.070055:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+158:2399:x1866777609550336:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298644.070064:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298644.070069:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bec00/0xaf35132e63034930 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 83 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3990 expref: 158 pid: 7411 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.070080:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bec00/0xaf35132e63034930 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 83 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3990 expref: 158 pid: 7411 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.070093:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bec00/0xaf35132e63034930 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 83 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3990 expref: 158 pid: 7411 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.070104:0:7400:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.070108:0:7400:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279e00/0xaf35132e63034a2c lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 82 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.070119:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279e00/0xaf35132e63034a2c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 82 type: IBT flags: 0x40290000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.070155:0:7400:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279e00/0xaf35132e63034a2c lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 82 type: IBT flags: 0x40290000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.070178:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298644.070205:0:6476:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279e00/0xaf35132e63034a2c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 81 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.070207:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+157:2399:x1866777609550336:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 154us (254us total) trans 0 rc 0/0 00010000:00010000:2.0:1780298644.070220:0:6476:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279e00/0xaf35132e63034a2c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 81 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.070225:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15148 00000001:00080000:2.0:1780298644.070277:0:6476:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991066 00000004:00010000:2.0:1780298644.070313:0:6476:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 0000000087e7fd4b reply state 00000000cba4b933 transno 4294991067 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279e00/0xaf35132e63034a2c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 81 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.070326:0:6476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279e00/0xaf35132e63034a2c lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 81 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.070333:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298644.070336:0:6476:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1780298644.070338:0:6476:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1780298644.070399:0:6476:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+120:29360:x1866777609550208:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1610us (1722us total) trans 4294991067 rc 0/0 00000100:00100000:2.0:1780298644.070404:0:6476:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62007 00000020:00080000:2.0:1780298644.070407:0:6476:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1780298644.070991:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.071020:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609550464 00000100:00100000:1.0:1780298644.071024:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609550464 00000100:00100000:1.0:1780298644.071046:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9696 00000100:00100000:1.0:1780298644.071051:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:29360:x1866777609550464:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.071058:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.071067:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63034a33 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3a8c expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.071082:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000001b8cdae1/0xaf35132e630349d1 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3047424->3051519) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a31 expref: 15 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.071088:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000004f507fef/0xaf35132e630349ed lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a4d expref: 19 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.071092:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(1 169) 00010000:00010000:1.0:1780298644.071095:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(1 169) 00010000:00010000:1.0:1780298644.071098:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63034a33 lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a8c expref: 15 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.071114:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000004f507fef/0xaf35132e630349ed lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a4d expref: 19 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.071121:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000004f507fef/0xaf35132e630349ed lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a4d expref: 19 pid: 8356 timeout: 933 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.071148:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548b9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604995072:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298644.071185:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000001b8cdae1/0xaf35132e630349d1 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3047424->3051519) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a31 expref: 15 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.071193:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000001b8cdae1/0xaf35132e630349d1 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3047424->3051519) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a31 expref: 15 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.071200:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548b8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604995200:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298644.071218:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a540 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.071830:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548b9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604995072:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298644.071841:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548b8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604995200:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298644.071858:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63034a33 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a8c expref: 15 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.071871:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279c00, rc 0) 00000100:00100000:1.0:1780298644.071899:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:29360:x1866777609550464:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 847us (908us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.071907:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9696 00000100:00100000:0.0:1780298644.072189:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.072215:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609550720 00000100:00100000:0.0:1780298644.072216:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.072222:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609550720 00010000:00010000:3.0:1780298644.072243:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000001b8cdae1/0xaf35132e630349d1 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3047424->3051519) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a31 expref: 15 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.072273:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609550592 00000100:00100000:3.0:1780298644.072276:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15149 00000100:00100000:2.0:1780298644.072291:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609550592 00000100:00100000:3.0:1780298644.072301:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2399:x1866777609550720:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298644.072311:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298644.072314:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000004f507fef/0xaf35132e630349ed lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a4d expref: 20 pid: 8356 timeout: 933 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.072329:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000001b8cdae1/0xaf35132e630349d1 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3047424->3051519) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a31 expref: 15 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.072341:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000001b8cdae1/0xaf35132e630349d1 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3047424->3051519) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a31 expref: 15 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.072342:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15150 00000100:00100000:2.0:1780298644.072352:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2400:x1866777609550592:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298644.072360:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298644.072364:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001b8cdae1/0xaf35132e630349d1 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3047424->3051519) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a31 expref: 15 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.072370:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000004f507fef/0xaf35132e630349ed lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a4d expref: 20 pid: 8356 timeout: 933 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.072383:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000004f507fef/0xaf35132e630349ed lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a4d expref: 20 pid: 8356 timeout: 933 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.072392:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298644.072398:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004f507fef/0xaf35132e630349ed lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a4d expref: 20 pid: 8356 timeout: 933 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.072407:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(1 169) 00010000:00010000:2.0:1780298644.072410:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(1 169) 00010000:00010000:2.0:1780298644.072415:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63034a33 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a8c expref: 14 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.072422:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63034a33 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a8c expref: 14 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.072426:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2399:x1866777609550720:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 125us (234us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298644.072436:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15149 00010000:00010000:2.0:1780298644.072447:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63034a33 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a8c expref: 13 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.072456:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f735880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777604995328:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298644.072498:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3629a4180 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.073177:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f735880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777604995328:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298644.073190:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298644.073215:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2400:x1866777609550592:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 865us (999us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.073221:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15150 00000100:00100000:1.0:1780298644.073317:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.073392:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609550848 00000100:00100000:3.0:1780298644.073396:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609550848 00000100:00100000:3.0:1780298644.073425:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9697 00000100:00100000:3.0:1780298644.073433:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:29360:x1866777609550848:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.073442:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.073452:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000004b2c7ae/0xaf35132e63034a3a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3a93 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.073470:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e630349ae lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2412544->3702783] (req 2412544->2420735) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a0e expref: 16 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.073478:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e6303498b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3702784->37257215] (req 3702784->3710975) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d39eb expref: 19 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.073484:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 2 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(2 122) 00010000:00010000:3.0:1780298644.073487:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(4 122) 00010000:00010000:3.0:1780298644.073491:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000004b2c7ae/0xaf35132e63034a3a lrc: 5/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a93 expref: 16 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.073509:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e6303498b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3702784->37257215] (req 3702784->3710975) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d39eb expref: 19 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.073520:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e6303498b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3702784->37257215] (req 3702784->3710975) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d39eb expref: 19 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.073530:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604995456:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298644.073578:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e630349ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2412544->3702783] (req 2412544->2420735) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a0e expref: 16 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.073588:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e630349ae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2412544->3702783] (req 2412544->2420735) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a0e expref: 16 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.073597:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604995584:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298644.073621:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.074311:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604995456:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298644.074325:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8b0380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604995584:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298644.074342:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000004b2c7ae/0xaf35132e63034a3a lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a93 expref: 16 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.074351:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156400, rc 0) 00000100:00100000:3.0:1780298644.074421:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:29360:x1866777609550848:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 946us (1061us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298644.074428:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9697 00000100:00100000:1.0:1780298644.074474:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.074512:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609550976 00000100:00100000:2.0:1780298644.074517:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609550976 00010000:00010000:2.0:1780298644.074536:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e6303498b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3702784->37257215] (req 3702784->3710975) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d39eb expref: 20 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.074565:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15151 00000100:00100000:2.0:1780298644.074572:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2399:x1866777609550976:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298644.074582:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298644.074594:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e6303498b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [3702784->37257215] (req 3702784->3710975) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d39eb expref: 20 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.074608:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e6303498b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [3702784->37257215] (req 3702784->3710975) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d39eb expref: 20 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.074609:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:2.0:1780298644.074627:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e6303498b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [3702784->37257215] (req 3702784->3710975) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d39eb expref: 20 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.074639:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298644.074667:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2399:x1866777609550976:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 96us (194us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.074675:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15151 00000100:00100000:2.0:1780298644.074688:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609551104 00000100:00100000:2.0:1780298644.074691:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609551104 00010000:00010000:2.0:1780298644.074705:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e630349ae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2412544->3702783] (req 2412544->2420735) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a0e expref: 16 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.074730:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15152 00000100:00100000:2.0:1780298644.074737:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2399:x1866777609551104:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298644.074744:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298644.074752:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e630349ae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2412544->3702783] (req 2412544->2420735) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a0e expref: 16 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.074761:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e630349ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2412544->3702783] (req 2412544->2420735) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a0e expref: 16 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.074772:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e630349ae lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2412544->3702783] (req 2412544->2420735) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a0e expref: 16 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.074782:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(4 122) 00010000:00010000:2.0:1780298644.074786:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(4 122) 00010000:00010000:2.0:1780298644.074791:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000004b2c7ae/0xaf35132e63034a3a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a93 expref: 15 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.074799:0:6463:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000004b2c7ae/0xaf35132e63034a3a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a93 expref: 15 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.074820:0:6463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000004b2c7ae/0xaf35132e63034a3a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a93 expref: 15 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.074834:0:6463:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f735880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777604995712:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298644.074863:0:6463:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3629a4180 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.075629:0:6463:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f735880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777604995712:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298644.075639:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298644.075664:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2399:x1866777609551104:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 928us (1059us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.075672:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15152 00000100:00100000:3.0:1780298644.075738:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.075794:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609551232 00000100:00100000:3.0:1780298644.075806:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.075807:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609551232 00000100:00100000:0.0:1780298644.075850:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609551360 00000100:00100000:0.0:1780298644.075855:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609551360 00002000:00010000:1.0:1780298644.075856:0:19950:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (153015->18446744073709551615) 00002000:00100000:1.0:1780298644.075861:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f8b0380 with x1866777609551232 ext(153015->18446744073709551615) 00002000:00010000:1.0:1780298644.075873:0:19950:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f8b0380. 00002000:00010000:0.0:1780298644.075882:0:8362:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00002000:00100000:0.0:1780298644.075888:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f8b1c00 with x1866777609551360 ext(0->18446744073709551615) 00002000:00010000:0.0:1780298644.075894:0:8362:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f8b1c00. 00000100:00100000:1.0:1780298644.075916:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6243 00000100:00100000:0.0:1780298644.075934:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6244 00000100:00100000:1.0:1780298644.075935:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609551232:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:0.0:1780298644.075951:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2399:x1866777609551360:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:0.0:1780298644.076394:0:8362:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4155114 -> 0 00002000:00010000:0.0:1780298644.076398:0:8362:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298644.076400:0:8362:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298644.076401:0:8362:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298644.076404:0:8362:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 96 -> 0 00000100:00100000:0.0:1780298644.076431:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2399:x1866777609551360:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 487us (627us total) trans 4294969255 rc 0/0 00000100:00100000:0.0:1780298644.076437:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6244 00002000:00010000:0.0:1780298644.076440:0:8362:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00002000:00100000:0.0:1780298644.076443:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f8b1c00 with x1866777609551360 ext(0->18446744073709551615) 00002000:00010000:0.0:1780298644.076448:0:8362:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f8b1c00. 00002000:00010000:1.0:1780298644.077359:0:19950:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 153015 00002000:00010000:1.0:1780298644.077364:0:19950:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298644.077366:0:19950:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298644.077368:0:19950:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298644.077371:0:19950:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 512 -> 304 00000100:00100000:1.0:1780298644.077432:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2400:x1866777609551232:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 1506us (1694us total) trans 4294969706 rc 0/0 00000100:00100000:1.0:1780298644.077439:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6243 00002000:00010000:1.0:1780298644.077444:0:19950:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (153015->18446744073709551615) 00002000:00100000:1.0:1780298644.077447:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f8b0380 with x1866777609551232 ext(153015->18446744073709551615) 00002000:00010000:1.0:1780298644.077452:0:19950:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f8b0380. 00000100:00100000:2.0:1780298644.078217:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.078257:0:17578:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609551488 00000100:00100000:1.0:1780298644.078264:0:17578:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609551488 00000100:00100000:1.0:1780298644.078316:0:17578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62008 00000100:00100000:1.0:1780298644.078325:0:17578:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f735880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+120:29360:x1866777609551488:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.078345:0:17578:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.078363:0:17578:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278400/0xaf35132e63034a41 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 82 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3a9a expref: -99 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.078442:0:17578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279800/0xaf35132e63034a48 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 83 type: IBT flags: 0x40000000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.078456:0:17578:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279800/0xaf35132e63034a48 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 83 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.078466:0:17578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279800/0xaf35132e63034a48 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 83 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1780298644.078491:0:17578:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 153015 00000004:00200000:1.0:1780298644.078495:0:17578:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1780298644.078514:0:17578:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279800/0xaf35132e63034a48 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 83 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.078538:0:17578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278400/0xaf35132e63034a41 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 83 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a9a expref: 122 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.078552:0:17578:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279800/0xaf35132e63034a48 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 82 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a9a expref: 121 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.078559:0:17578:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.078562:0:17578:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1780298644.078566:0:17578:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279800, rc 0) 00000100:00100000:1.0:1780298644.078598:0:17578:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f735880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+121:29360:x1866777609551488:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 275us (382us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.078607:0:17578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62008 00000100:00100000:0.0:1780298644.079786:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.079829:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609551616 00000100:00100000:1.0:1780298644.079836:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609551616 00000100:00100000:1.0:1780298644.079873:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9698 00000100:00100000:1.0:1780298644.079881:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:29360:x1866777609551616:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.079891:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.079900:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63034a4f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3aa1 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.079923:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63034a33 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a8c expref: 14 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.079933:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(1 169) 00010000:00010000:1.0:1780298644.079937:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(2 169) 00010000:00010000:1.0:1780298644.079942:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63034a4f lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [143360->151551] (req 143360->151551) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3aa1 expref: 20 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.079975:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63034a33 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a8c expref: 14 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.079988:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63034a33 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a8c expref: 14 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.080003:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2556aa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604995840:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298644.080069:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a540 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.080801:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2556aa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604995840:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298644.080822:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63034a4f lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [143360->151551] (req 143360->151551) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3aa1 expref: 20 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.080836:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278c00, rc 0) 00000100:00100000:1.0:1780298644.080868:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:29360:x1866777609551616:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 988us (1083us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.080876:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9698 00000100:00100000:0.0:1780298644.080970:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.081084:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609551744 00000100:00100000:3.0:1780298644.081092:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609551744 00010000:00010000:3.0:1780298644.081120:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63034a33 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a8c expref: 15 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.081211:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15153 00000100:00100000:2.0:1780298644.081226:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b5c850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2400:x1866777609551744:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298644.081236:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298644.081252:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63034a33 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a8c expref: 15 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.081263:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63034a33 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a8c expref: 15 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.081276:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63034a33 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a8c expref: 15 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.081294:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(2 169) 00010000:00010000:2.0:1780298644.081297:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(2 169) 00010000:00010000:2.0:1780298644.081303:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63034a4f lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 143360->151551) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3aa1 expref: 19 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.081309:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63034a4f lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 143360->151551) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3aa1 expref: 19 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.081329:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63034a4f lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 143360->151551) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3aa1 expref: 19 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.081336:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777604995968:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298644.081370:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3629a4180 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.082037:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777604995968:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298644.082055:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298644.082099:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b5c850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609551744:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 878us (1132us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.082109:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15153 00000100:00100000:3.0:1780298644.082458:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.082509:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609551872 00000100:00100000:0.0:1780298644.082514:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609551872 00002000:00100000:0.0:1780298644.082541:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (143360->151551) req@ffff9cf37f8b0700 x1866777609551872/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298644.082558:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f8b0700 with x1866777609551872 ext(143360->151551) 00002000:00010000:0.0:1780298644.082563:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f8b0700 00000100:00100000:0.0:1780298644.082587:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6245 00000100:00100000:0.0:1780298644.082595:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2399:x1866777609551872:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298644.082738:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2399:x1866777609551872:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 145us (282us total) trans 0 rc 8192/8192 00000100:00100000:0.0:1780298644.082744:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6245 00002000:00100000:0.0:1780298644.082749:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (143360->151551) req@ffff9cf37f8b0700 x1866777609551872/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/8632 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 8192/8192 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298644.082755:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f8b0700 with x1866777609551872 ext(143360->151551) 00002000:00010000:0.0:1780298644.082758:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f8b0700 00000100:00100000:1.0:1780298644.083927:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.084031:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609552000 00000100:00100000:3.0:1780298644.084037:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609552000 00000100:00100000:3.0:1780298644.084077:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9699 00000100:00100000:3.0:1780298644.084087:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556aa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2397:x1866777609552000:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.084099:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.084113:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034a56 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3aa8 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.084203:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(2 169) 00010000:00010000:2.0:1780298644.084207:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(2 169) 00010000:00010000:2.0:1780298644.084214:0:20252:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034a56 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3aa8 expref: 15 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.084227:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034a56 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3aa8 expref: 15 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.084235:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157c00, rc 0) 00000100:00100000:2.0:1780298644.084268:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556aa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2397:x1866777609552000:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 184us (346us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.084273:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9699 00000100:00100000:2.0:1780298644.085694:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.085748:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609552256 00000100:00100000:3.0:1780298644.085756:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609552256 00000100:00100000:3.0:1780298644.085804:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62009 00000100:00100000:3.0:1780298644.085814:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f735c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+121:29360:x1866777609552256:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:3.0:1780298644.085863:0:6477:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298644.085884:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279800/0xaf35132e63034a48 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 83 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a9a expref: 121 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.085898:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298644.085901:0:6477:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1780298644.086051:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157400/0xaf35132e63034a5d lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 82 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.086084:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157400/0xaf35132e63034a5d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 82 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.086094:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157400/0xaf35132e63034a5d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 82 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1780298644.086170:0:6477:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991068 00000004:00010000:2.0:1780298644.086248:0:6477:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000085d06dd reply state 00000000c1fd1ce1 transno 4294991069 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157400/0xaf35132e63034a5d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 82 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.086264:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157400/0xaf35132e63034a5d lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 82 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.086273:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298644.086276:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1780298644.086291:0:6477:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1780298644.086387:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f735c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+123:29360:x1866777609552256:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 575us (694us total) trans 4294991069 rc 0/0 00000100:00100000:2.0:1780298644.086395:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62009 00000020:00080000:2.0:1780298644.086400:0:6477:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1780298644.087210:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.087273:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609552384 00000100:00100000:1.0:1780298644.087278:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609552384 00000100:00100000:1.0:1780298644.087323:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9700 00000100:00100000:1.0:1780298644.087332:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:29360:x1866777609552384:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.087344:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.087356:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f4922155/0xaf35132e63034a64 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3aaf expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.087381:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63034a4f lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 143360->151551) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3aa1 expref: 19 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.087392:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034a56 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3aa8 expref: 16 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.087400:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(2 169) 00010000:00010000:1.0:1780298644.087404:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(2 169) 00010000:00010000:1.0:1780298644.087408:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f4922155/0xaf35132e63034a64 lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3aaf expref: 16 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.087435:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034a56 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3aa8 expref: 16 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.087446:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034a56 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3aa8 expref: 16 pid: 20252 timeout: 933 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.087457:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2556aad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604996096:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298644.087527:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63034a4f lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 143360->151551) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3aa1 expref: 19 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.087539:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63034a4f lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 143360->151551) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3aa1 expref: 19 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.087550:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2556a8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604996224:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298644.087578:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a540 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.088246:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2556aad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604996096:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298644.088352:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2556a8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604996224:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298644.088380:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f4922155/0xaf35132e63034a64 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3aaf expref: 16 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.088396:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278600, rc 0) 00000100:00100000:0.0:1780298644.088434:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:29360:x1866777609552384:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1104us (1226us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.088442:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9700 00000100:00100000:1.0:1780298644.088647:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.088676:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.088706:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609552512 00000100:00100000:0.0:1780298644.088712:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609552512 00000100:00100000:2.0:1780298644.088725:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609552640 00000100:00100000:2.0:1780298644.088737:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609552640 00010000:00010000:0.0:1780298644.088739:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034a56 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3aa8 expref: 16 pid: 20252 timeout: 933 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.088777:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15154 00010000:00010000:2.0:1780298644.088781:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63034a4f lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 143360->151551) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3aa1 expref: 20 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.088795:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609552512:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298644.088813:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298644.088835:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034a56 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3aa8 expref: 16 pid: 20252 timeout: 933 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.088844:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15155 00010000:00010000:0.0:1780298644.088866:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034a56 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3aa8 expref: 16 pid: 20252 timeout: 933 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.088870:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556aad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2399:x1866777609552640:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298644.088896:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298644.088904:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034a56 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3aa8 expref: 16 pid: 20252 timeout: 933 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.088918:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298644.088920:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63034a4f lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 143360->151551) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3aa1 expref: 20 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.088938:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63034a4f lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 143360->151551) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3aa1 expref: 20 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.088952:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2400:x1866777609552512:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 163us (302us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.088961:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15154 00010000:00010000:2.0:1780298644.088971:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63034a4f lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 143360->151551) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3aa1 expref: 20 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.088992:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(2 169) 00010000:00010000:2.0:1780298644.088997:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(2 169) 00010000:00010000:2.0:1780298644.089006:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f4922155/0xaf35132e63034a64 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3aaf expref: 14 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.089022:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f4922155/0xaf35132e63034a64 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3aaf expref: 14 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.089068:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000f4922155/0xaf35132e63034a64 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3aaf expref: 14 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.089084:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f737b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777604996352:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298644.089189:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3629a4180 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.089776:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f737b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777604996352:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298644.089793:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298644.089825:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556aad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2399:x1866777609552640:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 966us (1148us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.089836:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15155 00000100:00100000:3.0:1780298644.089976:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.090011:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609552768 00000100:00100000:0.0:1780298644.090016:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609552768 00002000:00010000:0.0:1780298644.090033:0:8364:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (1108580->18446744073709551615) 00002000:00100000:0.0:1780298644.090036:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f8b0000 with x1866777609552768 ext(1108580->18446744073709551615) 00000100:00100000:3.0:1780298644.090041:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00002000:00010000:0.0:1780298644.090046:0:8364:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f8b0000. 00000100:00100000:0.0:1780298644.090066:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609552896 00000100:00100000:0.0:1780298644.090068:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609552896 00002000:00010000:0.0:1780298644.090085:0:8364:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00002000:00100000:0.0:1780298644.090088:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f8b2300 with x1866777609552896 ext(0->18446744073709551615) 00002000:00010000:0.0:1780298644.090092:0:8364:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f8b2300. 00000100:00100000:2.0:1780298644.090099:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6246 00000100:00100000:1.0:1780298644.090118:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6247 00000100:00100000:2.0:1780298644.090119:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2399:x1866777609552768:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:1.0:1780298644.090164:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2399:x1866777609552896:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:1.0:1780298644.090275:0:16972:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:1.0:1780298644.090289:0:16972:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298644.090292:0:16972:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298644.090294:0:16972:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298644.090296:0:16972:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:1.0:1780298644.090323:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2399:x1866777609552896:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 167us (317us total) trans 4294969256 rc 0/0 00000100:00100000:1.0:1780298644.090329:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6247 00002000:00010000:1.0:1780298644.090333:0:16972:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00002000:00100000:1.0:1780298644.090335:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f8b2300 with x1866777609552896 ext(0->18446744073709551615) 00002000:00010000:1.0:1780298644.090341:0:16972:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f8b2300. 00002000:00010000:2.0:1780298644.090393:0:19951:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 153015 -> 1108580 00002000:00010000:2.0:1780298644.090398:0:19951:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298644.090400:0:19951:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298644.090402:0:19951:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298644.090405:0:19951:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 304 -> 304 00000100:00100000:2.0:1780298644.090448:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2399:x1866777609552768:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 339us (513us total) trans 4294969707 rc 0/0 00000100:00100000:2.0:1780298644.090458:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6246 00002000:00010000:2.0:1780298644.090463:0:19951:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (1108580->18446744073709551615) 00002000:00100000:2.0:1780298644.090467:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f8b0000 with x1866777609552768 ext(1108580->18446744073709551615) 00002000:00010000:2.0:1780298644.090475:0:19951:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f8b0000. 00000100:00100000:0.0:1780298644.091521:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.091578:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609553024 00000100:00100000:2.0:1780298644.091585:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609553024 00002000:00100000:2.0:1780298644.091622:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (1097728->1101823) req@ffff9cf24519c380 x1866777609553024/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.091632:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24519c380 with x1866777609553024 ext(1097728->1101823) 00002000:00010000:2.0:1780298644.091636:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24519c380 00000100:00100000:2.0:1780298644.091661:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6248 00000100:00100000:2.0:1780298644.091671:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609553024:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298644.091836:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609553024:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 166us (314us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298644.091847:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6248 00002000:00100000:2.0:1780298644.091854:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (1097728->1101823) req@ffff9cf24519c380 x1866777609553024/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/4536 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.091863:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24519c380 with x1866777609553024 ext(1097728->1101823) 00002000:00010000:2.0:1780298644.091867:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24519c380 00000100:00100000:1.0:1780298644.092886:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.092933:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609553152 00000100:00100000:2.0:1780298644.092940:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609553152 00002000:00100000:2.0:1780298644.092964:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (1101824->1105919) req@ffff9cf242b68380 x1866777609553152/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.092973:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b68380 with x1866777609553152 ext(1101824->1105919) 00002000:00010000:2.0:1780298644.092976:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf242b68380 00000100:00100000:2.0:1780298644.093022:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6249 00000100:00100000:2.0:1780298644.093036:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b68380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609553152:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298644.093204:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b68380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609553152:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 174us (319us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298644.093212:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6249 00002000:00100000:2.0:1780298644.093219:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (1101824->1105919) req@ffff9cf242b68380 x1866777609553152/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/4536 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.093227:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b68380 with x1866777609553152 ext(1101824->1105919) 00002000:00010000:2.0:1780298644.093230:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf242b68380 00000100:00100000:0.0:1780298644.094370:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.094419:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609553280 00000100:00100000:2.0:1780298644.094425:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609553280 00002000:00100000:2.0:1780298644.094450:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (1077248->1110015) req@ffff9cf24519ce00 x1866777609553280/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 504/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.094459:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24519ce00 with x1866777609553280 ext(1077248->1110015) 00002000:00010000:2.0:1780298644.094463:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24519ce00 00000100:00100000:2.0:1780298644.094488:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6250 00000100:00100000:2.0:1780298644.094497:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609553280:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298644.095378:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609553280:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 878us (1005us total) trans 0 rc 24576/24576 00000100:00100000:2.0:1780298644.095388:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6250 00002000:00100000:2.0:1780298644.095396:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (1077248->1110015) req@ffff9cf24519ce00 x1866777609553280/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 504/440 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 24576/24576 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.095405:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24519ce00 with x1866777609553280 ext(1077248->1110015) 00002000:00010000:2.0:1780298644.095409:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24519ce00 00000100:00100000:0.0:1780298644.096390:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.096433:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609553408 00000100:00100000:3.0:1780298644.096441:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609553408 00002000:00100000:3.0:1780298644.096475:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (1097728->1108579) req@ffff9cf24519f100 x1866777609553408/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 11344/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298644.096487:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24519f100 with x1866777609553408 ext(1097728->1108579) 00002000:00010000:3.0:1780298644.096492:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24519f100 00000100:00100000:3.0:1780298644.096519:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6251 00000100:00100000:3.0:1780298644.096532:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2397:x1866777609553408:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:3.0:1780298644.097036:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2397:x1866777609553408:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 507us (647us total) trans 4294969708 rc 0/0 00000100:00100000:3.0:1780298644.097045:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6251 00002000:00100000:3.0:1780298644.097052:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (1097728->1108579) req@ffff9cf24519f100 x1866777609553408/t4294969708(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 11344/448 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298644.097061:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24519f100 with x1866777609553408 ext(1097728->1108579) 00002000:00010000:3.0:1780298644.097064:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24519f100 00000100:00100000:1.0:1780298644.097869:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.097898:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609553536 00000100:00100000:3.0:1780298644.097903:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609553536 00000100:00100000:3.0:1780298644.097929:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9701 00000100:00100000:3.0:1780298644.097933:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2398:x1866777609553536:12345-192.168.203.25@tcp:16:fsx.0 00000001:00080000:1.0:1780298644.098836:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969708 is committed 00000001:00080000:1.0:1780298644.098853:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969705 is committed 00000100:00100000:3.0:1780298644.098985:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609553536:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 1050us (1116us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298644.098990:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9701 00000100:00100000:1.0:1780298644.099798:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.099833:0:7411:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609553664 00000100:00100000:2.0:1780298644.099838:0:7411:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609553664 00000100:00100000:2.0:1780298644.099867:0:7411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62010 00000100:00100000:2.0:1780298644.099873:0:7411:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+123:29360:x1866777609553664:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.099887:0:7411:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.099903:0:7411:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3582cb000/0xaf35132e63034a6b lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 83 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3ab6 expref: -99 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.099967:0:7411:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3582cbe00/0xaf35132e63034a72 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 84 type: IBT flags: 0x40000000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.099980:0:7411:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3582cbe00/0xaf35132e63034a72 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 84 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.099988:0:7411:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3582cbe00/0xaf35132e63034a72 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 84 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298644.100004:0:7411:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 1108580 00000004:00200000:2.0:1780298644.100007:0:7411:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298644.100019:0:7411:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3582cbe00/0xaf35132e63034a72 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 84 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.100036:0:7411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3582cb000/0xaf35132e63034a6b lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 84 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ab6 expref: 125 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.100049:0:7411:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3582cbe00/0xaf35132e63034a72 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 83 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ab6 expref: 124 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.100056:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298644.100058:0:7411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298644.100061:0:7411:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf3582cbe00, rc 0) 00000100:00100000:2.0:1780298644.100094:0:7411:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+124:29360:x1866777609553664:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 220us (297us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.100101:0:7411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62010 00000100:00100000:0.0:1780298644.102401:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.102440:0:6476:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609553920 00000100:00100000:2.0:1780298644.102447:0:6476:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609553920 00000100:00100000:2.0:1780298644.102485:0:6476:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62011 00000100:00100000:2.0:1780298644.102491:0:6476:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+124:29360:x1866777609553920:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:2.0:1780298644.102523:0:6476:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298644.102540:0:6476:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3582cbe00/0xaf35132e63034a72 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 84 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ab6 expref: 124 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.102552:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298644.102555:0:6476:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298644.102681:0:6476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3582cb400/0xaf35132e63034a79 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 83 type: IBT flags: 0x40000000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.102701:0:6476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3582cb400/0xaf35132e63034a79 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 83 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.102708:0:6476:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3582cb400/0xaf35132e63034a79 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 83 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1780298644.102741:0:6476:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991070 00000004:00010000:2.0:1780298644.102756:0:6476:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000316205fe reply state 0000000057c44bf2 transno 4294991071 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3582cb400/0xaf35132e63034a79 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 83 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.102766:0:6476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3582cb400/0xaf35132e63034a79 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 83 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.102773:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298644.102774:0:6476:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1780298644.102776:0:6476:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1780298644.102828:0:6476:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+126:29360:x1866777609553920:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 337us (431us total) trans 4294991071 rc 0/0 00000100:00100000:2.0:1780298644.102833:0:6476:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62011 00000020:00080000:2.0:1780298644.102837:0:6476:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1780298644.103699:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.103739:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609554048 00000100:00100000:2.0:1780298644.103743:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609554048 00002000:00010000:2.0:1780298644.103761:0:19951:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:2.0:1780298644.103766:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25548ad80 with x1866777609554048 ext(4194304->18446744073709551615) 00002000:00010000:2.0:1780298644.103776:0:19951:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf25548ad80. 00000100:00100000:2.0:1780298644.103797:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6252 00000100:00100000:2.0:1780298644.103807:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609554048:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:0.0:1780298644.103841:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.103889:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609554176 00000100:00100000:1.0:1780298644.103894:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609554176 00002000:00010000:1.0:1780298644.103910:0:8363:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (3017006->18446744073709551615) 00002000:00100000:1.0:1780298644.103913:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2548b9180 with x1866777609554176 ext(3017006->18446744073709551615) 00002000:00010000:1.0:1780298644.103918:0:8363:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2548b9180. 00002000:00010000:2.0:1780298644.103926:0:19951:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 1108580 -> 4194304 00002000:00010000:2.0:1780298644.103931:0:19951:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298644.103934:0:19951:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00000100:00100000:1.0:1780298644.103938:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6253 00002000:00010000:2.0:1780298644.103939:0:19951:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298644.103943:0:19951:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 304 -> 328 00000100:00100000:1.0:1780298644.103946:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2399:x1866777609554176:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:2.0:1780298644.103980:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609554048:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 174us (282us total) trans 4294969709 rc 0/0 00000100:00100000:2.0:1780298644.103989:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6252 00002000:00010000:2.0:1780298644.103993:0:19951:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:2.0:1780298644.103997:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25548ad80 with x1866777609554048 ext(4194304->18446744073709551615) 00002000:00010000:2.0:1780298644.104002:0:19951:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf25548ad80. 00002000:00010000:1.0:1780298644.104051:0:8363:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 0 -> 3017006 00002000:00010000:1.0:1780298644.104054:0:8363:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298644.104056:0:8363:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298644.104058:0:8363:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298644.104059:0:8363:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:1.0:1780298644.104090:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2399:x1866777609554176:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 146us (250us total) trans 4294969257 rc 0/0 00000100:00100000:1.0:1780298644.104095:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6253 00002000:00010000:1.0:1780298644.104098:0:8363:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (3017006->18446744073709551615) 00002000:00100000:1.0:1780298644.104100:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2548b9180 with x1866777609554176 ext(3017006->18446744073709551615) 00002000:00010000:1.0:1780298644.104103:0:8363:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2548b9180. 00000100:00100000:0.0:1780298644.104844:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.104886:0:17578:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609554304 00000100:00100000:1.0:1780298644.104891:0:17578:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609554304 00000100:00100000:1.0:1780298644.104926:0:17578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62012 00000100:00100000:1.0:1780298644.104936:0:17578:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243844000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+157:29360:x1866777609554304:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.104947:0:17578:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.104958:0:17578:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279200/0xaf35132e63034a80 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 84 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3abd expref: -99 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.105011:0:17578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278a00/0xaf35132e63034a87 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 85 type: IBT flags: 0x40000000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.105022:0:17578:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278a00/0xaf35132e63034a87 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 85 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.105031:0:17578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278a00/0xaf35132e63034a87 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 85 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1780298644.105050:0:17578:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 7211310 00000004:00200000:1.0:1780298644.105054:0:17578:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1780298644.105068:0:17578:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278a00/0xaf35132e63034a87 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 85 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.105088:0:17578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279200/0xaf35132e63034a80 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 85 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3abd expref: 159 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.105107:0:17578:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278a00/0xaf35132e63034a87 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 84 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3abd expref: 158 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.105116:0:17578:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.105118:0:17578:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1780298644.105121:0:17578:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278a00, rc 0) 00000100:00100000:1.0:1780298644.105184:0:17578:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243844000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+158:29360:x1866777609554304:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 250us (339us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.105192:0:17578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62012 00000100:00100000:2.0:1780298644.106037:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.106067:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609554560 00000100:00100000:2.0:1780298644.106071:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.106073:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609554560 00000100:00100000:1.0:1780298644.106096:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609554432 00000100:00100000:1.0:1780298644.106098:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609554432 00000100:00100000:0.0:1780298644.106112:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9702 00000100:00100000:2.0:1780298644.106120:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9703 00000100:00100000:2.0:1780298644.106147:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f734e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2398:x1866777609554432:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298644.106150:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f737b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2398:x1866777609554560:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.106155:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.106165:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004f507fef/0xaf35132e63034a8e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3ac4 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.106176:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.106188:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e63034a95 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3acb expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.106207:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604996480:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.106228:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255489f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604996608:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.106269:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb13c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.106269:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.106998:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604996480:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.106998:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255489f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777604996608:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.107027:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e63034a95 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3acb expref: 20 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.107040:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004f507fef/0xaf35132e63034a8e lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ac4 expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.107045:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e63034a95 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3acb expref: 20 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.107067:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004f507fef/0xaf35132e63034a8e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ac4 expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.107070:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279a00, rc 0) 00010000:00010000:2.0:1780298644.107084:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf3582ca800, rc 0) 00000100:00100000:1.0:1780298644.107108:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f737b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2398:x1866777609554560:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 988us (1071us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.107116:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f734e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2398:x1866777609554432:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 968us (1042us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.107121:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9702 00000100:00100000:2.0:1780298644.107153:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9703 00000100:00100000:0.0:1780298644.108172:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.108192:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.108207:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609554688 00000100:00100000:1.0:1780298644.108212:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609554688 00000100:00100000:2.0:1780298644.108224:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609554816 00000100:00100000:2.0:1780298644.108230:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609554816 00000100:00100000:1.0:1780298644.108260:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9704 00000100:00100000:2.0:1780298644.108273:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9705 00000100:00100000:1.0:1780298644.108274:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243844a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2397:x1866777609554688:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.108296:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298644.108297:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243847480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2398:x1866777609554816:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.108308:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63034a9c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3ad2 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.108310:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.108322:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007544ea9c/0xaf35132e63034aa3 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3ad9 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.108344:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255489f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604996736:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.108357:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604996864:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.108395:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb13c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.108396:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.109110:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255489f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777604996736:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.109153:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63034a9c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ad2 expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.109168:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63034a9c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ad2 expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.109177:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278c00, rc 0) 00000100:00100000:1.0:1780298644.109208:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243844a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2397:x1866777609554688:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 940us (1035us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.109215:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9704 00000100:00100000:2.0:1780298644.109245:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604996864:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.109262:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007544ea9c/0xaf35132e63034aa3 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ad9 expref: 20 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.109276:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007544ea9c/0xaf35132e63034aa3 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ad9 expref: 20 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.109297:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf3582cba00, rc 0) 00000100:00100000:2.0:1780298644.109321:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243847480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2398:x1866777609554816:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1040us (1128us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.109327:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9705 00000100:00100000:0.0:1780298644.110019:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.110051:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609554944 00000100:00100000:2.0:1780298644.110054:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609554944 00000100:00100000:2.0:1780298644.110082:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62013 00000100:00100000:2.0:1780298644.110087:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243846300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+126:29360:x1866777609554944:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.110098:0:6477:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.110108:0:6477:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3582cae00/0xaf35132e63034aaa lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 85 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3ae0 expref: -99 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.110181:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bec00/0xaf35132e63034ab1 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 86 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.110190:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bec00/0xaf35132e63034ab1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 86 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.110196:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bec00/0xaf35132e63034ab1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 86 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298644.110210:0:6477:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 7211310 00000004:00200000:2.0:1780298644.110213:0:6477:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298644.110225:0:6477:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bec00/0xaf35132e63034ab1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 86 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.110237:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3582cae00/0xaf35132e63034aaa lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 86 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ae0 expref: 128 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.110257:0:6477:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bec00/0xaf35132e63034ab1 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 85 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ae0 expref: 127 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.110262:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298644.110264:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298644.110266:0:6477:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf2445bec00, rc 0) 00000100:00100000:2.0:1780298644.110298:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243846300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+127:29360:x1866777609554944:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 211us (287us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.110302:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62013 00000100:00100000:3.0:1780298644.111635:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.111670:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609555072 00000100:00100000:1.0:1780298644.111677:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609555072 00000100:00100000:1.0:1780298644.111709:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9706 00000100:00100000:1.0:1780298644.111718:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2400:x1866777609555072:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.111728:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.111739:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e63034ab8 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3ae7 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.111768:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.111777:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255489f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604996992:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.111805:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609555200 00000100:00100000:2.0:1780298644.111810:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609555200 00000100:00100000:1.0:1780298644.111837:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.111843:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9707 00000100:00100000:2.0:1780298644.111853:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243845c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2400:x1866777609555200:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.111862:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.111871:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fedd83c9/0xaf35132e63034abf lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3aee expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.111897:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604997120:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.111918:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb13c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.112505:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255489f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604996992:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.112519:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e63034ab8 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ae7 expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.112528:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e63034ab8 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ae7 expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.112535:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279800, rc 0) 00000100:00100000:1.0:1780298644.112558:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2400:x1866777609555072:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 844us (927us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.112563:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9706 00000100:00100000:2.0:1780298644.112714:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604997120:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.112730:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000fedd83c9/0xaf35132e63034abf lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3aee expref: 20 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.112740:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fedd83c9/0xaf35132e63034abf lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3aee expref: 20 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.112747:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf2445be000, rc 0) 00000100:00100000:2.0:1780298644.112768:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243845c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2400:x1866777609555200:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 919us (1002us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.112774:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9707 00000100:00100000:3.0:1780298644.113539:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.113569:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609555328 00000100:00100000:3.0:1780298644.113573:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.113574:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609555328 00000100:00100000:1.0:1780298644.113602:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609555456 00000100:00100000:1.0:1780298644.113605:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609555456 00000100:00100000:1.0:1780298644.113634:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9708 00000100:00100000:2.0:1780298644.113641:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9709 00000100:00100000:1.0:1780298644.113641:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609555328:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298644.113649:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2400:x1866777609555456:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.113650:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.113656:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.113659:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e63034ac6 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3af5 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.113661:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63034acd lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3afc expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.113685:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604997376:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.113685:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255489f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604997248:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.113723:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb13c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.113723:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.114441:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604997376:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.114456:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255489f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777604997248:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.114457:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63034acd lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3afc expref: 20 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.114468:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e63034ac6 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3af5 expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.114469:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63034acd lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3afc expref: 20 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.114479:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf2445be600, rc 0) 00010000:00010000:1.0:1780298644.114480:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e63034ac6 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3af5 expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.114487:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278400, rc 0) 00000100:00100000:2.0:1780298644.114506:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2400:x1866777609555456:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 858us (934us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.114506:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609555328:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 865us (968us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.114513:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9708 00000100:00100000:2.0:1780298644.114518:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9709 00000100:00100000:0.0:1780298644.115395:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.115429:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609555584 00000100:00100000:0.0:1780298644.115430:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.115435:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609555584 00000100:00100000:1.0:1780298644.115466:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609555712 00000100:00100000:1.0:1780298644.115472:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609555712 00000100:00100000:3.0:1780298644.115473:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9710 00000100:00100000:3.0:1780298644.115485:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243846a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2397:x1866777609555584:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.115494:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.115504:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000eae09c9b/0xaf35132e63034ad4 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3b03 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.115507:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9711 00000100:00100000:1.0:1780298644.115519:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e4f0050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2398:x1866777609555712:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.115528:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298644.115536:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604997504:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.115537:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63034adb lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3b0a expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.115571:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255489f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604997632:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.115575:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.115601:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a9c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.116275:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8b2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777604997504:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.116306:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000eae09c9b/0xaf35132e63034ad4 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b03 expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.116317:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000eae09c9b/0xaf35132e63034ad4 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b03 expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.116329:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157a00, rc 0) 00000100:00100000:3.0:1780298644.116354:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243846a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2397:x1866777609555584:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 874us (960us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.116360:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9710 00000100:00100000:1.0:1780298644.116364:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255489f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604997632:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.116378:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63034adb lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b0a expref: 20 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.116389:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63034adb lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b0a expref: 20 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.116398:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279200, rc 0) 00000100:00100000:1.0:1780298644.116424:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e4f0050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2398:x1866777609555712:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 909us (995us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.116429:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9711 00000100:00100000:0.0:1780298644.117118:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.117157:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609555840 00000100:00100000:1.0:1780298644.117161:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609555840 00000100:00100000:1.0:1780298644.117183:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9712 00000100:00100000:1.0:1780298644.117188:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243844700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:29360:x1866777609555840:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.117195:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.117201:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034ae2 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3b11 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.117213:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000004b2c7ae/0xaf35132e63034a3a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a93 expref: 17 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.117223:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(4 122) 00010000:00010000:1.0:1780298644.117226:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(5 122) 00010000:00010000:1.0:1780298644.117230:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034ae2 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2691072->2695167] (req 2691072->2695167) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b11 expref: 20 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.117243:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000004b2c7ae/0xaf35132e63034a3a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a93 expref: 17 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.117256:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000004b2c7ae/0xaf35132e63034a3a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a93 expref: 17 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.117266:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604997760:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298644.117310:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.117850:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604997760:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298644.117863:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034ae2 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2691072->2695167] (req 2691072->2695167) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b11 expref: 20 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.117873:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279400, rc 0) 00000100:00100000:1.0:1780298644.117899:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243844700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:29360:x1866777609555840:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 711us (781us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.117905:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9712 00000100:00100000:2.0:1780298644.118094:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.118120:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609555968 00000100:00100000:3.0:1780298644.118147:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609555968 00010000:00010000:3.0:1780298644.118171:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000004b2c7ae/0xaf35132e63034a3a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a93 expref: 18 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.118201:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15156 00000100:00100000:3.0:1780298644.118209:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f736300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2399:x1866777609555968:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298644.118219:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298644.118229:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000004b2c7ae/0xaf35132e63034a3a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a93 expref: 18 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.118241:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000004b2c7ae/0xaf35132e63034a3a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a93 expref: 18 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.118258:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000004b2c7ae/0xaf35132e63034a3a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3a93 expref: 18 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.118269:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(5 122) 00010000:00010000:3.0:1780298644.118272:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(5 122) 00010000:00010000:3.0:1780298644.118289:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034ae2 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2691072->2695167) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b11 expref: 19 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.118299:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034ae2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2691072->2695167) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b11 expref: 19 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.118318:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034ae2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2691072->2695167) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b11 expref: 19 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.118326:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b2d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6461:1866777604997888:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298644.118353:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.118915:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8b2d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6461:1866777604997888:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298644.118934:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298644.118952:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f736300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2399:x1866777609555968:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 746us (859us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298644.118957:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15156 00000100:00100000:2.0:1780298644.119213:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.119244:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609556096 00000100:00100000:1.0:1780298644.119249:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609556096 00002000:00100000:1.0:1780298644.119274:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (2691072->2695167) req@ffff9cf37f735f80 x1866777609556096/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.119298:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f735f80 with x1866777609556096 ext(2691072->2695167) 00002000:00010000:1.0:1780298644.119303:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f735f80 00000100:00100000:1.0:1780298644.119322:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6254 00000100:00100000:1.0:1780298644.119330:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f735f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2399:x1866777609556096:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298644.119468:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f735f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2399:x1866777609556096:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 139us (254us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298644.119475:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6254 00002000:00100000:1.0:1780298644.119482:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (2691072->2695167) req@ffff9cf37f735f80 x1866777609556096/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/4536 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.119491:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f735f80 with x1866777609556096 ext(2691072->2695167) 00002000:00010000:1.0:1780298644.119494:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f735f80 00000100:00100000:2.0:1780298644.120423:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.120480:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609556224 00000100:00100000:0.0:1780298644.120485:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609556224 00002000:00100000:0.0:1780298644.120507:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x12:0x0] (2695168->2699263) req@ffff9cf37f737800 x1866777609556224/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298644.120515:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f737800 with x1866777609556224 ext(2695168->2699263) 00002000:00010000:0.0:1780298644.120519:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f737800 00000100:00100000:0.0:1780298644.120541:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6255 00000100:00100000:0.0:1780298644.120548:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f737800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2400:x1866777609556224:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298644.120639:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f737800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2400:x1866777609556224:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 91us (217us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298644.120648:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6255 00002000:00100000:0.0:1780298644.120655:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x12:0x0] (2695168->2699263) req@ffff9cf37f737800 x1866777609556224/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/4536 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298644.120665:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f737800 with x1866777609556224 ext(2695168->2699263) 00002000:00010000:0.0:1780298644.120668:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f737800 00000100:00100000:0.0:1780298644.121501:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.121535:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609556352 00000100:00100000:2.0:1780298644.121540:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609556352 00000100:00100000:2.0:1780298644.121572:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9713 00000100:00100000:2.0:1780298644.121580:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243844a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609556352:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.121590:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.121602:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007544ea9c/0xaf35132e63034ae9 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3b18 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.121645:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f737b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604998016:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.121686:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb13c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.122261:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f737b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604998016:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.122289:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007544ea9c/0xaf35132e63034ae9 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b18 expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.122303:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007544ea9c/0xaf35132e63034ae9 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b18 expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.122310:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf3582cba00, rc 0) 00000100:00100000:1.0:1780298644.122337:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243844a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609556352:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 759us (836us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.122342:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9713 00000100:00100000:3.0:1780298644.123006:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.123039:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609556480 00000100:00100000:0.0:1780298644.123043:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609556480 00000100:00100000:0.0:1780298644.123066:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9714 00000100:00100000:0.0:1780298644.123071:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609556480:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.123077:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.123084:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000441cddf1/0xaf35132e63034af0 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3b1f expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.123106:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243847480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604998144:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.123164:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.123756:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243847480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604998144:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.123768:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000441cddf1/0xaf35132e63034af0 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b1f expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.123779:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000441cddf1/0xaf35132e63034af0 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b1f expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.123786:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf400, rc 0) 00000100:00100000:3.0:1780298644.123807:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609556480:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 736us (803us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.123813:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9714 00000100:00100000:2.0:1780298644.124614:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.124644:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609556608 00000100:00100000:1.0:1780298644.124648:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609556608 00000100:00100000:1.0:1780298644.124671:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9715 00000100:00100000:1.0:1780298644.124676:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f734e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:29360:x1866777609556608:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.124684:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.124691:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e63034af7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3b26 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.124704:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034ae2 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2691072->2695167) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b11 expref: 19 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.124710:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(5 122) 00010000:00010000:1.0:1780298644.124713:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(5 122) 00010000:00010000:1.0:1780298644.124716:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e63034af7 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3043328->3047423] (req 3043328->3047423) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b26 expref: 18 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.124734:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034ae2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2691072->2695167) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b11 expref: 19 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.124743:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034ae2 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2691072->2695167) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b11 expref: 19 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.124749:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604998272:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298644.124779:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837ae40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.125333:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777604998272:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298644.125348:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e63034af7 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3043328->3047423] (req 3043328->3047423) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b26 expref: 18 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.125357:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278400, rc 0) 00000100:00100000:1.0:1780298644.125378:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f734e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:29360:x1866777609556608:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 702us (766us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.125383:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9715 00000100:00100000:0.0:1780298644.125655:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.125689:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609556736 00000100:00100000:1.0:1780298644.125692:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609556736 00010000:00010000:1.0:1780298644.125707:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034ae2 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2691072->2695167) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b11 expref: 20 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.125727:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15157 00000100:00100000:1.0:1780298644.125732:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243845c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2400:x1866777609556736:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298644.125740:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298644.125749:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034ae2 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2691072->2695167) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b11 expref: 20 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.125758:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034ae2 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2691072->2695167) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b11 expref: 20 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.125770:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034ae2 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2691072->2695167) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b11 expref: 20 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.125777:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(5 122) 00010000:00010000:1.0:1780298644.125779:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(5 122) 00010000:00010000:1.0:1780298644.125785:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e63034af7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3043328->36597759] (req 3043328->3047423) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b26 expref: 17 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.125791:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e63034af7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3043328->36597759] (req 3043328->3047423) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b26 expref: 17 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.125806:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e63034af7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3043328->36597759] (req 3043328->3047423) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b26 expref: 17 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.125812:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777604998400:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298644.125845:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837ae40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.126583:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777604998400:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298644.126593:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298644.126621:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243845c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2400:x1866777609556736:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 887us (964us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.126627:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15157 00000100:00100000:2.0:1780298644.127399:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.127431:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609556864 00000100:00100000:1.0:1780298644.127436:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609556864 00000100:00100000:1.0:1780298644.127465:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9716 00000100:00100000:1.0:1780298644.127471:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f736680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2400:x1866777609556864:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.127480:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.127487:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017dab243/0xaf35132e63034afe lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3b2d expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.127511:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604998528:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.127541:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837ae40 going to sleep for 11 seconds 00002000:00010000:1.0:1780298644.128255:0:12101:0:(ofd_lvb.c:259:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size: 3017006 -> 3046549 00002000:00010000:1.0:1780298644.128259:0:12101:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks: 0 -> 1 00000100:00100000:1.0:1780298644.128267:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604998528:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.128289:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000017dab243/0xaf35132e63034afe lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b2d expref: 19 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.128301:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017dab243/0xaf35132e63034afe lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b2d expref: 19 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.128308:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279800, rc 0) 00000100:00100000:1.0:1780298644.128332:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f736680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2400:x1866777609556864:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 862us (936us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.128337:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9716 00000100:00100000:2.0:1780298644.129252:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.129288:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609556992 00000100:00100000:1.0:1780298644.129291:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609556992 00000100:00100000:1.0:1780298644.129313:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9717 00000100:00100000:1.0:1780298644.129319:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f737b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2400:x1866777609556992:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.129326:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.129332:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63034b05 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3b34 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.129351:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604998656:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.129379:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837ae40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.129939:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777604998656:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.129951:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63034b05 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b34 expref: 19 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.129960:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63034b05 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b34 expref: 19 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.129967:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279200, rc 0) 00000100:00100000:1.0:1780298644.129988:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f737b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2400:x1866777609556992:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 670us (739us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.129993:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9717 00000100:00100000:0.0:1780298644.131624:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.131661:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609557120 00000100:00100000:3.0:1780298644.131666:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609557120 00000100:00100000:3.0:1780298644.131703:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9718 00000100:00100000:3.0:1780298644.131712:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243846a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:29360:x1866777609557120:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.131724:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.131737:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e63034b0c lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3b3b expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.131758:0:19992:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f4922155/0xaf35132e63034a64 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3aaf expref: 7 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.131767:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(2 169) 00010000:00010000:3.0:1780298644.131771:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(3 169) 00010000:00010000:3.0:1780298644.131776:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e63034b0c lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [5750784->5763071] (req 5750784->5763071) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b3b expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.131797:0:19992:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f4922155/0xaf35132e63034a64 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3aaf expref: 7 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.131808:0:19992:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f4922155/0xaf35132e63034a64 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3aaf expref: 7 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.131816:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b3100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604998784:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298644.131852:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.132550:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8b3100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777604998784:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298644.132569:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e63034b0c lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [5750784->5763071] (req 5750784->5763071) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b3b expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.132584:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156400, rc 0) 00000100:00100000:3.0:1780298644.132617:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243846a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609557120:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 907us (996us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298644.132624:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9718 00000100:00100000:2.0:1780298644.132838:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.132873:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609557248 00000100:00100000:0.0:1780298644.132877:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609557248 00010000:00010000:0.0:1780298644.132898:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000f4922155/0xaf35132e63034a64 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3aaf expref: 8 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.132931:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15158 00000100:00100000:0.0:1780298644.132941:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fa680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609557248:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298644.132951:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298644.132966:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f4922155/0xaf35132e63034a64 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3aaf expref: 8 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.132974:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f4922155/0xaf35132e63034a64 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3aaf expref: 8 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.132988:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f4922155/0xaf35132e63034a64 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3aaf expref: 8 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.132995:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(3 169) 00010000:00010000:0.0:1780298644.132998:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(3 169) 00010000:00010000:0.0:1780298644.133004:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e63034b0c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [5750784->39305215] (req 5750784->5763071) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b3b expref: 8 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.133009:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e63034b0c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [5750784->39305215] (req 5750784->5763071) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b3b expref: 8 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.133027:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e63034b0c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [5750784->39305215] (req 5750784->5763071) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b3b expref: 8 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.133033:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243847480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777604998912:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298644.133073:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.133738:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243847480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777604998912:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298644.133748:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298644.133772:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fa680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609557248:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 834us (933us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.133779:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15158 00000100:00100000:2.0:1780298644.134349:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.134381:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.134400:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609557376 00000100:00100000:1.0:1780298644.134405:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609557376 00000100:00100000:0.0:1780298644.134419:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609557504 00000100:00100000:0.0:1780298644.134423:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609557504 00000100:00100000:1.0:1780298644.134446:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9719 00000100:00100000:0.0:1780298644.134452:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9720 00000100:00100000:1.0:1780298644.134458:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481faa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609557376:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298644.134463:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2399:x1866777609557504:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.134467:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.134471:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.134476:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63034b13 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3b42 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.134481:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e63034b1a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3b49 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.134506:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604999040:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.134512:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243847480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604999168:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.134547:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837ae40 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.134547:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00002000:00010000:1.0:1780298644.135230:0:9128:0:(ofd_lvb.c:259:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size: 4194304 -> 5759720 00000100:00100000:1.0:1780298644.135240:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777604999040:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.135254:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63034b13 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b42 expref: 8 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.135260:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243847480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777604999168:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.135270:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63034b13 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b42 expref: 8 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.135290:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e63034b1a lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b49 expref: 19 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.135291:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278c00, rc 0) 00010000:00010000:0.0:1780298644.135306:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e63034b1a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b49 expref: 19 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.135315:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf200, rc 0) 00000100:00100000:1.0:1780298644.135332:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481faa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609557376:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 872us (977us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.135336:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2399:x1866777609557504:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 876us (953us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.135344:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9719 00000100:00100000:0.0:1780298644.135351:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9720 00000100:00100000:3.0:1780298644.136293:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.136316:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.136365:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609557632 00000100:00100000:0.0:1780298644.136365:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609557760 00000100:00100000:1.0:1780298644.136370:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609557632 00000100:00100000:0.0:1780298644.136371:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609557760 00000100:00100000:1.0:1780298644.136407:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9721 00000100:00100000:0.0:1780298644.136415:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9722 00000100:00100000:1.0:1780298644.136418:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609557632:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.136426:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298644.136427:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2397:x1866777609557760:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.136434:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.136435:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63034b21 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3b50 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.136442:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63034b28 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3b57 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.136460:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604999296:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.136467:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243847480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604999424:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.136501:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.136507:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837ae40 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.137204:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243847480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777604999424:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.137222:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63034b28 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b57 expref: 19 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.137229:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604999296:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.137235:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63034b28 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b57 expref: 19 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.137243:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae000, rc 0) 00010000:00010000:1.0:1780298644.137246:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63034b21 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b50 expref: 8 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.137258:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63034b21 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b50 expref: 8 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.137267:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279a00, rc 0) 00000100:00100000:0.0:1780298644.137271:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2397:x1866777609557760:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 847us (952us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.137276:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9722 00000100:00100000:1.0:1780298644.137302:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609557632:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 888us (1009us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.137307:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9721 00000100:00100000:2.0:1780298644.138311:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.138347:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609557888 00000100:00100000:3.0:1780298644.138351:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609557888 00000100:00100000:3.0:1780298644.138381:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9723 00000100:00100000:3.0:1780298644.138389:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609557888:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.138399:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.138409:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000eae09c9b/0xaf35132e63034b2f lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3b5e expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.138426:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(3 169) 00010000:00010000:3.0:1780298644.138431:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(3 169) 00010000:00010000:3.0:1780298644.138440:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000eae09c9b/0xaf35132e63034b2f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3837952->5750783] (req 3837952->3850239) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b5e expref: 10 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.138454:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000eae09c9b/0xaf35132e63034b2f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3837952->5750783] (req 3837952->3850239) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b5e expref: 10 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.138470:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157a00, rc 0) 00000100:00100000:3.0:1780298644.138498:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609557888:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 110us (188us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298644.138505:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9723 00000100:00100000:3.0:1780298644.139307:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.139336:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609558016 00000100:00100000:2.0:1780298644.139341:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609558016 00002000:00100000:2.0:1780298644.139372:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (3837952->3842047) req@ffff9cf37f8b1500 x1866777609558016/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.139383:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f8b1500 with x1866777609558016 ext(3837952->3842047) 00002000:00010000:2.0:1780298644.139388:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f8b1500 00000100:00100000:2.0:1780298644.139406:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6256 00000100:00100000:2.0:1780298644.139412:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609558016:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298644.139557:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609558016:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 145us (249us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298644.139563:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6256 00002000:00100000:2.0:1780298644.139568:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (3837952->3842047) req@ffff9cf37f8b1500 x1866777609558016/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/4536 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.139574:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f8b1500 with x1866777609558016 ext(3837952->3842047) 00002000:00010000:2.0:1780298644.139577:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f8b1500 00000100:00100000:2.0:1780298644.140338:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.140375:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609558144 00000100:00100000:0.0:1780298644.140378:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609558144 00002000:00100000:0.0:1780298644.140396:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (3846144->3850239) req@ffff9cf3481f8a80 x1866777609558144/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298644.140402:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481f8a80 with x1866777609558144 ext(3846144->3850239) 00002000:00010000:0.0:1780298644.140405:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481f8a80 00000100:00100000:0.0:1780298644.140419:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6257 00000100:00100000:0.0:1780298644.140424:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609558144:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298644.140491:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609558144:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 67us (155us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298644.140496:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6257 00002000:00100000:0.0:1780298644.140500:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (3846144->3850239) req@ffff9cf3481f8a80 x1866777609558144/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/4536 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298644.140506:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481f8a80 with x1866777609558144 ext(3846144->3850239) 00002000:00010000:0.0:1780298644.140509:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481f8a80 00000100:00100000:3.0:1780298644.141574:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.141605:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.141627:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609558272 00000100:00100000:0.0:1780298644.141633:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609558272 00000100:00100000:1.0:1780298644.141664:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609558400 00000100:00100000:1.0:1780298644.141669:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609558400 00000100:00100000:0.0:1780298644.141673:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9724 00000100:00100000:0.0:1780298644.141684:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609558272:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.141694:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298644.141706:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9725 00010000:00010000:0.0:1780298644.141707:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e63034b36 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3b65 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.141719:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2400:x1866777609558400:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.141729:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.141739:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007544ea9c/0xaf35132e63034b3d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3b6c expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.141762:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243847480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604999552:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.141775:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fb100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604999680:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.141834:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1540 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.141834:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.142553:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243847480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777604999552:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.142574:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e63034b36 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b65 expref: 8 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.142588:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e63034b36 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b65 expref: 8 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.142597:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fb100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777604999680:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.142602:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae600, rc 0) 00010000:00010000:2.0:1780298644.142621:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007544ea9c/0xaf35132e63034b3d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b6c expref: 19 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.142628:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609558272:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 947us (1088us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.142639:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9724 00010000:00010000:2.0:1780298644.142648:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007544ea9c/0xaf35132e63034b3d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b6c expref: 19 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.142661:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf3582cba00, rc 0) 00000100:00100000:2.0:1780298644.142693:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2400:x1866777609558400:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 979us (1087us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.142701:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9725 00000100:00100000:3.0:1780298644.143555:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.143591:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609558528 00000100:00100000:1.0:1780298644.143596:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609558528 00000100:00100000:1.0:1780298644.143629:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9726 00000100:00100000:1.0:1780298644.143636:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609558528:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.143643:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.143652:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e63034b44 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3b73 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.143685:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.143689:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604999808:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.143724:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609558656 00000100:00100000:0.0:1780298644.143728:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609558656 00000100:00100000:1.0:1780298644.143738:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837ae40 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.143757:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9727 00000100:00100000:0.0:1780298644.143764:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2398:x1866777609558656:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.143771:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.143780:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e63034b4b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3b7a expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.143810:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243847480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604999936:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.143842:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.144343:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777604999808:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.144355:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e63034b44 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b73 expref: 8 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.144365:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e63034b44 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b73 expref: 8 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.144371:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279800, rc 0) 00000100:00100000:1.0:1780298644.144400:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609558528:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 764us (845us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.144405:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9726 00000100:00100000:1.0:1780298644.144478:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243847480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777604999936:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.144488:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e63034b4b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b7a expref: 19 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.144496:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e63034b4b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b7a expref: 19 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.144502:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf600, rc 0) 00000100:00100000:1.0:1780298644.144523:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2398:x1866777609558656:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 761us (839us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.144528:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9727 00000100:00100000:0.0:1780298644.145305:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.145337:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.145347:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609558784 00000100:00100000:1.0:1780298644.145351:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609558784 00000100:00100000:1.0:1780298644.145376:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609558912 00000100:00100000:1.0:1780298644.145378:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609558912 00000100:00100000:2.0:1780298644.145390:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9728 00000100:00100000:2.0:1780298644.145403:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243847480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609558784:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298644.145408:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9729 00010000:00010000:2.0:1780298644.145416:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298644.145417:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481ff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2397:x1866777609558912:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.145425:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.145428:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000b8a2561/0xaf35132e63034b52 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3b81 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.145433:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63034b59 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3b88 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.145467:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605000064:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.145477:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fb100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605000192:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.145507:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837ae40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.145511:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1540 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.146153:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605000064:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.146171:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63034b59 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b88 expref: 20 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.146176:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fb100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605000192:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.146190:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63034b59 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b88 expref: 20 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.146197:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000b8a2561/0xaf35132e63034b52 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b81 expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.146198:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279200, rc 0) 00010000:00010000:2.0:1780298644.146211:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000b8a2561/0xaf35132e63034b52 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b81 expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.146219:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf3582cb000, rc 0) 00000100:00100000:1.0:1780298644.146230:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481ff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2397:x1866777609558912:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 812us (892us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.146240:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243847480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609558784:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 841us (935us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.146241:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9729 00000100:00100000:2.0:1780298644.146257:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9728 00000100:00100000:3.0:1780298644.147055:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.147092:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609559040 00000100:00100000:2.0:1780298644.147098:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609559040 00000100:00100000:2.0:1780298644.147145:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9730 00000100:00100000:2.0:1780298644.147155:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:29360:x1866777609559040:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.147164:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.147174:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b8cdae1/0xaf35132e63034b60 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3b8f expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.147189:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(5 122) 00010000:00010000:2.0:1780298644.147193:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(5 122) 00010000:00010000:2.0:1780298644.147203:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001b8cdae1/0xaf35132e63034b60 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [466944->3043327] (req 466944->479231) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b8f expref: 20 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.147218:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001b8cdae1/0xaf35132e63034b60 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [466944->3043327] (req 466944->479231) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b8f expref: 20 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.147228:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf3582cae00, rc 0) 00000100:00100000:2.0:1780298644.147257:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:29360:x1866777609559040:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 105us (203us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.147264:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9730 00000100:00100000:0.0:1780298644.147905:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.147934:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609559168 00000100:00100000:1.0:1780298644.147937:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609559168 00000100:00100000:1.0:1780298644.147966:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6258 00000100:00100000:1.0:1780298644.147971:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2400:x1866777609559168:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298644.148111:0:8363:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 3046549 -> 3017006 00002000:00010000:1.0:1780298644.148114:0:8363:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298644.148116:0:8363:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298644.148117:0:8363:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298644.148119:0:8363:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 1 -> 24 00000100:00100000:1.0:1780298644.148168:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2400:x1866777609559168:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 196us (263us total) trans 4294969258 rc 0/0 00000100:00100000:1.0:1780298644.148173:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6258 00000100:00100000:3.0:1780298644.148966:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.149040:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609559296 00000100:00100000:1.0:1780298644.149045:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609559296 00000100:00100000:3.0:1780298644.149054:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.149082:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609559424 00000100:00100000:1.0:1780298644.149085:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609559424 00000100:00100000:2.0:1780298644.149097:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9731 00000100:00100000:2.0:1780298644.149106:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609559296:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.149113:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298644.149119:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9732 00010000:00010000:2.0:1780298644.149121:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007544ea9c/0xaf35132e63034b67 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3b96 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.149155:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2399:x1866777609559424:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.149163:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298644.149170:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fb100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605000320:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.149171:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e63034b6e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3b9d expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.149201:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605000448:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.149215:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1540 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.149229:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837ae40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.149852:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fb100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605000320:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.149874:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007544ea9c/0xaf35132e63034b67 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b96 expref: 8 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.149888:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007544ea9c/0xaf35132e63034b67 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b96 expref: 8 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.149906:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf3582cba00, rc 0) 00002000:00010000:1.0:1780298644.149923:0:8356:0:(ofd_lvb.c:259:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size: 3017006 -> 3046549 00000100:00100000:1.0:1780298644.149934:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605000448:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.149945:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609559296:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 839us (978us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.149955:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9731 00010000:00010000:1.0:1780298644.149955:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e63034b6e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b9d expref: 19 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.149974:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e63034b6e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b9d expref: 19 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.149985:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279a00, rc 0) 00000100:00100000:1.0:1780298644.150003:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2399:x1866777609559424:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 854us (952us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.150008:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9732 00000100:00100000:0.0:1780298644.150906:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.150937:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609559680 00000100:00100000:2.0:1780298644.150941:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609559680 00000100:00100000:0.0:1780298644.150949:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.150973:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609559552 00000100:00100000:2.0:1780298644.150976:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609559552 00000100:00100000:1.0:1780298644.150996:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9733 00000100:00100000:2.0:1780298644.151006:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9734 00000100:00100000:1.0:1780298644.151010:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2397:x1866777609559680:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.151018:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298644.151019:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609559552:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.151026:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6416acb/0xaf35132e63034b75 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3bab expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.151029:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.151038:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004f507fef/0xaf35132e63034b7c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3ba4 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.151065:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fb100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605000704:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.151074:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605000576:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.151099:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1540 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.151109:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837ae40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.151724:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fb100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605000704:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.151740:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004f507fef/0xaf35132e63034b7c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ba4 expref: 8 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.151752:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605000576:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.151753:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004f507fef/0xaf35132e63034b7c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ba4 expref: 8 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.151767:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf3582ca800, rc 0) 00010000:00010000:1.0:1780298644.151773:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a6416acb/0xaf35132e63034b75 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3bab expref: 19 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.151784:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6416acb/0xaf35132e63034b75 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3bab expref: 19 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.151792:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278c00, rc 0) 00000100:00100000:2.0:1780298644.151793:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609559552:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 777us (844us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.151800:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9734 00000100:00100000:1.0:1780298644.151816:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2397:x1866777609559680:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 811us (910us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.151821:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9733 00000100:00100000:3.0:1780298644.152851:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.152880:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609559808 00000100:00100000:1.0:1780298644.152884:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609559808 00002000:00100000:1.0:1780298644.152904:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (2416640->2420735) req@ffff9cf2558f3100 x1866777609559808/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.152911:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2558f3100 with x1866777609559808 ext(2416640->2420735) 00002000:00010000:1.0:1780298644.152916:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2558f3100 00000100:00100000:1.0:1780298644.152932:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6259 00000100:00100000:1.0:1780298644.152937:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2398:x1866777609559808:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298644.153067:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2398:x1866777609559808:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 125us (212us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298644.153077:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6259 00002000:00100000:1.0:1780298644.153082:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (2416640->2420735) req@ffff9cf2558f3100 x1866777609559808/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/4536 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.153087:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2558f3100 with x1866777609559808 ext(2416640->2420735) 00002000:00010000:1.0:1780298644.153089:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2558f3100 00000100:00100000:0.0:1780298644.153863:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.153923:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609559936 00000100:00100000:3.0:1780298644.153927:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609559936 00000100:00100000:2.0:1780298644.153952:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.153959:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9735 00000100:00100000:3.0:1780298644.153968:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481ffb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609559936:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298644.153976:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609560064 00010000:00010000:3.0:1780298644.153977:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298644.153979:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609560064 00010000:00010000:3.0:1780298644.153986:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009c841b52/0xaf35132e63034b83 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3bb2 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.154006:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9736 00000100:00100000:0.0:1780298644.154014:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2398:x1866777609560064:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298644.154018:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f2680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605000832:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.154023:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.154031:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d4d8e3d7/0xaf35132e63034b8a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3bb9 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.154056:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605000960:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.154062:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.154091:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.154721:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f2680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605000832:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.154740:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009c841b52/0xaf35132e63034b83 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3bb2 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.154750:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009c841b52/0xaf35132e63034b83 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3bb2 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.154757:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157600, rc 0) 00000100:00100000:3.0:1780298644.154780:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481ffb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609559936:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 814us (918us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.154784:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9735 00000100:00100000:1.0:1780298644.154833:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605000960:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.154854:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d4d8e3d7/0xaf35132e63034b8a lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3bb9 expref: 22 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.154869:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d4d8e3d7/0xaf35132e63034b8a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3bb9 expref: 22 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.154879:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf000, rc 0) 00000100:00100000:1.0:1780298644.154910:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2398:x1866777609560064:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 897us (958us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.154917:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9736 00000100:00100000:2.0:1780298644.155677:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.155695:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609560192 00000100:00100000:3.0:1780298644.155699:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609560192 00000100:00100000:2.0:1780298644.155703:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.155722:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609560320 00000100:00100000:3.0:1780298644.155726:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609560320 00000100:00100000:3.0:1780298644.155751:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9737 00000100:00100000:1.0:1780298644.155759:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9738 00000100:00100000:3.0:1780298644.155761:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609560192:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.155771:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298644.155771:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2397:x1866777609560320:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.155779:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.155782:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034b91 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3bc0 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.155786:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017dab243/0xaf35132e63034b98 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3bc7 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.155820:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605001088:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.155826:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f2680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605001216:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.155858:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837ae40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.155859:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.156569:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f2680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605001216:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.156589:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034b91 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3bc0 expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.156601:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605001088:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.156613:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034b91 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3bc0 expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.156621:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157c00, rc 0) 00010000:00010000:1.0:1780298644.156625:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000017dab243/0xaf35132e63034b98 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3bc7 expref: 22 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.156639:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017dab243/0xaf35132e63034b98 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3bc7 expref: 22 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.156647:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609560192:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 888us (968us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.156655:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9737 00010000:00010000:1.0:1780298644.156655:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279800, rc 0) 00000100:00100000:1.0:1780298644.156679:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2397:x1866777609560320:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 913us (975us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.156684:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9738 00000100:00100000:0.0:1780298644.157616:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.157636:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.157653:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609560576 00000100:00100000:1.0:1780298644.157657:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609560576 00000100:00100000:3.0:1780298644.157670:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609560448 00000100:00100000:3.0:1780298644.157675:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609560448 00000100:00100000:1.0:1780298644.157685:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9739 00000100:00100000:1.0:1780298644.157694:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2398:x1866777609560576:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.157700:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.157708:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034b9f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3bd5 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.157714:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9740 00000100:00100000:3.0:1780298644.157730:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609560448:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298644.157736:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605001344:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.157739:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.157748:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e63034ba6 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3bce expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.157771:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837ae40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.157775:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f2680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605001472:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.157800:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.158474:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605001344:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.158481:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f2680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605001472:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.158490:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034b9f lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3bd5 expref: 19 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.158499:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e63034ba6 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3bce expref: 8 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.158509:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e63034ba6 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3bce expref: 8 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.158519:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034b9f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3bd5 expref: 19 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.158525:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157200, rc 0) 00010000:00010000:1.0:1780298644.158529:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279400, rc 0) 00000100:00100000:3.0:1780298644.158550:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609560448:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 825us (912us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.158552:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2398:x1866777609560576:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 860us (935us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.158563:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9740 00000100:00100000:1.0:1780298644.158570:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9739 00000100:00100000:0.0:1780298644.159389:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.159415:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609560704 00000100:00100000:2.0:1780298644.159418:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609560704 00000100:00100000:2.0:1780298644.159439:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9741 00000100:00100000:2.0:1780298644.159445:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:29360:x1866777609560704:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.159451:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.159457:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63034bad lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3bdc expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.159469:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(5 122) 00010000:00010000:2.0:1780298644.159472:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(5 122) 00010000:00010000:2.0:1780298644.159479:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63034bad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [376832->466943] (req 376832->385023) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3bdc expref: 19 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.159489:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63034bad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [376832->466943] (req 376832->385023) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3bdc expref: 19 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.159493:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf2445be600, rc 0) 00000100:00100000:2.0:1780298644.159514:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+19:29360:x1866777609560704:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 69us (126us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.159519:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9741 00000100:00100000:0.0:1780298644.160077:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.160120:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609560832 00000100:00100000:2.0:1780298644.160148:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609560832 00000100:00100000:2.0:1780298644.160182:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6260 00000100:00100000:2.0:1780298644.160192:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343965880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+19:2399:x1866777609560832:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298644.160375:0:19949:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 3046549 -> 3017006 00002000:00010000:2.0:1780298644.160379:0:19949:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298644.160382:0:19949:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298644.160384:0:19949:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298644.160386:0:19949:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 24 -> 40 00000100:00100000:2.0:1780298644.160414:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343965880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2399:x1866777609560832:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 224us (338us total) trans 4294969259 rc 0/0 00000100:00100000:2.0:1780298644.160419:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6260 00000100:00100000:3.0:1780298644.161213:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.161244:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609560960 00000100:00100000:1.0:1780298644.161249:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609560960 00000100:00100000:1.0:1780298644.161288:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9742 00000100:00100000:1.0:1780298644.161296:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609560960:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.161302:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.161310:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63034bb4 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3be3 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.161338:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605001600:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.161347:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.161380:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609561088 00000100:00100000:2.0:1780298644.161385:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609561088 00000100:00100000:1.0:1780298644.161400:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837ae40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.161410:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9743 00000100:00100000:2.0:1780298644.161418:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2400:x1866777609561088:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.161426:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.161434:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fedd83c9/0xaf35132e63034bbb lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3bea expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.161461:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605001728:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.161491:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1540 going to sleep for 11 seconds 00002000:00010000:2.0:1780298644.162023:0:20252:0:(ofd_lvb.c:259:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size: 3017006 -> 3046549 00000100:00100000:1.0:1780298644.162025:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605001600:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.162032:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605001728:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.162046:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63034bb4 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3be3 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.162050:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000fedd83c9/0xaf35132e63034bbb lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3bea expref: 22 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.162065:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63034bb4 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3be3 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.162074:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fedd83c9/0xaf35132e63034bbb lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3bea expref: 22 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.162084:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279200, rc 0) 00010000:00010000:2.0:1780298644.162087:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf2445be000, rc 0) 00000100:00100000:1.0:1780298644.162111:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609560960:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 817us (898us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.162117:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2400:x1866777609561088:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 698us (768us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.162145:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9742 00000100:00100000:2.0:1780298644.162154:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9743 00000100:00100000:3.0:1780298644.162928:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.162966:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609561216 00000100:00100000:1.0:1780298644.162970:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609561216 00000100:00100000:3.0:1780298644.162993:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.162996:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9744 00000100:00100000:1.0:1780298644.163003:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609561216:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.163010:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298644.163012:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609561344 00000100:00100000:2.0:1780298644.163016:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609561344 00010000:00010000:1.0:1780298644.163018:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007fddc766/0xaf35132e63034bc2 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3bf1 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.163042:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605001856:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.163047:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9745 00000100:00100000:2.0:1780298644.163055:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2398:x1866777609561344:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.163062:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.163069:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fbbd4b6c/0xaf35132e63034bc9 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3bf8 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.163084:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837ae40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.163091:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605001984:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.163115:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1540 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.163669:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605001856:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.163683:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605001984:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.163686:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007fddc766/0xaf35132e63034bc2 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3bf1 expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.163698:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007fddc766/0xaf35132e63034bc2 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3bf1 expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.163701:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000fbbd4b6c/0xaf35132e63034bc9 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3bf8 expref: 22 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.163710:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278000, rc 0) 00010000:00010000:2.0:1780298644.163717:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fbbd4b6c/0xaf35132e63034bc9 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3bf8 expref: 22 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.163729:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf36d473e00, rc 0) 00000100:00100000:1.0:1780298644.163736:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609561216:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 733us (807us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.163742:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9744 00000100:00100000:2.0:1780298644.163751:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2398:x1866777609561344:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 697us (758us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.163757:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9745 00000100:00100000:0.0:1780298644.164646:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.164677:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609561472 00000100:00100000:3.0:1780298644.164680:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609561472 00000100:00100000:3.0:1780298644.164702:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9746 00000100:00100000:3.0:1780298644.164709:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343965f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609561472:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.164716:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.164722:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e63034bd0 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3bff expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.164735:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.164743:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605002112:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.164765:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609561600 00000100:00100000:0.0:1780298644.164769:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609561600 00000100:00100000:0.0:1780298644.164791:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9747 00000100:00100000:0.0:1780298644.164797:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2400:x1866777609561600:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.164803:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298644.164811:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856cc0 going to sleep for 11 seconds 00010000:00010000:0.0:1780298644.164811:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63034bd7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3c06 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.164834:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343966300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605002240:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.164856:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.165399:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605002112:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.165417:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e63034bd0 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3bff expref: 8 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.165428:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e63034bd0 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3bff expref: 8 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.165434:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343966300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605002240:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.165442:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156a00, rc 0) 00010000:00010000:2.0:1780298644.165460:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63034bd7 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c06 expref: 21 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.165470:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343965f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609561472:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 760us (822us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.165475:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9746 00010000:00010000:2.0:1780298644.165479:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63034bd7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c06 expref: 21 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.165489:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf400, rc 0) 00000100:00100000:2.0:1780298644.165509:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2400:x1866777609561600:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 714us (777us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.165515:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9747 00000100:00100000:1.0:1780298644.166216:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.166261:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609561728 00000100:00100000:3.0:1780298644.166265:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609561728 00000100:00100000:3.0:1780298644.166305:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9748 00000100:00100000:3.0:1780298644.166314:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609561728:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.166323:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.166330:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e63034bde lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3c0d expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.166339:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.166351:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605002368:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.166370:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609561856 00000100:00100000:2.0:1780298644.166373:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609561856 00000100:00100000:2.0:1780298644.166397:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9749 00000100:00100000:2.0:1780298644.166404:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343964380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2400:x1866777609561856:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.166411:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.166420:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000beb94698/0xaf35132e63034be5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3c14 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.166424:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.166449:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605002496:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.166475:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1540 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.167009:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605002368:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.167025:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e63034bde lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c0d expref: 8 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.167037:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605002496:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.167038:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e63034bde lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c0d expref: 8 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.167051:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157200, rc 0) 00010000:00010000:2.0:1780298644.167063:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000beb94698/0xaf35132e63034be5 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c14 expref: 21 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.167075:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609561728:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 764us (859us total) trans 0 rc 301/301 00010000:00010000:2.0:1780298644.167077:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000beb94698/0xaf35132e63034be5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c14 expref: 21 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.167088:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9748 00010000:00010000:2.0:1780298644.167095:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf3821e6a00, rc 0) 00000100:00100000:2.0:1780298644.167117:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343964380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2400:x1866777609561856:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 714us (779us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.167142:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9749 00000100:00100000:1.0:1780298644.168005:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.168044:0:7411:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609562112 00000100:00100000:2.0:1780298644.168050:0:7411:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609562112 00000100:00100000:2.0:1780298644.168084:0:7411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62014 00000100:00100000:2.0:1780298644.168093:0:7411:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255489f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+158:29360:x1866777609562112:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:2.0:1780298644.168167:0:7411:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298644.168183:0:7411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278a00/0xaf35132e63034a87 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 86 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3abd expref: 158 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.168196:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298644.168199:0:7411:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298644.168335:0:7411:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3821e6c00/0xaf35132e63034bec lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 85 type: IBT flags: 0x40000000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.168347:0:7411:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bec00/0xaf35132e63034ab1 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 85 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ae0 expref: 126 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.168364:0:7411:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3821e6c00/0xaf35132e63034bec lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 85 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.168380:0:7411:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bec00/0xaf35132e63034ab1 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 85 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ae0 expref: 126 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.168390:0:7411:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bec00/0xaf35132e63034ab1 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 85 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ae0 expref: 126 pid: 6477 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.168398:0:7411:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f9180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7411:1866777605002624:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298644.168427:0:7411:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.168953:0:7411:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f9180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7411:1866777605002624:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298644.168965:0:7411:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3821e6c00/0xaf35132e63034bec lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 85 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.169167:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.169202:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609562240 00000100:00100000:0.0:1780298644.169206:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609562240 00010000:00010000:0.0:1780298644.169227:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bec00/0xaf35132e63034ab1 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 85 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ae0 expref: 127 pid: 6477 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.169260:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15159 00000100:00100000:0.0:1780298644.169270:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255488a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+127:2399:x1866777609562240:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298644.169289:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298644.169294:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bec00/0xaf35132e63034ab1 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 86 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ae0 expref: 127 pid: 6477 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.169305:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bec00/0xaf35132e63034ab1 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 86 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ae0 expref: 127 pid: 6477 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.169318:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bec00/0xaf35132e63034ab1 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 86 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ae0 expref: 127 pid: 6477 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.169329:0:8374:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298644.169333:0:8374:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3821e6c00/0xaf35132e63034bec lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 85 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.169344:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3821e6c00/0xaf35132e63034bec lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 85 type: IBT flags: 0x40290000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.169351:0:8374:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3821e6c00/0xaf35132e63034bec lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 85 type: IBT flags: 0x40290000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.169368:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298644.169384:0:7411:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3821e6c00/0xaf35132e63034bec lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 84 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.169390:0:7411:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3821e6c00/0xaf35132e63034bec lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 84 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.169396:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255488a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+126:2399:x1866777609562240:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 127us (226us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.169402:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15159 00000001:00080000:2.0:1780298644.169433:0:7411:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991072 00000004:00010000:2.0:1780298644.169451:0:7411:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000a60a3bcd reply state 0000000005994d35 transno 4294991073 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3821e6c00/0xaf35132e63034bec lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 84 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.169462:0:7411:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3821e6c00/0xaf35132e63034bec lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 84 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.169467:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298644.169469:0:7411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1780298644.169471:0:7411:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1780298644.169514:0:7411:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255489f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+160:29360:x1866777609562112:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1423us (1509us total) trans 4294991073 rc 0/0 00000100:00100000:2.0:1780298644.169519:0:7411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62014 00000020:00080000:2.0:1780298644.169523:0:7411:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1780298644.170103:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.170149:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609562368 00000100:00100000:2.0:1780298644.170153:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609562368 00000100:00100000:2.0:1780298644.170175:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9750 00000100:00100000:2.0:1780298644.170180:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609562368:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.170187:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.170193:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fedd83c9/0xaf35132e63034bf3 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3c1b expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.170207:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000eae09c9b/0xaf35132e63034b2f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3837952->5750783] (req 3837952->3850239) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b5e expref: 11 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.170213:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e63034b0c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [5750784->39305215] (req 5750784->5763071) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b3b expref: 11 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.170218:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 2 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(3 169) 00010000:00010000:2.0:1780298644.170221:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(5 169) 00010000:00010000:2.0:1780298644.170224:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fedd83c9/0xaf35132e63034bf3 lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c1b expref: 11 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.170235:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e63034b0c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [5750784->39305215] (req 5750784->5763071) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b3b expref: 11 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.170251:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e63034b0c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [5750784->39305215] (req 5750784->5763071) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b3b expref: 11 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.170265:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605002752:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298644.170318:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000eae09c9b/0xaf35132e63034b2f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3837952->5750783] (req 3837952->3850239) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b5e expref: 11 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.170330:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000eae09c9b/0xaf35132e63034b2f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3837952->5750783] (req 3837952->3850239) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b5e expref: 11 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.170337:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fbb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605002880:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298644.170355:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1540 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.170752:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605002752:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298644.170896:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fbb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605002880:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298644.170915:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000fedd83c9/0xaf35132e63034bf3 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c1b expref: 11 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.170926:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf2445be000, rc 0) 00000100:00100000:2.0:1780298644.170954:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:29360:x1866777609562368:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 773us (850us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.170960:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9750 00000100:00100000:3.0:1780298644.171119:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.171167:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609562496 00000100:00100000:2.0:1780298644.171172:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609562496 00002000:00100000:2.0:1780298644.171194:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (3837952->5759719) req@ffff9cf2558f1f80 x1866777609562496/t0(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 504/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.171201:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2558f1f80 with x1866777609562496 ext(3837952->5759719) 00010000:00010000:2.0:1780298644.171207:0:19928:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000eae09c9b/0xaf35132e63034b2f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3837952->5750783] (req 3837952->3850239) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b5e expref: 11 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.171214:0:19928:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000eae09c9b/0xaf35132e63034b2f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3837952->5750783] (req 3837952->3850239) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b5e expref: 11 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.171219:0:19928:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e63034b0c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [5750784->39305215] (req 5750784->5763071) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b3b expref: 11 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.171224:0:19928:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e63034b0c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [5750784->39305215] (req 5750784->5763071) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b3b expref: 11 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1780298644.171228:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 2 locks timeout for req ffff9cf2558f1f80 00000100:00100000:2.0:1780298644.171244:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 202 00000100:00100000:2.0:1780298644.171253:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609562496:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:2.0:1780298644.172277:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609562496:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 1025us (1157us total) trans 4294969710 rc 0/0 00000100:00100000:2.0:1780298644.172295:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 202 00002000:00100000:2.0:1780298644.172303:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (3837952->5759719) req@ffff9cf2558f1f80 x1866777609562496/t4294969710(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 504/448 e 0 to 0 dl 1780298655 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.172313:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2558f1f80 with x1866777609562496 ext(3837952->5759719) 00010000:00010000:2.0:1780298644.172320:0:19928:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000eae09c9b/0xaf35132e63034b2f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3837952->5750783] (req 3837952->3850239) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b5e expref: 13 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.172332:0:19928:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000eae09c9b/0xaf35132e63034b2f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3837952->5750783] (req 3837952->3850239) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b5e expref: 13 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.172341:0:19928:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e63034b0c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [5750784->39305215] (req 5750784->5763071) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b3b expref: 13 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.172349:0:19928:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e63034b0c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [5750784->39305215] (req 5750784->5763071) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b3b expref: 13 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1780298644.172353:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 2 locks timeout for req ffff9cf2558f1f80 00000100:00100000:0.0:1780298644.172970:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.172989:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.173003:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609562752 00000100:00100000:2.0:1780298644.173007:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609562752 00000100:00100000:1.0:1780298644.173029:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609562624 00010000:00010000:2.0:1780298644.173031:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000eae09c9b/0xaf35132e63034b2f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3837952->5750783] (req 3837952->3850239) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b5e expref: 13 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.173033:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609562624 00010000:00010000:1.0:1780298644.173052:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e63034b0c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [5750784->39305215] (req 5750784->5763071) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b3b expref: 14 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.173055:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15160 00000100:00100000:2.0:1780298644.173064:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b5c050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609562752:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298644.173071:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:2.0:1780298644.173077:0:6463:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 328 -> 376 00010000:00010000:2.0:1780298644.173082:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000eae09c9b/0xaf35132e63034b2f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3837952->5750783] (req 3837952->3850239) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b5e expref: 14 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.173090:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000eae09c9b/0xaf35132e63034b2f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3837952->5750783] (req 3837952->3850239) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b5e expref: 14 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.173097:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15161 00010000:00010000:2.0:1780298644.173104:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000eae09c9b/0xaf35132e63034b2f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3837952->5750783] (req 3837952->3850239) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b5e expref: 14 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.173110:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343964a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2397:x1866777609562624:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298644.173115:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298644.173120:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298644.173155:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e63034b0c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [5750784->39305215] (req 5750784->5763071) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b3b expref: 13 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.173169:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e63034b0c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [5750784->39305215] (req 5750784->5763071) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b3b expref: 13 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.173171:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b5c050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609562752:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 106us (199us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.173188:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15160 00010000:00010000:1.0:1780298644.173194:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e63034b0c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [5750784->39305215] (req 5750784->5763071) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b3b expref: 13 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.173203:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(5 169) 00010000:00010000:1.0:1780298644.173206:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(5 169) 00010000:00010000:1.0:1780298644.173211:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000fedd83c9/0xaf35132e63034bf3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c1b expref: 11 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.173218:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fedd83c9/0xaf35132e63034bf3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c1b expref: 11 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.173237:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000fedd83c9/0xaf35132e63034bf3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c1b expref: 11 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.173244:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548b480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777605003008:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298644.173296:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837ae40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.173849:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548b480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777605003008:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298644.173857:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298644.173876:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343964a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609562624:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 773us (886us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.173880:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15161 00000100:00100000:3.0:1780298644.173953:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.173989:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609562880 00000100:00100000:2.0:1780298644.173992:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609562880 00000100:00100000:2.0:1780298644.174014:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9751 00000100:00100000:2.0:1780298644.174020:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:29360:x1866777609562880:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.174028:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.174038:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fbbd4b6c/0xaf35132e63034bfa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3c22 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.174057:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63034bad lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [376832->466943] (req 376832->385023) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3bdc expref: 19 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.174066:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000001b8cdae1/0xaf35132e63034b60 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [466944->3043327] (req 466944->479231) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b8f expref: 22 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.174074:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e63034af7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [3043328->36597759] (req 3043328->3047423) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b26 expref: 19 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.174081:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 3 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(5 122) 00010000:00010000:2.0:1780298644.174084:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(8 122) 00010000:00010000:2.0:1780298644.174088:0:20252:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fbbd4b6c/0xaf35132e63034bfa lrc: 6/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c22 expref: 22 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.174108:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e63034af7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [3043328->36597759] (req 3043328->3047423) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b26 expref: 19 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.174116:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e63034af7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [3043328->36597759] (req 3043328->3047423) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b26 expref: 19 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.174151:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605003136:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298644.174200:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000001b8cdae1/0xaf35132e63034b60 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [466944->3043327] (req 466944->479231) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b8f expref: 22 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.174207:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000001b8cdae1/0xaf35132e63034b60 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [466944->3043327] (req 466944->479231) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b8f expref: 22 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.174214:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fbb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605003264:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298644.174236:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63034bad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [376832->466943] (req 376832->385023) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3bdc expref: 19 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.174241:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63034bad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [376832->466943] (req 376832->385023) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3bdc expref: 19 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.174251:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605003392:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298644.174290:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb16c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.174683:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605003136:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298644.174900:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fbb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605003264:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298644.174972:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605003392:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298644.174990:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000fbbd4b6c/0xaf35132e63034bfa lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c22 expref: 22 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.175001:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf36d473e00, rc 0) 00000100:00100000:2.0:1780298644.175027:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:29360:x1866777609562880:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1006us (1074us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.175035:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9751 00000100:00100000:1.0:1780298644.175257:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.175268:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.175297:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609563136 00000100:00100000:3.0:1780298644.175302:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609563136 00000100:00100000:2.0:1780298644.175311:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609563008 00000100:00100000:2.0:1780298644.175317:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609563008 00002000:00100000:2.0:1780298644.175336:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x12:0x0] (3043328->3046548) req@ffff9cf343964000 x1866777609563008/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 3712/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.175347:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf343964000 with x1866777609563008 ext(3043328->3046548) 00010000:00010000:2.0:1780298644.175353:0:19943:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e63034af7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [3043328->36597759] (req 3043328->3047423) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b26 expref: 20 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.175364:0:19943:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e63034af7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [3043328->36597759] (req 3043328->3047423) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b26 expref: 20 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.175371:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000001b8cdae1/0xaf35132e63034b60 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [466944->3043327] (req 466944->479231) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b8f expref: 22 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1780298644.175392:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9cf343964000 00000100:00100000:0.0:1780298644.175403:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.175419:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609563264 00000100:00100000:2.0:1780298644.175423:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 203 00000100:00100000:1.0:1780298644.175430:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15162 00000100:00100000:2.0:1780298644.175435:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343964000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2399:x1866777609563008:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:1.0:1780298644.175446:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:2399:x1866777609563136:12345-192.168.203.25@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1780298644.175452:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609563264 00010000:00010000:1.0:1780298644.175458:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298644.175469:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63034bad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [376832->466943] (req 376832->385023) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3bdc expref: 21 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.175471:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000001b8cdae1/0xaf35132e63034b60 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [466944->3043327] (req 466944->479231) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b8f expref: 22 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.175495:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000001b8cdae1/0xaf35132e63034b60 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [466944->3043327] (req 466944->479231) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b8f expref: 22 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.175509:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15163 00010000:00010000:1.0:1780298644.175513:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b8cdae1/0xaf35132e63034b60 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [466944->3043327] (req 466944->479231) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b8f expref: 22 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.175520:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343966300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2400:x1866777609563264:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298644.175524:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1780298644.175528:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298644.175536:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63034bad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [376832->466943] (req 376832->385023) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3bdc expref: 21 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.175545:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63034bad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [376832->466943] (req 376832->385023) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3bdc expref: 21 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.175553:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2399:x1866777609563136:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 113us (298us total) trans 0 rc 0/0 00010000:00010000:3.0:1780298644.175558:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000019a622b7/0xaf35132e63034bad lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [376832->466943] (req 376832->385023) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3bdc expref: 21 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.175564:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15162 00010000:00010000:3.0:1780298644.175573:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298644.175593:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343966300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2400:x1866777609563264:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 76us (193us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298644.175599:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15163 00000100:00100000:2.0:1780298644.175749:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343964000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2399:x1866777609563008:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 319us (482us total) trans 4294969260 rc 0/0 00000100:00100000:2.0:1780298644.175756:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 203 00002000:00100000:2.0:1780298644.175761:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x12:0x0] (3043328->3046548) req@ffff9cf343964000 x1866777609563008/t4294969260(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 3712/448 e 0 to 0 dl 1780298655 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.175767:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf343964000 with x1866777609563008 ext(3043328->3046548) 00010000:00010000:2.0:1780298644.175771:0:19943:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e63034af7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3043328->36597759] (req 3043328->3047423) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b26 expref: 21 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.175782:0:19943:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e63034af7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3043328->36597759] (req 3043328->3047423) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b26 expref: 21 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1780298644.175790:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9cf343964000 00000100:00100000:0.0:1780298644.176309:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.176339:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609563392 00000100:00100000:2.0:1780298644.176344:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609563392 00010000:00010000:2.0:1780298644.176360:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e63034af7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3043328->36597759] (req 3043328->3047423) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b26 expref: 21 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.176383:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15164 00000100:00100000:2.0:1780298644.176391:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56dc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2400:x1866777609563392:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298644.176400:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:2.0:1780298644.176405:0:6463:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 40 -> 48 00010000:00010000:2.0:1780298644.176408:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e63034af7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3043328->36597759] (req 3043328->3047423) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b26 expref: 21 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.176415:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e63034af7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3043328->36597759] (req 3043328->3047423) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b26 expref: 21 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.176423:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e63034af7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3043328->36597759] (req 3043328->3047423) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3b26 expref: 21 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.176430:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(8 122) 00010000:00010000:2.0:1780298644.176432:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(8 122) 00010000:00010000:2.0:1780298644.176435:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000fbbd4b6c/0xaf35132e63034bfa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c22 expref: 20 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.176441:0:6463:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fbbd4b6c/0xaf35132e63034bfa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c22 expref: 20 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.176453:0:6463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000fbbd4b6c/0xaf35132e63034bfa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c22 expref: 20 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.176460:0:6463:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f9180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777605003520:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298644.176487:0:6463:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb16c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.177088:0:6463:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f9180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777605003520:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298644.177096:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298644.177113:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56dc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2400:x1866777609563392:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 725us (805us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.177119:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15164 00000100:00100000:0.0:1780298644.177335:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.177361:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.177377:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609563648 00000100:00100000:1.0:1780298644.177382:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609563648 00000100:00100000:2.0:1780298644.177393:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609563520 00000100:00100000:2.0:1780298644.177397:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609563520 00002000:00010000:1.0:1780298644.177400:0:8363:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (2920145->18446744073709551615) 00002000:00100000:1.0:1780298644.177405:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e56df80 with x1866777609563648 ext(2920145->18446744073709551615) 00002000:00010000:2.0:1780298644.177408:0:16972:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00010000:1.0:1780298644.177411:0:8363:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf36e56df80. 00002000:00100000:2.0:1780298644.177412:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e56f800 with x1866777609563520 ext(4194304->18446744073709551615) 00002000:00010000:2.0:1780298644.177419:0:16972:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf36e56f800. 00000100:00100000:1.0:1780298644.177445:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6261 00000100:00100000:2.0:1780298644.177452:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6262 00000100:00100000:1.0:1780298644.177455:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2399:x1866777609563648:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:2.0:1780298644.177462:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609563520:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:2.0:1780298644.177602:0:16972:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 5759720 -> 4194304 00002000:00010000:2.0:1780298644.177604:0:16972:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298644.177606:0:16972:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298644.177608:0:16972:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298644.177608:0:8363:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 3046549 -> 2920145 00002000:00010000:2.0:1780298644.177609:0:16972:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 376 -> 352 00002000:00010000:1.0:1780298644.177614:0:8363:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298644.177616:0:8363:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298644.177618:0:8363:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298644.177620:0:8363:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 48 -> 40 00000100:00100000:2.0:1780298644.177636:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609563520:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 179us (276us total) trans 4294969711 rc 0/0 00000100:00100000:2.0:1780298644.177642:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6262 00002000:00010000:2.0:1780298644.177645:0:16972:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:2.0:1780298644.177647:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e56f800 with x1866777609563520 ext(4194304->18446744073709551615) 00000100:00100000:1.0:1780298644.177648:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:2399:x1866777609563648:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 189us (306us total) trans 4294969261 rc 0/0 00002000:00010000:2.0:1780298644.177651:0:16972:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf36e56f800. 00000100:00100000:1.0:1780298644.177656:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6261 00002000:00010000:1.0:1780298644.177660:0:8363:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (2920145->18446744073709551615) 00002000:00100000:1.0:1780298644.177662:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e56df80 with x1866777609563648 ext(2920145->18446744073709551615) 00002000:00010000:1.0:1780298644.177668:0:8363:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf36e56df80. 00000100:00100000:0.0:1780298644.178472:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.178499:0:6476:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609563776 00000100:00100000:2.0:1780298644.178503:0:6476:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609563776 00000100:00100000:2.0:1780298644.178527:0:6476:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62015 00000100:00100000:2.0:1780298644.178533:0:6476:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+160:29360:x1866777609563776:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.178543:0:6476:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.178552:0:6476:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3821e6a00/0xaf35132e63034c01 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 85 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3c29 expref: -99 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.178607:0:6476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09600/0xaf35132e63034c08 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 86 type: IBT flags: 0x40000000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.178617:0:6476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09600/0xaf35132e63034c08 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 86 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.178624:0:6476:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09600/0xaf35132e63034c08 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 86 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298644.178638:0:6476:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 7114449 00000004:00200000:2.0:1780298644.178641:0:6476:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298644.178652:0:6476:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09600/0xaf35132e63034c08 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 86 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.178667:0:6476:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3821e6a00/0xaf35132e63034c01 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 86 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c29 expref: 162 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.178676:0:6476:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09600/0xaf35132e63034c08 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 85 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c29 expref: 161 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.178681:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298644.178683:0:6476:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298644.178685:0:6476:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d09600, rc 0) 00000100:00100000:2.0:1780298644.178705:0:6476:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+161:29360:x1866777609563776:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 173us (234us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.178709:0:6476:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62015 00000100:00100000:0.0:1780298644.180034:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.180064:0:17578:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609564032 00000100:00100000:1.0:1780298644.180070:0:17578:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609564032 00000100:00100000:1.0:1780298644.180097:0:17578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62016 00000100:00100000:1.0:1780298644.180105:0:17578:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+161:29360:x1866777609564032:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:1.0:1780298644.180145:0:17578:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298644.180157:0:17578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09600/0xaf35132e63034c08 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 86 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c29 expref: 161 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.180170:0:17578:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.180173:0:17578:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298644.180295:0:17578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278200/0xaf35132e63034c0f lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 85 type: IBT flags: 0x40000000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.180312:0:17578:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278200/0xaf35132e63034c0f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 85 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.180317:0:17578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278200/0xaf35132e63034c0f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 85 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1780298644.180350:0:17578:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991074 00000004:00010000:1.0:1780298644.180370:0:17578:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 000000005cd12096 reply state 00000000d0abac3b transno 4294991075 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278200/0xaf35132e63034c0f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 85 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.180381:0:17578:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278200/0xaf35132e63034c0f lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 85 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.180387:0:17578:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.180390:0:17578:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1780298644.180392:0:17578:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1780298644.180441:0:17578:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+163:29360:x1866777609564032:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 338us (407us total) trans 4294991075 rc 0/0 00000100:00100000:1.0:1780298644.180446:0:17578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62016 00000020:00080000:1.0:1780298644.180449:0:17578:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1780298644.181116:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.181167:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609564160 00000100:00100000:2.0:1780298644.181173:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609564160 00002000:00010000:2.0:1780298644.181190:0:19949:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (2521941->18446744073709551615) 00002000:00100000:2.0:1780298644.181193:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e56ea00 with x1866777609564160 ext(2521941->18446744073709551615) 00002000:00010000:2.0:1780298644.181198:0:19949:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf36e56ea00. 00000100:00100000:2.0:1780298644.181217:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6263 00000100:00100000:2.0:1780298644.181224:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609564160:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:0.0:1780298644.181265:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.181315:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609564288 00000100:00100000:3.0:1780298644.181319:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609564288 00002000:00010000:3.0:1780298644.181331:0:19928:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00002000:00100000:3.0:1780298644.181335:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e56d880 with x1866777609564288 ext(0->18446744073709551615) 00002000:00010000:3.0:1780298644.181340:0:19928:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf36e56d880. 00000100:00100000:3.0:1780298644.181356:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6264 00000100:00100000:3.0:1780298644.181363:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:2398:x1866777609564288:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:2.0:1780298644.181378:0:19949:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 2521941 00002000:00010000:2.0:1780298644.181381:0:19949:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298644.181383:0:19949:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298644.181385:0:19949:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298644.181386:0:19949:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 352 -> 328 00000100:00100000:2.0:1780298644.181413:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609564160:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 189us (295us total) trans 4294969712 rc 0/0 00000100:00100000:2.0:1780298644.181418:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6263 00002000:00010000:2.0:1780298644.181421:0:19949:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (2521941->18446744073709551615) 00002000:00100000:2.0:1780298644.181424:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e56ea00 with x1866777609564160 ext(2521941->18446744073709551615) 00002000:00010000:2.0:1780298644.181427:0:19949:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf36e56ea00. 00002000:00010000:3.0:1780298644.181478:0:19928:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 2920145 -> 0 00002000:00010000:3.0:1780298644.181481:0:19928:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:3.0:1780298644.181482:0:19928:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:3.0:1780298644.181484:0:19928:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:3.0:1780298644.181485:0:19928:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 40 -> 0 00000100:00100000:3.0:1780298644.181543:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2398:x1866777609564288:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 179us (278us total) trans 4294969262 rc 0/0 00000100:00100000:3.0:1780298644.181552:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6264 00002000:00010000:3.0:1780298644.181555:0:19928:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00002000:00100000:3.0:1780298644.181559:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e56d880 with x1866777609564288 ext(0->18446744073709551615) 00002000:00010000:3.0:1780298644.181564:0:19928:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf36e56d880. 00000100:00100000:0.0:1780298644.182245:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.182287:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609564416 00000100:00100000:2.0:1780298644.182291:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609564416 00000100:00100000:2.0:1780298644.182325:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62017 00000100:00100000:2.0:1780298644.182333:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+163:29360:x1866777609564416:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.182344:0:6477:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.182354:0:6477:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08600/0xaf35132e63034c16 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 86 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3c30 expref: -99 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.182406:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08a00/0xaf35132e63034c1d lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 87 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.182416:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08a00/0xaf35132e63034c1d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 87 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.182424:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08a00/0xaf35132e63034c1d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 87 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298644.182440:0:6477:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 2521941 00000004:00200000:2.0:1780298644.182444:0:6477:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298644.182455:0:6477:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08a00/0xaf35132e63034c1d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 87 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.182474:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08600/0xaf35132e63034c16 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 87 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c30 expref: 165 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.182485:0:6477:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08a00/0xaf35132e63034c1d lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 86 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c30 expref: 164 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.182490:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298644.182492:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298644.182494:0:6477:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d08a00, rc 0) 00000100:00100000:2.0:1780298644.182516:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+164:29360:x1866777609564416:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 186us (273us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.182520:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62017 00000100:00100000:0.0:1780298644.184111:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.184161:0:7411:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609564672 00000100:00100000:2.0:1780298644.184166:0:7411:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609564672 00000100:00100000:2.0:1780298644.184193:0:7411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62018 00000100:00100000:2.0:1780298644.184199:0:7411:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25488a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+164:29360:x1866777609564672:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:2.0:1780298644.184219:0:7411:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298644.184229:0:7411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08a00/0xaf35132e63034c1d lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 87 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c30 expref: 164 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.184237:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298644.184239:0:7411:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298644.184336:0:7411:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08200/0xaf35132e63034c24 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 86 type: IBT flags: 0x40000000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.184354:0:7411:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08200/0xaf35132e63034c24 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 86 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.184360:0:7411:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08200/0xaf35132e63034c24 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 86 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1780298644.184386:0:7411:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991076 00000004:00010000:2.0:1780298644.184400:0:7411:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 000000001d94b8ff reply state 0000000046e972cf transno 4294991077 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08200/0xaf35132e63034c24 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 86 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.184407:0:7411:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08200/0xaf35132e63034c24 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 86 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.184411:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298644.184413:0:7411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1780298644.184414:0:7411:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1780298644.184450:0:7411:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25488a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+166:29360:x1866777609564672:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 251us (340us total) trans 4294991077 rc 0/0 00000100:00100000:2.0:1780298644.184458:0:7411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62018 00000020:00080000:2.0:1780298644.184462:0:7411:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1780298644.185216:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.185245:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609564800 00000100:00100000:0.0:1780298644.185248:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.185249:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609564800 00002000:00010000:2.0:1780298644.185265:0:19943:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:2.0:1780298644.185268:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25488a680 with x1866777609564800 ext(4194304->18446744073709551615) 00002000:00010000:2.0:1780298644.185273:0:19943:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf25488a680. 00000100:00100000:3.0:1780298644.185290:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609564928 00000100:00100000:3.0:1780298644.185296:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609564928 00000100:00100000:2.0:1780298644.185304:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6265 00002000:00010000:3.0:1780298644.185310:0:19950:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (82019->18446744073709551615) 00000100:00100000:2.0:1780298644.185313:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25488a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609564800:12345-192.168.203.25@tcp:10:fsx.0 00002000:00100000:3.0:1780298644.185315:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25488aa00 with x1866777609564928 ext(82019->18446744073709551615) 00002000:00010000:3.0:1780298644.185321:0:19950:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf25488aa00. 00000100:00100000:3.0:1780298644.185342:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6266 00000100:00100000:3.0:1780298644.185349:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25488aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2399:x1866777609564928:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:2.0:1780298644.185392:0:19943:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 2521941 -> 4194304 00002000:00010000:2.0:1780298644.185395:0:19943:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298644.185397:0:19943:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298644.185399:0:19943:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298644.185400:0:19943:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 328 -> 328 00000100:00100000:2.0:1780298644.185425:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25488a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609564800:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 114us (210us total) trans 4294969713 rc 0/0 00000100:00100000:2.0:1780298644.185430:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6265 00002000:00010000:3.0:1780298644.185432:0:19950:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 0 -> 82019 00002000:00010000:2.0:1780298644.185432:0:19943:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00010000:3.0:1780298644.185437:0:19950:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00100000:2.0:1780298644.185437:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25488a680 with x1866777609564800 ext(4194304->18446744073709551615) 00002000:00010000:3.0:1780298644.185442:0:19950:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:3.0:1780298644.185444:0:19950:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298644.185444:0:19943:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf25488a680. 00002000:00010000:3.0:1780298644.185449:0:19950:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:3.0:1780298644.185476:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25488aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+24:2399:x1866777609564928:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 128us (229us total) trans 4294969263 rc 0/0 00000100:00100000:3.0:1780298644.185484:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6266 00002000:00010000:3.0:1780298644.185487:0:19950:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (82019->18446744073709551615) 00002000:00100000:3.0:1780298644.185491:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25488aa00 with x1866777609564928 ext(82019->18446744073709551615) 00002000:00010000:3.0:1780298644.185496:0:19950:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf25488aa00. 00000100:00100000:0.0:1780298644.186694:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.186736:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609565056 00000100:00100000:1.0:1780298644.186741:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609565056 00002000:00100000:1.0:1780298644.186763:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x12:0x0] (81920->86015) req@ffff9cf25488b800 x1866777609565056/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.186770:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25488b800 with x1866777609565056 ext(81920->86015) 00002000:00010000:1.0:1780298644.186774:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf25488b800 00000100:00100000:1.0:1780298644.186790:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6267 00000100:00100000:1.0:1780298644.186796:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25488b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+24:2400:x1866777609565056:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298644.186926:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25488b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+24:2400:x1866777609565056:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 129us (232us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298644.186935:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6267 00002000:00100000:1.0:1780298644.186942:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x12:0x0] (81920->86015) req@ffff9cf25488b800 x1866777609565056/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/4536 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.186951:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25488b800 with x1866777609565056 ext(81920->86015) 00002000:00010000:1.0:1780298644.186955:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf25488b800 00000100:00100000:0.0:1780298644.188076:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.188108:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609565184 00000100:00100000:2.0:1780298644.188112:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609565184 00002000:00100000:2.0:1780298644.188141:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (77824->81919) req@ffff9cf25488ad80 x1866777609565184/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.188148:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25488ad80 with x1866777609565184 ext(77824->81919) 00002000:00010000:2.0:1780298644.188151:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf25488ad80 00000100:00100000:2.0:1780298644.188165:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6268 00000100:00100000:2.0:1780298644.188171:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25488ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+24:2397:x1866777609565184:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298644.188239:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25488ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+24:2397:x1866777609565184:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 69us (166us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298644.188244:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6268 00002000:00100000:2.0:1780298644.188248:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (77824->81919) req@ffff9cf25488ad80 x1866777609565184/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/4536 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.188254:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25488ad80 with x1866777609565184 ext(77824->81919) 00002000:00010000:2.0:1780298644.188257:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf25488ad80 00000100:00100000:0.0:1780298644.189111:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.189197:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609565312 00000100:00100000:3.0:1780298644.189203:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609565312 00002000:00100000:3.0:1780298644.189224:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x12:0x0] (77824->82018) req@ffff9cf254888a80 x1866777609565312/t0(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 4688/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298644.189232:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf254888a80 with x1866777609565312 ext(77824->82018) 00002000:00010000:3.0:1780298644.189235:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf254888a80 00000100:00100000:3.0:1780298644.189254:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6269 00000100:00100000:3.0:1780298644.189262:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254888a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+24:2398:x1866777609565312:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:3.0:1780298644.189729:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254888a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+26:2398:x1866777609565312:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 467us (618us total) trans 4294969264 rc 0/0 00000100:00100000:3.0:1780298644.189737:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6269 00002000:00100000:3.0:1780298644.189744:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x12:0x0] (77824->82018) req@ffff9cf254888a80 x1866777609565312/t4294969264(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 4688/448 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298644.189753:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf254888a80 with x1866777609565312 ext(77824->82018) 00002000:00010000:3.0:1780298644.189755:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf254888a80 00000100:00100000:0.0:1780298644.190438:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.190487:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609565440 00000100:00100000:1.0:1780298644.190492:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609565440 00000100:00100000:1.0:1780298644.190526:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9752 00000100:00100000:1.0:1780298644.190535:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254889c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+26:2398:x1866777609565440:12345-192.168.203.25@tcp:16:fsx.0 00000001:00080000:0.0:1780298644.191225:0:8351:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0000: transno 4294969264 is committed 00000001:00080000:0.0:1780298644.191240:0:8351:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0000: transno 4294969260 is committed 00000100:00100000:1.0:1780298644.191369:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254889c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609565440:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 834us (930us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.191377:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9752 00000100:00100000:0.0:1780298644.192118:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.192170:0:6476:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609565568 00000100:00100000:2.0:1780298644.192175:0:6476:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609565568 00000100:00100000:2.0:1780298644.192205:0:6476:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62019 00000100:00100000:2.0:1780298644.192211:0:6476:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254889500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+166:29360:x1866777609565568:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.192223:0:6476:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.192238:0:6476:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08600/0xaf35132e63034c2b lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 87 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3c37 expref: -99 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.192331:0:6476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09e00/0xaf35132e63034c32 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 88 type: IBT flags: 0x40000000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.192343:0:6476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09e00/0xaf35132e63034c32 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 88 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.192350:0:6476:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09e00/0xaf35132e63034c32 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 88 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298644.192368:0:6476:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 4276323 00000004:00200000:2.0:1780298644.192371:0:6476:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298644.192383:0:6476:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09e00/0xaf35132e63034c32 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 88 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.192400:0:6476:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08600/0xaf35132e63034c2b lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 88 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c37 expref: 168 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.192412:0:6476:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09e00/0xaf35132e63034c32 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 87 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c37 expref: 167 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.192418:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298644.192420:0:6476:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298644.192422:0:6476:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d09e00, rc 0) 00000100:00100000:2.0:1780298644.192445:0:6476:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254889500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+167:29360:x1866777609565568:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 236us (330us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.192451:0:6476:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62019 00000100:00100000:0.0:1780298644.193615:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.193682:0:17578:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609565696 00000100:00100000:1.0:1780298644.193692:0:17578:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609565696 00000100:00100000:1.0:1780298644.193725:0:17578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62020 00000100:00100000:1.0:1780298644.193734:0:17578:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254888700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+126:29360:x1866777609565696:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.193743:0:17578:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.193751:0:17578:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279c00/0xaf35132e63034c39 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 88 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3c3e expref: -99 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.193784:0:17578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278400/0xaf35132e63034c40 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 89 type: IBT flags: 0x40000000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.193791:0:17578:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278400/0xaf35132e63034c40 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 89 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.193797:0:17578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278400/0xaf35132e63034c40 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 89 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1780298644.193807:0:17578:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 4276323 00000004:00200000:1.0:1780298644.193809:0:17578:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1780298644.193816:0:17578:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278400/0xaf35132e63034c40 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 89 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.193828:0:17578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279c00/0xaf35132e63034c39 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 89 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c3e expref: 128 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.193836:0:17578:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278400/0xaf35132e63034c40 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 88 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c3e expref: 127 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.193844:0:17578:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.193846:0:17578:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1780298644.193847:0:17578:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278400, rc 0) 00000100:00100000:1.0:1780298644.193870:0:17578:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254888700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+127:29360:x1866777609565696:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 139us (256us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.193874:0:17578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62020 00000100:00100000:0.0:1780298644.194648:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.194673:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.194683:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609565824 00000100:00100000:2.0:1780298644.194687:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609565824 00000100:00100000:2.0:1780298644.194717:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609565952 00000100:00100000:2.0:1780298644.194720:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609565952 00000100:00100000:0.0:1780298644.194749:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9753 00000100:00100000:2.0:1780298644.194758:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9754 00000100:00100000:0.0:1780298644.194760:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254889180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609565824:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.194770:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298644.194772:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609565952:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.194782:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.194782:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005d10d3aa/0xaf35132e63034c47 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3c45 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.194795:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e31d8c5/0xaf35132e63034c4e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3c4c expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.194833:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605003648:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.194835:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605003776:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.194904:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.194904:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00002000:00010000:2.0:1780298644.195650:0:8358:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks: 0 -> 16 00000100:00100000:2.0:1780298644.195661:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605003776:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.195681:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003e31d8c5/0xaf35132e63034c4e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c4c expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.195695:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e31d8c5/0xaf35132e63034c4e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c4c expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.195716:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d08a00, rc 0) 00000100:00100000:2.0:1780298644.195746:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609565952:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 979us (1072us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.195753:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9754 00000100:00100000:1.0:1780298644.195756:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605003648:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.195773:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005d10d3aa/0xaf35132e63034c47 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c45 expref: 8 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.195786:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005d10d3aa/0xaf35132e63034c47 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c45 expref: 8 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.195795:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baec00, rc 0) 00000100:00100000:1.0:1780298644.195819:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254889180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609565824:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1063us (1173us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.195823:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9753 00000100:00100000:0.0:1780298644.196723:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.196756:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609566080 00000100:00100000:2.0:1780298644.196760:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609566080 00000100:00100000:2.0:1780298644.196784:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9755 00000100:00100000:2.0:1780298644.196789:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:29360:x1866777609566080:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.196797:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.196804:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ba4a4a6/0xaf35132e63034c55 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3c53 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.196819:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000fbbd4b6c/0xaf35132e63034bfa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c22 expref: 9 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.196825:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(8 122) 00010000:00010000:2.0:1780298644.196829:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(9 122) 00010000:00010000:2.0:1780298644.196832:0:20252:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ba4a4a6/0xaf35132e63034c55 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1114112->1118207] (req 1114112->1118207) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c53 expref: 7 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.196850:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000fbbd4b6c/0xaf35132e63034bfa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c22 expref: 9 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.196870:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000fbbd4b6c/0xaf35132e63034bfa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c22 expref: 9 pid: 20252 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.196878:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605003904:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298644.196915:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.197543:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605003904:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298644.197555:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007ba4a4a6/0xaf35132e63034c55 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1114112->1118207] (req 1114112->1118207) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c53 expref: 7 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.197563:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d09c00, rc 0) 00000100:00100000:2.0:1780298644.197591:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609566080:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 801us (867us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.197599:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9755 00000100:00100000:3.0:1780298644.197813:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.197847:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609566208 00000100:00100000:0.0:1780298644.197850:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609566208 00010000:00010000:0.0:1780298644.197867:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000fbbd4b6c/0xaf35132e63034bfa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c22 expref: 10 pid: 20252 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.197887:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15165 00000100:00100000:0.0:1780298644.197892:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609566208:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298644.197900:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298644.197909:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000fbbd4b6c/0xaf35132e63034bfa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c22 expref: 10 pid: 20252 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.197917:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000fbbd4b6c/0xaf35132e63034bfa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c22 expref: 10 pid: 20252 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.197929:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fbbd4b6c/0xaf35132e63034bfa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c22 expref: 10 pid: 20252 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.197936:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(9 122) 00010000:00010000:0.0:1780298644.197939:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(9 122) 00010000:00010000:0.0:1780298644.197945:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000007ba4a4a6/0xaf35132e63034c55 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1114112->34668543] (req 1114112->1118207) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c53 expref: 6 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.197952:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ba4a4a6/0xaf35132e63034c55 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1114112->34668543] (req 1114112->1118207) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c53 expref: 6 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.197967:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000007ba4a4a6/0xaf35132e63034c55 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1114112->34668543] (req 1114112->1118207) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c53 expref: 6 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.197973:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304fb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:20253:1866777605004032:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298644.198004:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.198673:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304fb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:20253:1866777605004032:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298644.198681:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298644.198702:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f0000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609566208:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 811us (892us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.198708:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15165 00000100:00100000:3.0:1780298644.198887:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.198918:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609566336 00000100:00100000:0.0:1780298644.198921:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609566336 00000100:00100000:0.0:1780298644.198948:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6270 00000100:00100000:0.0:1780298644.198954:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609566336:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298644.199104:0:16972:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 82019 -> 1114799 00002000:00010000:0.0:1780298644.199107:0:16972:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298644.199108:0:16972:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298644.199110:0:16972:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298644.199112:0:16972:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 16 -> 24 00000100:00100000:0.0:1780298644.199151:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609566336:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 198us (266us total) trans 4294969265 rc 0/0 00000100:00100000:0.0:1780298644.199156:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6270 00000100:00100000:1.0:1780298644.200020:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.200058:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609566464 00000100:00100000:2.0:1780298644.200062:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609566464 00000100:00100000:2.0:1780298644.200085:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9756 00000100:00100000:2.0:1780298644.200091:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609566464:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.200099:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.200109:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000debe9bb3/0xaf35132e63034c5c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3c5a expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.200170:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605004160:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.200210:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.200911:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605004160:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.200930:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000debe9bb3/0xaf35132e63034c5c lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c5a expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.200942:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000debe9bb3/0xaf35132e63034c5c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c5a expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.200949:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d09600, rc 0) 00000100:00100000:1.0:1780298644.200975:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609566464:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 885us (957us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.200980:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9756 00000100:00100000:3.0:1780298644.201773:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.201803:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609566592 00000100:00100000:0.0:1780298644.201807:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609566592 00000100:00100000:0.0:1780298644.201833:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9757 00000100:00100000:0.0:1780298644.201840:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609566592:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.201848:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.201857:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a3f03a/0xaf35132e63034c63 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3c61 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.201878:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605004288:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.201912:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.202550:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605004288:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.202568:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000001a3f03a/0xaf35132e63034c63 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c61 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.202583:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a3f03a/0xaf35132e63034c63 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c61 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.202594:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae600, rc 0) 00000100:00100000:3.0:1780298644.202623:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609566592:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 784us (852us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.202631:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9757 00000100:00100000:2.0:1780298644.203518:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.203556:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609566720 00000100:00100000:1.0:1780298644.203561:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609566720 00000100:00100000:1.0:1780298644.203598:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9758 00000100:00100000:1.0:1780298644.203608:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e4f0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609566720:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.203618:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.203638:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63034c6a lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3c68 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.203651:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.203679:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255488a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605004416:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.203681:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609566848 00000100:00100000:2.0:1780298644.203686:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609566848 00000100:00100000:2.0:1780298644.203713:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9759 00000100:00100000:2.0:1780298644.203721:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609566848:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.203729:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.203738:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fba32a6f/0xaf35132e63034c71 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3c6f expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.203742:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a540 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.203766:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fbb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605004544:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.203787:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.204387:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255488a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605004416:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.204401:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fbb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605004544:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.204412:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63034c6a lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c68 expref: 8 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.204419:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000fba32a6f/0xaf35132e63034c71 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c6f expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.204441:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63034c6a lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c68 expref: 8 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.204441:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fba32a6f/0xaf35132e63034c71 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c6f expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.204455:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279c00, rc 0) 00010000:00010000:0.0:1780298644.204464:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d08600, rc 0) 00000100:00100000:1.0:1780298644.204492:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e4f0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609566720:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 886us (973us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.204500:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609566848:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 778us (846us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.204506:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9758 00000100:00100000:0.0:1780298644.204515:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9759 00000100:00100000:3.0:1780298644.205459:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.205477:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.205495:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609566976 00000100:00100000:1.0:1780298644.205499:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609566976 00000100:00100000:2.0:1780298644.205524:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609567104 00000100:00100000:1.0:1780298644.205528:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9760 00000100:00100000:2.0:1780298644.205529:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609567104 00000100:00100000:1.0:1780298644.205536:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609566976:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.205543:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.205552:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e63034c78 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3c76 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.205561:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9761 00000100:00100000:2.0:1780298644.205569:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609567104:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.205576:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298644.205580:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605004672:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.205584:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e31d8c5/0xaf35132e63034c7f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3c7d expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.205612:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605004800:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.205630:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a540 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.205637:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.206408:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605004672:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.206417:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605004800:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.206432:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e63034c78 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c76 expref: 8 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.206436:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003e31d8c5/0xaf35132e63034c7f lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c7d expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.206456:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e63034c78 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c76 expref: 8 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.206457:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e31d8c5/0xaf35132e63034c7f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c7d expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.206475:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279800, rc 0) 00010000:00010000:3.0:1780298644.206476:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244d08a00, rc 0) 00000100:00100000:1.0:1780298644.206507:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609566976:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 973us (1048us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.206513:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9760 00000100:00100000:3.0:1780298644.206518:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609567104:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 948us (1037us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.206524:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9761 00000100:00100000:0.0:1780298644.207524:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.207555:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609567232 00000100:00100000:1.0:1780298644.207559:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609567232 00000100:00100000:1.0:1780298644.207582:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9762 00000100:00100000:1.0:1780298644.207587:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609567232:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.207595:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.207602:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63034c86 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3c84 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.207617:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000fedd83c9/0xaf35132e63034bf3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c1b expref: 13 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.207624:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(5 169) 00010000:00010000:1.0:1780298644.207628:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(6 169) 00010000:00010000:1.0:1780298644.207631:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63034c86 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [4059136->4067327] (req 4059136->4067327) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c84 expref: 8 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.207653:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000fedd83c9/0xaf35132e63034bf3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c1b expref: 13 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.207663:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000fedd83c9/0xaf35132e63034bf3 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c1b expref: 13 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.207670:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605004928:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298644.207702:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a540 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.208329:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605004928:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298644.208346:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63034c86 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [4059136->4067327] (req 4059136->4067327) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c84 expref: 8 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.208363:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279200, rc 0) 00000100:00100000:1.0:1780298644.208391:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:29360:x1866777609567232:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 803us (868us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.208397:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9762 00000100:00100000:2.0:1780298644.209079:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.209116:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609567360 00000100:00100000:1.0:1780298644.209120:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609567360 00010000:00010000:1.0:1780298644.209160:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000fedd83c9/0xaf35132e63034bf3 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c1b expref: 14 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.209191:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15166 00000100:00100000:1.0:1780298644.209199:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609567360:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298644.209209:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298644.209222:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000fedd83c9/0xaf35132e63034bf3 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c1b expref: 14 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.209234:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000fedd83c9/0xaf35132e63034bf3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c1b expref: 14 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.209265:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000fedd83c9/0xaf35132e63034bf3 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c1b expref: 14 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.209277:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(6 169) 00010000:00010000:1.0:1780298644.209293:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(6 169) 00010000:00010000:1.0:1780298644.209302:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63034c86 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [4059136->37613567] (req 4059136->4067327) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c84 expref: 7 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.209311:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63034c86 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [4059136->37613567] (req 4059136->4067327) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c84 expref: 7 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.209335:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63034c86 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [4059136->37613567] (req 4059136->4067327) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c84 expref: 7 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.209345:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777605005056:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298644.209384:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a540 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.210085:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777605005056:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298644.210097:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298644.210138:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609567360:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 940us (1059us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.210147:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15166 00000100:00100000:2.0:1780298644.210251:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.210334:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609567488 00000100:00100000:3.0:1780298644.210339:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609567488 00000100:00100000:3.0:1780298644.210381:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6271 00000100:00100000:3.0:1780298644.210391:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609567488:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:3.0:1780298644.210558:0:19950:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:3.0:1780298644.210562:0:19950:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:3.0:1780298644.210565:0:19950:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:3.0:1780298644.210567:0:19950:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:3.0:1780298644.210570:0:19950:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 328 -> 328 00000100:00100000:3.0:1780298644.210602:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609567488:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 212us (351us total) trans 4294969714 rc 0/0 00000100:00100000:3.0:1780298644.210610:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6271 00000100:00100000:0.0:1780298644.211573:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.211616:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.211618:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609567616 00000100:00100000:1.0:1780298644.211622:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609567616 00000100:00100000:1.0:1780298644.211655:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609567744 00000100:00100000:1.0:1780298644.211657:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609567744 00000100:00100000:3.0:1780298644.211662:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9763 00000100:00100000:3.0:1780298644.211673:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609567616:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298644.211678:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9764 00010000:00010000:3.0:1780298644.211686:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298644.211687:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609567744:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.211696:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.211699:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009c841b52/0xaf35132e63034c8d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3c8b expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.211709:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007fddc766/0xaf35132e63034c94 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3c92 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.211744:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605005184:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.211750:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605005312:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.211799:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.211799:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a540 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.212665:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605005184:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.212686:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605005312:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.212693:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009c841b52/0xaf35132e63034c8d lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c8b expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.212708:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007fddc766/0xaf35132e63034c94 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c92 expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.212715:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009c841b52/0xaf35132e63034c8d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c8b expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.212729:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007fddc766/0xaf35132e63034c94 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c92 expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.212731:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157600, rc 0) 00010000:00010000:1.0:1780298644.212743:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278000, rc 0) 00000100:00100000:3.0:1780298644.212767:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609567616:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1097us (1195us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.212771:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609567744:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1083us (1154us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.212783:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9763 00000100:00100000:1.0:1780298644.212791:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9764 00000100:00100000:2.0:1780298644.213969:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.214001:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.214019:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609567872 00000100:00100000:3.0:1780298644.214025:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609567872 00000100:00100000:3.0:1780298644.214056:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609568000 00000100:00100000:3.0:1780298644.214057:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609568000 00000100:00100000:0.0:1780298644.214064:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9765 00000100:00100000:1.0:1780298644.214080:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9766 00000100:00100000:0.0:1780298644.214081:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609567872:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298644.214095:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609568000:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.214095:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.214106:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.214110:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e63034c9b lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3c99 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.214117:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017dab243/0xaf35132e63034ca2 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3ca0 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.214164:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605005440:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.214177:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605005568:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.214217:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a540 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.214217:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.214996:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605005440:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.215017:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000017dab243/0xaf35132e63034ca2 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ca0 expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.215038:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017dab243/0xaf35132e63034ca2 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ca0 expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.215044:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605005568:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.215053:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279800, rc 0) 00010000:00010000:2.0:1780298644.215066:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e63034c9b lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c99 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.215081:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e63034c9b lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c99 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.215091:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae600, rc 0) 00000100:00100000:1.0:1780298644.215091:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609568000:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 999us (1088us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.215101:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9766 00000100:00100000:2.0:1780298644.215120:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609567872:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1044us (1160us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.215161:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9765 00000100:00100000:2.0:1780298644.216334:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.216369:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609568128 00000100:00100000:1.0:1780298644.216375:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609568128 00000100:00100000:1.0:1780298644.216411:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9767 00000100:00100000:1.0:1780298644.216419:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:29360:x1866777609568128:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.216428:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.216438:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63034ca9 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3ca7 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.216456:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(6 169) 00010000:00010000:1.0:1780298644.216461:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(6 169) 00010000:00010000:1.0:1780298644.216470:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63034ca9 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [667648->4059135] (req 667648->671743) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ca7 expref: 14 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.216485:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63034ca9 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [667648->4059135] (req 667648->671743) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ca7 expref: 14 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.216494:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279c00, rc 0) 00000100:00100000:1.0:1780298644.216522:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:29360:x1866777609568128:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 105us (190us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.216529:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9767 00000100:00100000:0.0:1780298644.217309:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.217344:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609568256 00000100:00100000:1.0:1780298644.217349:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609568256 00002000:00100000:1.0:1780298644.217377:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (667648->671743) req@ffff9cf24304f800 x1866777609568256/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.217388:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24304f800 with x1866777609568256 ext(667648->671743) 00002000:00010000:1.0:1780298644.217393:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24304f800 00000100:00100000:1.0:1780298644.217415:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6272 00000100:00100000:1.0:1780298644.217422:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2398:x1866777609568256:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298644.217566:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2398:x1866777609568256:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 144us (257us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298644.217573:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6272 00002000:00100000:1.0:1780298644.217579:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (667648->671743) req@ffff9cf24304f800 x1866777609568256/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/4536 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.217588:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24304f800 with x1866777609568256 ext(667648->671743) 00002000:00010000:1.0:1780298644.217591:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24304f800 00000100:00100000:0.0:1780298644.218615:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.218649:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609568384 00000100:00100000:1.0:1780298644.218653:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609568384 00000100:00100000:1.0:1780298644.218678:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9768 00000100:00100000:1.0:1780298644.218684:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609568384:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.218690:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.218699:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000488d1c2a/0xaf35132e63034cb0 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3cae expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.218735:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.218738:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605005696:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.218777:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609568512 00000100:00100000:2.0:1780298644.218783:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609568512 00000100:00100000:1.0:1780298644.218788:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a6c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.218819:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9769 00000100:00100000:2.0:1780298644.218831:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609568512:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.218840:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.218849:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000beb94698/0xaf35132e63034cb7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3cb5 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.218880:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605005824:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.218913:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb10c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.219539:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605005696:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.219560:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000488d1c2a/0xaf35132e63034cb0 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3cae expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.219575:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000488d1c2a/0xaf35132e63034cb0 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3cae expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.219586:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279400, rc 0) 00000100:00100000:1.0:1780298644.219616:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609568384:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 931us (1001us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.219623:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9768 00000100:00100000:2.0:1780298644.219826:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605005824:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.219842:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000beb94698/0xaf35132e63034cb7 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3cb5 expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.219855:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000beb94698/0xaf35132e63034cb7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3cb5 expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.219864:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf3821e6a00, rc 0) 00000100:00100000:2.0:1780298644.219893:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609568512:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1067us (1159us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.219899:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9769 00000100:00100000:0.0:1780298644.220875:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.220895:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.220909:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609568640 00000100:00100000:2.0:1780298644.220914:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609568640 00000100:00100000:3.0:1780298644.220929:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609568768 00000100:00100000:3.0:1780298644.220934:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609568768 00000100:00100000:2.0:1780298644.220944:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9770 00000100:00100000:2.0:1780298644.220953:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609568640:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.220960:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298644.220966:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9771 00010000:00010000:2.0:1780298644.220970:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007dc48e1c/0xaf35132e63034cbe lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3cbc expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.220978:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609568768:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.220989:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.220998:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017dab243/0xaf35132e63034cc5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3cc3 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.221008:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605005952:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.221026:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605006080:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.221054:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb10c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.221059:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.221955:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605006080:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.221956:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605005952:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.221984:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000017dab243/0xaf35132e63034cc5 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3cc3 expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.221989:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007dc48e1c/0xaf35132e63034cbe lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3cbc expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.222003:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017dab243/0xaf35132e63034cc5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3cc3 expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.222009:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007dc48e1c/0xaf35132e63034cbe lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3cbc expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.222022:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830c800, rc 0) 00010000:00010000:1.0:1780298644.222025:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279800, rc 0) 00000100:00100000:2.0:1780298644.222071:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609568640:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1112us (1187us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.222071:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609568768:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1090us (1166us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.222088:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9771 00000100:00100000:2.0:1780298644.222094:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9770 00000100:00100000:0.0:1780298644.223657:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.223694:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609568896 00000100:00100000:1.0:1780298644.223700:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609568896 00000100:00100000:1.0:1780298644.223741:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9772 00000100:00100000:1.0:1780298644.223750:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:29360:x1866777609568896:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.223761:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.223773:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007fddc766/0xaf35132e63034ccc lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3cca expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.223797:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63034ca9 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [667648->4059135] (req 667648->671743) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ca7 expref: 15 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.223806:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(6 169) 00010000:00010000:1.0:1780298644.223810:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(7 169) 00010000:00010000:1.0:1780298644.223815:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007fddc766/0xaf35132e63034ccc lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2691072->2699263] (req 2691072->2699263) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3cca expref: 15 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.223846:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63034ca9 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [667648->4059135] (req 667648->671743) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ca7 expref: 15 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.223860:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63034ca9 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [667648->4059135] (req 667648->671743) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ca7 expref: 15 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.223872:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605006208:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298644.223927:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.224628:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605006208:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298644.224647:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007fddc766/0xaf35132e63034ccc lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2691072->2699263] (req 2691072->2699263) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3cca expref: 15 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.224660:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278000, rc 0) 00000100:00100000:1.0:1780298644.224699:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:29360:x1866777609568896:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 951us (1044us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.224707:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9772 00000100:00100000:0.0:1780298644.224886:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.224933:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609569024 00000100:00100000:1.0:1780298644.224938:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609569024 00010000:00010000:1.0:1780298644.224962:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63034ca9 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [667648->4059135] (req 667648->671743) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ca7 expref: 15 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.224992:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15167 00000100:00100000:1.0:1780298644.225000:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b5c450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2400:x1866777609569024:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298644.225022:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298644.225041:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63034ca9 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [667648->4059135] (req 667648->671743) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ca7 expref: 15 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.225053:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63034ca9 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [667648->4059135] (req 667648->671743) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ca7 expref: 15 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.225072:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63034ca9 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [667648->4059135] (req 667648->671743) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ca7 expref: 15 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.225083:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(7 169) 00010000:00010000:1.0:1780298644.225087:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(7 169) 00010000:00010000:1.0:1780298644.225096:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000007fddc766/0xaf35132e63034ccc lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2691072->4059135] (req 2691072->2699263) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3cca expref: 14 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.225106:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007fddc766/0xaf35132e63034ccc lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2691072->4059135] (req 2691072->2699263) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3cca expref: 14 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.225159:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000007fddc766/0xaf35132e63034ccc lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2691072->4059135] (req 2691072->2699263) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3cca expref: 14 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.225171:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777605006336:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298644.225210:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.225964:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548b100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777605006336:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298644.225975:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298644.226014:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b5c450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609569024:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1015us (1129us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.226023:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15167 00000100:00100000:3.0:1780298644.226241:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.226272:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609569152 00000100:00100000:2.0:1780298644.226276:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609569152 00000100:00100000:2.0:1780298644.226316:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6273 00000100:00100000:2.0:1780298644.226322:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609569152:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298644.226493:0:8364:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:2.0:1780298644.226497:0:8364:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298644.226499:0:8364:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298644.226502:0:8364:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298644.226504:0:8364:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 328 -> 328 00000100:00100000:2.0:1780298644.226536:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2399:x1866777609569152:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 214us (296us total) trans 4294969715 rc 0/0 00000100:00100000:2.0:1780298644.226543:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6273 00000100:00100000:0.0:1780298644.227459:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.227498:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.227503:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609569280 00000100:00100000:2.0:1780298644.227510:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609569280 00000100:00100000:2.0:1780298644.227540:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609569408 00000100:00100000:2.0:1780298644.227543:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609569408 00000100:00100000:3.0:1780298644.227562:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9773 00000100:00100000:3.0:1780298644.227574:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2399:x1866777609569280:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298644.227576:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9774 00010000:00010000:3.0:1780298644.227587:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298644.227589:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609569408:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.227598:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.227602:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009c841b52/0xaf35132e63034cd3 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3cd1 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.227608:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63034cda lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3cd8 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.227655:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605006464:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.227692:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605006592:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.227698:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.227725:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb10c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.228447:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605006464:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.228468:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605006592:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.228473:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63034cda lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3cd8 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.228491:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009c841b52/0xaf35132e63034cd3 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3cd1 expref: 16 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.228494:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63034cda lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3cd8 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.228517:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae000, rc 0) 00010000:00010000:2.0:1780298644.228522:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009c841b52/0xaf35132e63034cd3 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3cd1 expref: 16 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.228535:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157600, rc 0) 00000100:00100000:3.0:1780298644.228545:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609569408:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 957us (1045us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.228553:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9774 00000100:00100000:2.0:1780298644.228561:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2399:x1866777609569280:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 990us (1102us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.228569:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9773 00000100:00100000:2.0:1780298644.229606:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.229660:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609569536 00000100:00100000:0.0:1780298644.229666:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609569536 00000100:00100000:0.0:1780298644.229709:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9775 00000100:00100000:0.0:1780298644.229719:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2397:x1866777609569536:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.229732:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298644.229742:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:0.0:1780298644.229743:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63034ce1 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3cdf expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.229770:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609569664 00000100:00100000:1.0:1780298644.229775:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609569664 00000100:00100000:0.0:1780298644.229781:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605006720:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.229806:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9776 00000100:00100000:1.0:1780298644.229815:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609569664:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.229824:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.229835:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017dab243/0xaf35132e63034ce8 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3ce6 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.229838:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.229867:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605006848:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.229898:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a840 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.230653:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605006720:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.230666:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605006848:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.230678:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63034ce1 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3cdf expref: 16 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.230682:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000017dab243/0xaf35132e63034ce8 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ce6 expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.230707:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63034ce1 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3cdf expref: 16 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.230708:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017dab243/0xaf35132e63034ce8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ce6 expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.230731:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baee00, rc 0) 00010000:00010000:1.0:1780298644.230732:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279800, rc 0) 00000100:00100000:1.0:1780298644.230767:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609569664:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 953us (1026us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.230767:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2397:x1866777609569536:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1049us (1160us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.230779:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9776 00000100:00100000:0.0:1780298644.230789:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9775 00000100:00100000:2.0:1780298644.231804:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.231841:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609569920 00000100:00100000:2.0:1780298644.231846:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.231846:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609569920 00000100:00100000:1.0:1780298644.231878:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609569792 00000100:00100000:1.0:1780298644.231881:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609569792 00000100:00100000:3.0:1780298644.231887:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9777 00000100:00100000:3.0:1780298644.231897:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609569920:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.231905:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298644.231908:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9778 00010000:00010000:3.0:1780298644.231914:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e0b8e7b/0xaf35132e63034cef lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3cf4 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.231920:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2400:x1866777609569792:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.231930:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.231939:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000488d1c2a/0xaf35132e63034cf6 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3ced expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.231950:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605006976:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.231973:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605007104:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.231995:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.232003:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.232680:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605007104:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.232712:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000488d1c2a/0xaf35132e63034cf6 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ced expref: 16 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.232730:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000488d1c2a/0xaf35132e63034cf6 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ced expref: 16 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.232740:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279400, rc 0) 00000100:00100000:1.0:1780298644.232766:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2400:x1866777609569792:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 851us (921us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.232774:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9778 00000100:00100000:0.0:1780298644.232812:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605006976:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.232827:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009e0b8e7b/0xaf35132e63034cef lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3cf4 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.232837:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e0b8e7b/0xaf35132e63034cef lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3cf4 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.232848:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156c00, rc 0) 00000100:00100000:0.0:1780298644.232870:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609569920:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 977us (1069us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.232876:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9777 00000100:00100000:3.0:1780298644.233668:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.233701:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609570048 00000100:00100000:2.0:1780298644.233707:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609570048 00000100:00100000:2.0:1780298644.233740:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9779 00000100:00100000:2.0:1780298644.233748:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609570048:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.233757:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.233766:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ccb4ef2a/0xaf35132e63034cfd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3cfb expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.233788:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(9 122) 00010000:00010000:2.0:1780298644.233792:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(9 122) 00010000:00010000:2.0:1780298644.233801:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ccb4ef2a/0xaf35132e63034cfd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [892928->1114111] (req 892928->897023) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3cfb expref: 10 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.233816:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ccb4ef2a/0xaf35132e63034cfd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [892928->1114111] (req 892928->897023) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3cfb expref: 10 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.233824:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830cc00, rc 0) 00000100:00100000:2.0:1780298644.233853:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609570048:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 107us (185us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.233859:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9779 00000100:00100000:3.0:1780298644.234668:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.234707:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609570176 00000100:00100000:2.0:1780298644.234712:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609570176 00002000:00100000:2.0:1780298644.234740:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x12:0x0] (892928->897023) req@ffff9cf2558f0e00 x1866777609570176/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.234749:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2558f0e00 with x1866777609570176 ext(892928->897023) 00002000:00010000:2.0:1780298644.234754:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2558f0e00 00000100:00100000:2.0:1780298644.234777:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6274 00000100:00100000:2.0:1780298644.234784:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609570176:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298644.234937:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609570176:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 154us (269us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298644.234945:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6274 00002000:00100000:2.0:1780298644.234952:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x12:0x0] (892928->897023) req@ffff9cf2558f0e00 x1866777609570176/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/4536 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.234961:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2558f0e00 with x1866777609570176 ext(892928->897023) 00002000:00010000:2.0:1780298644.234964:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2558f0e00 00000100:00100000:0.0:1780298644.236099:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.236158:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609570304 00000100:00100000:2.0:1780298644.236163:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609570304 00002000:00100000:2.0:1780298644.236187:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x12:0x0] (897024->901119) req@ffff9cf24304fb80 x1866777609570304/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.236196:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24304fb80 with x1866777609570304 ext(897024->901119) 00002000:00010000:2.0:1780298644.236199:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24304fb80 00000100:00100000:2.0:1780298644.236219:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6275 00000100:00100000:2.0:1780298644.236227:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609570304:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298644.236345:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609570304:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 118us (247us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298644.236353:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6275 00002000:00100000:2.0:1780298644.236360:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x12:0x0] (897024->901119) req@ffff9cf24304fb80 x1866777609570304/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/4536 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.236368:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24304fb80 with x1866777609570304 ext(897024->901119) 00002000:00010000:2.0:1780298644.236371:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24304fb80 00000100:00100000:3.0:1780298644.237617:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.237658:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609570432 00000100:00100000:1.0:1780298644.237662:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609570432 00002000:00100000:1.0:1780298644.237683:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (901120->905215) req@ffff9cf2558f0700 x1866777609570432/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.237710:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2558f0700 with x1866777609570432 ext(901120->905215) 00002000:00010000:1.0:1780298644.237714:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2558f0700 00000100:00100000:1.0:1780298644.237739:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6276 00000100:00100000:1.0:1780298644.237748:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609570432:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298644.237835:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609570432:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 88us (218us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298644.237842:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6276 00002000:00100000:1.0:1780298644.237849:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (901120->905215) req@ffff9cf2558f0700 x1866777609570432/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/4536 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.237858:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2558f0700 with x1866777609570432 ext(901120->905215) 00002000:00010000:1.0:1780298644.237862:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2558f0700 00000100:00100000:3.0:1780298644.239114:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.239184:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609570560 00000100:00100000:1.0:1780298644.239190:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609570560 00002000:00100000:1.0:1780298644.239216:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x12:0x0] (892928->905215) req@ffff9cf2558f0000 x1866777609570560/t0(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 12776/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.239225:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2558f0000 with x1866777609570560 ext(892928->905215) 00002000:00010000:1.0:1780298644.239229:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2558f0000 00000100:00100000:1.0:1780298644.239250:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6277 00000100:00100000:1.0:1780298644.239258:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609570560:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:1.0:1780298644.239705:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609570560:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 446us (591us total) trans 4294969266 rc 0/0 00000100:00100000:1.0:1780298644.239712:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6277 00002000:00100000:1.0:1780298644.239720:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x12:0x0] (892928->905215) req@ffff9cf2558f0000 x1866777609570560/t4294969266(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 12776/448 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.239729:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2558f0000 with x1866777609570560 ext(892928->905215) 00002000:00010000:1.0:1780298644.239731:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2558f0000 00000100:00100000:0.0:1780298644.240703:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.240759:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609570688 00000100:00100000:2.0:1780298644.240764:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609570688 00000100:00100000:2.0:1780298644.240799:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9780 00000100:00100000:2.0:1780298644.240809:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2399:x1866777609570688:12345-192.168.203.25@tcp:16:fsx.0 00000001:00080000:0.0:1780298644.241469:0:8351:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0000: transno 4294969266 is committed 00000001:00080000:0.0:1780298644.241478:0:8351:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0000: transno 4294969265 is committed 00000100:00100000:2.0:1780298644.241548:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609570688:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 742us (847us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.241557:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9780 00000100:00100000:3.0:1780298644.242595:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.242626:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.242643:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609570816 00000100:00100000:2.0:1780298644.242647:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609570816 00000100:00100000:2.0:1780298644.242672:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609570944 00000100:00100000:2.0:1780298644.242674:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609570944 00000100:00100000:1.0:1780298644.242687:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9781 00000100:00100000:1.0:1780298644.242700:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a82d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2398:x1866777609570816:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298644.242703:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9782 00010000:00010000:1.0:1780298644.242715:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298644.242716:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a81f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609570944:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.242726:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.242734:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000488d1c2a/0xaf35132e63034d04 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3d02 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.242738:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e4811bb2/0xaf35132e63034d0b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3d09 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.242803:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481faa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605007360:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.242805:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605007232:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.242873:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb13c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.242873:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00002000:00010000:2.0:1780298644.243748:0:8356:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 24 -> 48 00000100:00100000:2.0:1780298644.243756:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481faa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605007360:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.243756:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605007232:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.243785:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e4811bb2/0xaf35132e63034d0b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d09 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.243787:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000488d1c2a/0xaf35132e63034d04 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d02 expref: 16 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.243810:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e4811bb2/0xaf35132e63034d0b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d09 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.243812:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000488d1c2a/0xaf35132e63034d04 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d02 expref: 16 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.243823:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830ca00, rc 0) 00010000:00010000:1.0:1780298644.243830:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279400, rc 0) 00000100:00100000:2.0:1780298644.243851:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a81f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609570944:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1138us (1226us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.243857:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a82d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2398:x1866777609570816:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1160us (1261us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.243858:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9782 00000100:00100000:1.0:1780298644.243869:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9781 00000100:00100000:0.0:1780298644.244951:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.244992:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609571072 00000100:00100000:2.0:1780298644.244997:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609571072 00000100:00100000:0.0:1780298644.245031:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.245038:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9783 00000100:00100000:2.0:1780298644.245044:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2399:x1866777609571072:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.245052:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.245062:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001011cd71/0xaf35132e63034d12 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3d10 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.245076:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609571200 00000100:00100000:1.0:1780298644.245081:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609571200 00000100:00100000:2.0:1780298644.245093:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481faa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605007488:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.245118:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9784 00000100:00100000:3.0:1780298644.245152:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609571200:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.245160:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.245169:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008746f5b4/0xaf35132e63034d19 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3d17 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.245169:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb13c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.245231:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241a80380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605007616:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.245256:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.245827:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481faa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605007488:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.245848:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001011cd71/0xaf35132e63034d12 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d10 expref: 16 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.245863:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001011cd71/0xaf35132e63034d12 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d10 expref: 16 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.245874:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830c600, rc 0) 00000100:00100000:2.0:1780298644.245903:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2399:x1866777609571072:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 859us (953us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.245911:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9783 00000100:00100000:0.0:1780298644.245932:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241a80380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605007616:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.245958:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008746f5b4/0xaf35132e63034d19 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d17 expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.245974:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008746f5b4/0xaf35132e63034d19 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d17 expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.245985:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157200, rc 0) 00000100:00100000:0.0:1780298644.246017:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609571200:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 869us (987us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.246022:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9784 00000100:00100000:1.0:1780298644.247593:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.247639:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609571456 00000100:00100000:2.0:1780298644.247645:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609571456 00000100:00100000:2.0:1780298644.247693:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62021 00000100:00100000:2.0:1780298644.247702:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+167:29360:x1866777609571456:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:2.0:1780298644.247748:0:6477:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298644.247768:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09e00/0xaf35132e63034c32 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 89 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c37 expref: 167 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.247784:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298644.247788:0:6477:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298644.247937:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c400/0xaf35132e63034d20 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 88 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.247953:0:6477:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278400/0xaf35132e63034c40 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 88 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c3e expref: 126 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.247974:0:6477:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c400/0xaf35132e63034d20 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 88 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.248000:0:6477:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278400/0xaf35132e63034c40 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 88 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c3e expref: 126 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.248014:0:6477:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278400/0xaf35132e63034c40 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 88 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c3e expref: 126 pid: 17578 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.248026:0:6477:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481faa00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6477:1866777605007744:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298644.248083:0:6477:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb10c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.248801:0:6477:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481faa00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6477:1866777605007744:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298644.248816:0:6477:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c400/0xaf35132e63034d20 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 88 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.249010:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.249109:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609571584 00000100:00100000:0.0:1780298644.249122:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609571584 00010000:00010000:0.0:1780298644.249200:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278400/0xaf35132e63034c40 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 88 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c3e expref: 127 pid: 17578 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.249241:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15168 00000100:00100000:0.0:1780298644.249250:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+127:2400:x1866777609571584:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298644.249266:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298644.249272:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278400/0xaf35132e63034c40 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 89 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c3e expref: 127 pid: 17578 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.249295:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278400/0xaf35132e63034c40 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 89 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c3e expref: 127 pid: 17578 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.249323:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278400/0xaf35132e63034c40 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 89 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c3e expref: 127 pid: 17578 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.249342:0:7400:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298644.249347:0:7400:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c400/0xaf35132e63034d20 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 88 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.249368:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c400/0xaf35132e63034d20 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 88 type: IBT flags: 0x40290000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.249377:0:7400:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c400/0xaf35132e63034d20 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 88 type: IBT flags: 0x40290000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.249403:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298644.249428:0:6477:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c400/0xaf35132e63034d20 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 87 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.249437:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c400/0xaf35132e63034d20 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 87 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.249462:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+126:2400:x1866777609571584:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 212us (450us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.249470:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15168 00000001:00080000:2.0:1780298644.249496:0:6477:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991078 00000004:00010000:2.0:1780298644.249525:0:6477:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 0000000017b2ff5c reply state 00000000e30778c0 transno 4294991079 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c400/0xaf35132e63034d20 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 87 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.249540:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c400/0xaf35132e63034d20 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 87 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.249548:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298644.249550:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1780298644.249553:0:6477:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1780298644.249621:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+169:29360:x1866777609571456:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1919us (2029us total) trans 4294991079 rc 0/0 00000100:00100000:2.0:1780298644.249629:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62021 00000020:00080000:2.0:1780298644.249636:0:6477:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1780298644.250464:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.250506:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609571712 00000100:00100000:3.0:1780298644.250512:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609571712 00000100:00100000:3.0:1780298644.250545:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9785 00000100:00100000:3.0:1780298644.250553:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:29360:x1866777609571712:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.250565:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.250583:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e63034d27 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3d1e expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.250605:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000007fddc766/0xaf35132e63034ccc lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2691072->4059135] (req 2691072->2699263) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3cca expref: 16 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.250615:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63034c86 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [4059136->37613567] (req 4059136->4067327) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c84 expref: 8 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.250620:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 2 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(7 169) 00010000:00010000:3.0:1780298644.250624:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(9 169) 00010000:00010000:3.0:1780298644.250630:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e63034d27 lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d1e expref: 16 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.250653:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63034c86 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [4059136->37613567] (req 4059136->4067327) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c84 expref: 8 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.250668:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63034c86 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [4059136->37613567] (req 4059136->4067327) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c84 expref: 8 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.250678:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241a80a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605007872:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298644.250725:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000007fddc766/0xaf35132e63034ccc lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2691072->4059135] (req 2691072->2699263) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3cca expref: 16 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.250736:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000007fddc766/0xaf35132e63034ccc lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2691072->4059135] (req 2691072->2699263) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3cca expref: 16 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.250746:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241a83b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605008000:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298644.250769:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.251456:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241a80a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605007872:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298644.251526:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241a83b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605008000:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298644.251545:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e63034d27 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d1e expref: 16 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.251599:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156a00, rc 0) 00000100:00100000:3.0:1780298644.251627:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:29360:x1866777609571712:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1075us (1164us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298644.251634:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9785 00000100:00100000:0.0:1780298644.251773:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.251796:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.251809:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609571840 00000100:00100000:3.0:1780298644.251814:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609571840 00010000:00010000:3.0:1780298644.251835:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000007fddc766/0xaf35132e63034ccc lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2691072->4059135] (req 2691072->2699263) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3cca expref: 16 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.251837:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609571968 00000100:00100000:1.0:1780298644.251843:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609571968 00010000:00010000:1.0:1780298644.251865:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63034c86 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [4059136->37613567] (req 4059136->4067327) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c84 expref: 9 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.251871:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15169 00000100:00100000:3.0:1780298644.251885:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b6300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2399:x1866777609571840:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298644.251895:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1780298644.251904:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15170 00010000:00010000:3.0:1780298644.251910:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000007fddc766/0xaf35132e63034ccc lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2691072->4059135] (req 2691072->2699263) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3cca expref: 16 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.251917:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b6680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609571968:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298644.251923:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000007fddc766/0xaf35132e63034ccc lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2691072->4059135] (req 2691072->2699263) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3cca expref: 16 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.251927:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298644.251940:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007fddc766/0xaf35132e63034ccc lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [2691072->4059135] (req 2691072->2699263) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3cca expref: 16 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.251946:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63034c86 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [4059136->37613567] (req 4059136->4067327) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c84 expref: 9 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.251950:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298644.251961:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63034c86 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [4059136->37613567] (req 4059136->4067327) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c84 expref: 9 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.251978:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63034c86 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [4059136->37613567] (req 4059136->4067327) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c84 expref: 9 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.251979:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b6300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2399:x1866777609571840:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 96us (204us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298644.251995:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15169 00010000:00010000:1.0:1780298644.252008:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(9 169) 00010000:00010000:1.0:1780298644.252011:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(9 169) 00010000:00010000:1.0:1780298644.252016:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e63034d27 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d1e expref: 14 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.252023:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e63034d27 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d1e expref: 14 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.252042:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e63034d27 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d1e expref: 14 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.252049:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548aa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777605008128:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298644.252085:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.252764:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548aa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777605008128:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298644.252772:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298644.252794:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b6680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609571968:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 885us (999us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.252799:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15170 00000100:00100000:2.0:1780298644.252882:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.252908:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609572096 00000100:00100000:1.0:1780298644.252912:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609572096 00000100:00100000:1.0:1780298644.252933:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9786 00000100:00100000:1.0:1780298644.252937:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609572096:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.252945:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.252953:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034d2e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3d25 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.252966:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000ccb4ef2a/0xaf35132e63034cfd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [892928->1114111] (req 892928->897023) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3cfb expref: 11 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.252972:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000007ba4a4a6/0xaf35132e63034c55 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1114112->34668543] (req 1114112->1118207) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c53 expref: 6 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.252980:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 2 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(9 122) 00010000:00010000:1.0:1780298644.252982:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(11 122) 00010000:00010000:1.0:1780298644.252985:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034d2e lrc: 5/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d25 expref: 11 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.253000:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000007ba4a4a6/0xaf35132e63034c55 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1114112->34668543] (req 1114112->1118207) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c53 expref: 6 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.253011:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000007ba4a4a6/0xaf35132e63034c55 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1114112->34668543] (req 1114112->1118207) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c53 expref: 6 pid: 20252 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.253020:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605008256:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298644.253065:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000ccb4ef2a/0xaf35132e63034cfd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [892928->1114111] (req 892928->897023) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3cfb expref: 11 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.253074:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000ccb4ef2a/0xaf35132e63034cfd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [892928->1114111] (req 892928->897023) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3cfb expref: 11 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.253084:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255488a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605008384:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298644.253105:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.253792:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605008256:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298644.253800:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255488a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605008384:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298644.253810:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034d2e lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d25 expref: 11 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.253822:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279400, rc 0) 00000100:00100000:1.0:1780298644.253854:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:29360:x1866777609572096:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 915us (971us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.253861:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9786 00000100:00100000:0.0:1780298644.253932:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.253973:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609572224 00000100:00100000:2.0:1780298644.253977:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609572224 00010000:00010000:2.0:1780298644.254006:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000007ba4a4a6/0xaf35132e63034c55 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1114112->34668543] (req 1114112->1118207) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c53 expref: 7 pid: 20252 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.254033:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15171 00000100:00100000:2.0:1780298644.254045:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609572224:12345-192.168.203.25@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1780298644.254049:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:2.0:1780298644.254056:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298644.254069:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000007ba4a4a6/0xaf35132e63034c55 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [1114112->34668543] (req 1114112->1118207) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c53 expref: 7 pid: 20252 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.254081:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000007ba4a4a6/0xaf35132e63034c55 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [1114112->34668543] (req 1114112->1118207) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c53 expref: 7 pid: 20252 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.254096:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ba4a4a6/0xaf35132e63034c55 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [1114112->34668543] (req 1114112->1118207) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3c53 expref: 7 pid: 20252 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.254099:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609572352 00000100:00100000:1.0:1780298644.254104:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609572352 00010000:00010000:2.0:1780298644.254110:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298644.254149:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000ccb4ef2a/0xaf35132e63034cfd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [892928->1114111] (req 892928->897023) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3cfb expref: 11 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.254168:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609572224:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 126us (234us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.254177:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15171 00000100:00100000:1.0:1780298644.254183:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15172 00000100:00100000:1.0:1780298644.254193:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a83b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609572352:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298644.254202:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298644.254213:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000ccb4ef2a/0xaf35132e63034cfd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [892928->1114111] (req 892928->897023) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3cfb expref: 11 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.254224:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000ccb4ef2a/0xaf35132e63034cfd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [892928->1114111] (req 892928->897023) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3cfb expref: 11 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.254236:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ccb4ef2a/0xaf35132e63034cfd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [892928->1114111] (req 892928->897023) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3cfb expref: 11 pid: 12101 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.254246:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(11 122) 00010000:00010000:1.0:1780298644.254249:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(11 122) 00010000:00010000:1.0:1780298644.254254:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034d2e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d25 expref: 10 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.254263:0:6463:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034d2e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d25 expref: 10 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.254296:0:6463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034d2e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d25 expref: 10 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.254307:0:6463:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255488a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777605008512:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298644.254348:0:6463:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.255491:0:6463:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255488a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777605008512:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298644.255501:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:1.0:1780298644.255502:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298644.255534:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a83b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609572352:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1343us (1491us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.255539:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609572480 00000100:00100000:2.0:1780298644.255543:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609572480 00000100:00100000:1.0:1780298644.255544:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15172 00000100:00100000:3.0:1780298644.255550:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00002000:00010000:2.0:1780298644.255564:0:8364:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:2.0:1780298644.255568:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf241a80a80 with x1866777609572480 ext(4194304->18446744073709551615) 00000100:00100000:0.0:1780298644.255570:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609572608 00002000:00010000:2.0:1780298644.255575:0:8364:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf241a80a80. 00000100:00100000:0.0:1780298644.255576:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609572608 00002000:00010000:0.0:1780298644.255596:0:19943:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4113282->18446744073709551615) 00000100:00100000:2.0:1780298644.255601:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6278 00002000:00100000:0.0:1780298644.255604:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf241a81500 with x1866777609572608 ext(4113282->18446744073709551615) 00000100:00100000:2.0:1780298644.255611:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a80a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2400:x1866777609572480:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:0.0:1780298644.255612:0:19943:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf241a81500. 00000100:00100000:0.0:1780298644.255638:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6279 00000100:00100000:0.0:1780298644.255647:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a81500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609572608:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:2.0:1780298644.255731:0:8364:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:2.0:1780298644.255735:0:8364:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298644.255738:0:8364:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298644.255741:0:8364:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298644.255743:0:8364:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 328 -> 328 00002000:00010000:0.0:1780298644.255777:0:19943:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 1114799 -> 4113282 00000100:00100000:2.0:1780298644.255782:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a80a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609572480:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 171us (282us total) trans 4294969716 rc 0/0 00002000:00010000:0.0:1780298644.255783:0:19943:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298644.255789:0:19943:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298644.255794:0:19943:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00000100:00100000:2.0:1780298644.255797:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6278 00002000:00010000:0.0:1780298644.255799:0:19943:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 48 -> 48 00002000:00010000:2.0:1780298644.255802:0:8364:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:2.0:1780298644.255807:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf241a80a80 with x1866777609572480 ext(4194304->18446744073709551615) 00002000:00010000:2.0:1780298644.255812:0:8364:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf241a80a80. 00000100:00100000:0.0:1780298644.255830:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a81500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609572608:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 183us (281us total) trans 4294969267 rc 0/0 00000100:00100000:0.0:1780298644.255839:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6279 00002000:00010000:0.0:1780298644.255842:0:19943:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4113282->18446744073709551615) 00002000:00100000:0.0:1780298644.255846:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf241a81500 with x1866777609572608 ext(4113282->18446744073709551615) 00002000:00010000:0.0:1780298644.255851:0:19943:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf241a81500. 00000100:00100000:3.0:1780298644.256599:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.256628:0:7411:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609572736 00000100:00100000:2.0:1780298644.256632:0:7411:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609572736 00000100:00100000:2.0:1780298644.256657:0:7411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62022 00000100:00100000:2.0:1780298644.256661:0:7411:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a80e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+126:29360:x1866777609572736:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.256670:0:7411:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.256681:0:7411:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830de00/0xaf35132e63034d35 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 88 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3d2c expref: -99 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.256737:0:7411:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c800/0xaf35132e63034d3c lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 89 type: IBT flags: 0x40000000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.256748:0:7411:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c800/0xaf35132e63034d3c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 89 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.256755:0:7411:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c800/0xaf35132e63034d3c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 89 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298644.256770:0:7411:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 8307586 00000004:00200000:2.0:1780298644.256773:0:7411:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298644.256785:0:7411:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c800/0xaf35132e63034d3c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 89 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.256800:0:7411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830de00/0xaf35132e63034d35 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 89 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d2c expref: 128 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.256809:0:7411:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c800/0xaf35132e63034d3c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 88 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d2c expref: 127 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.256814:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298644.256816:0:7411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298644.256818:0:7411:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830c800, rc 0) 00000100:00100000:2.0:1780298644.256840:0:7411:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a80e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+127:29360:x1866777609572736:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 179us (241us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.256844:0:7411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62022 00000100:00100000:1.0:1780298644.257728:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.257762:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609572864 00000100:00100000:2.0:1780298644.257766:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609572864 00000100:00100000:2.0:1780298644.257792:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9787 00000100:00100000:2.0:1780298644.257798:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255488a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609572864:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.257805:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.257814:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001011cd71/0xaf35132e63034d43 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3d33 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.257843:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.257851:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605008640:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.257889:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609572992 00000100:00100000:2.0:1780298644.257894:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb13c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.257895:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609572992 00000100:00100000:0.0:1780298644.257930:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9788 00000100:00100000:0.0:1780298644.257938:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a82300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609572992:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.257947:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.257956:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e63034d4a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3d3a expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.257985:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36d4b5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605008768:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.258016:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.258548:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605008640:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.258565:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001011cd71/0xaf35132e63034d43 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d33 expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.258577:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001011cd71/0xaf35132e63034d43 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d33 expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.258584:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830c600, rc 0) 00000100:00100000:2.0:1780298644.258610:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255488a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609572864:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 811us (883us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.258617:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9787 00000100:00100000:0.0:1780298644.258752:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36d4b5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605008768:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.258766:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e63034d4a lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d3a expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.258775:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e63034d4a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d3a expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.258781:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baec00, rc 0) 00000100:00100000:0.0:1780298644.258802:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a82300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609572992:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 867us (962us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.258808:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9788 00010000:00010000:0.0:1780298644.259223:0:6465:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1780298644.259537:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.259610:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609573120 00000100:00100000:3.0:1780298644.259615:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609573120 00000100:00100000:1.0:1780298644.259632:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.259673:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609573248 00000100:00100000:0.0:1780298644.259677:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609573248 00000100:00100000:2.0:1780298644.259681:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9789 00000100:00100000:2.0:1780298644.259690:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609573120:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.259697:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298644.259701:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9790 00010000:00010000:2.0:1780298644.259705:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a777b004/0xaf35132e63034d51 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3d41 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.259714:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609573248:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.259723:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298644.259730:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605008896:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.259733:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63034d58 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3d48 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.259768:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241a81880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605009024:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.259770:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb13c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.259803:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.260354:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605008896:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.260371:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a777b004/0xaf35132e63034d51 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d41 expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.260383:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a777b004/0xaf35132e63034d51 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d41 expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.260393:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830da00, rc 0) 00000100:00100000:2.0:1780298644.260417:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609573120:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 729us (879us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.260422:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9789 00000100:00100000:3.0:1780298644.260450:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241a81880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605009024:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.260464:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63034d58 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d48 expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.260474:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63034d58 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d48 expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.260480:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157c00, rc 0) 00000100:00100000:3.0:1780298644.260554:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609573248:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 845us (922us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.260563:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9790 00000100:00100000:1.0:1780298644.261419:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.261450:0:6476:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609573504 00000100:00100000:2.0:1780298644.261454:0:6476:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609573504 00000100:00100000:2.0:1780298644.261482:0:6476:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62023 00000100:00100000:2.0:1780298644.261489:0:6476:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+127:29360:x1866777609573504:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:2.0:1780298644.261523:0:6476:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298644.261536:0:6476:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c800/0xaf35132e63034d3c lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 89 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d2c expref: 127 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.261547:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298644.261550:0:6476:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298644.261684:0:6476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830ca00/0xaf35132e63034d5f lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 88 type: IBT flags: 0x40000000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.261709:0:6476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830ca00/0xaf35132e63034d5f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 88 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.261716:0:6476:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830ca00/0xaf35132e63034d5f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 88 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1780298644.261751:0:6476:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991080 00000004:00010000:2.0:1780298644.261764:0:6476:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000d184957b reply state 000000009939b931 transno 4294991081 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830ca00/0xaf35132e63034d5f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 88 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.261771:0:6476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830ca00/0xaf35132e63034d5f lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 88 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.261775:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298644.261777:0:6476:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1780298644.261778:0:6476:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1780298644.261820:0:6476:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+129:29360:x1866777609573504:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 333us (402us total) trans 4294991081 rc 0/0 00000100:00100000:2.0:1780298644.261825:0:6476:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62023 00000020:00080000:2.0:1780298644.261827:0:6476:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1780298644.262551:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.262580:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609573632 00000100:00100000:2.0:1780298644.262583:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609573632 00000100:00100000:2.0:1780298644.262614:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9791 00000100:00100000:2.0:1780298644.262622:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:29360:x1866777609573632:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.262630:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.262639:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002dc12793/0xaf35132e63034d66 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3d4f expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.262658:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e63034d27 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d1e expref: 15 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.262667:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(9 169) 00010000:00010000:2.0:1780298644.262671:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(10 169) 00010000:00010000:2.0:1780298644.262676:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002dc12793/0xaf35132e63034d66 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d4f expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.262701:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e63034d27 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d1e expref: 15 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.262714:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e63034d27 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d1e expref: 15 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.262725:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605009152:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298644.262764:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb13c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.263378:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605009152:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298644.263393:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002dc12793/0xaf35132e63034d66 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d4f expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.263404:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830d200, rc 0) 00000100:00100000:2.0:1780298644.263432:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:29360:x1866777609573632:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 811us (882us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.263440:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9791 00000100:00100000:1.0:1780298644.263651:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.263680:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609573760 00000100:00100000:3.0:1780298644.263684:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609573760 00010000:00010000:3.0:1780298644.263700:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e63034d27 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d1e expref: 16 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.263721:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15173 00000100:00100000:3.0:1780298644.263726:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b3100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609573760:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298644.263733:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298644.263743:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e63034d27 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d1e expref: 16 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.263751:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e63034d27 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d1e expref: 16 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.263762:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e63034d27 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d1e expref: 16 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.263769:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(10 169) 00010000:00010000:3.0:1780298644.263772:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(10 169) 00010000:00010000:3.0:1780298644.263775:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000002dc12793/0xaf35132e63034d66 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d4f expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.263781:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002dc12793/0xaf35132e63034d66 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d4f expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.263795:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000002dc12793/0xaf35132e63034d66 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d4f expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.263802:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241a81880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777605009280:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298644.263830:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.264451:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241a81880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777605009280:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298644.264459:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298644.264478:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b3100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2400:x1866777609573760:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 753us (828us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298644.264535:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15173 00000100:00100000:1.0:1780298644.264561:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.264612:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609573888 00000100:00100000:0.0:1780298644.264617:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609573888 00000100:00100000:0.0:1780298644.264650:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9792 00000100:00100000:0.0:1780298644.264659:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:29360:x1866777609573888:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.264668:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.264677:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e63034d6d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3d56 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.264698:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034d2e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d25 expref: 10 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.264706:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(11 122) 00010000:00010000:0.0:1780298644.264709:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(12 122) 00010000:00010000:0.0:1780298644.264713:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e63034d6d lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d56 expref: 7 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.264730:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034d2e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d25 expref: 10 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.264741:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034d2e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d25 expref: 10 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.264751:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36d4b5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605009408:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298644.264790:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.265324:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36d4b5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605009408:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298644.265336:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e63034d6d lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d56 expref: 7 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.265344:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baec00, rc 0) 00000100:00100000:0.0:1780298644.265365:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609573888:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 709us (806us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.265370:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9792 00000100:00100000:1.0:1780298644.265518:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.265548:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609574016 00000100:00100000:0.0:1780298644.265551:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609574016 00010000:00010000:0.0:1780298644.265563:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034d2e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d25 expref: 11 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.265583:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15174 00000100:00100000:0.0:1780298644.265590:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b5d050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609574016:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298644.265598:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298644.265610:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034d2e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d25 expref: 11 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.265620:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034d2e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d25 expref: 11 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.265634:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034d2e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d25 expref: 11 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.265644:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(12 122) 00010000:00010000:0.0:1780298644.265647:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(12 122) 00010000:00010000:0.0:1780298644.265652:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e63034d6d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d56 expref: 6 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.265661:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e63034d6d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d56 expref: 6 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.265683:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e63034d6d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d56 expref: 6 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.265694:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36d4b5180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:7400:1866777605009536:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298644.265727:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf348417840 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.266412:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36d4b5180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:7400:1866777605009536:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298644.266424:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298644.266444:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b5d050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609574016:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 856us (928us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.266450:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15174 00000100:00100000:2.0:1780298644.266539:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.266577:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609574144 00000100:00100000:0.0:1780298644.266582:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609574144 00002000:00010000:0.0:1780298644.266601:0:16972:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (3633916->18446744073709551615) 00002000:00100000:0.0:1780298644.266605:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fb480 with x1866777609574144 ext(3633916->18446744073709551615) 00002000:00010000:0.0:1780298644.266611:0:16972:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481fb480. 00000100:00100000:0.0:1780298644.266631:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6280 00000100:00100000:0.0:1780298644.266638:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609574144:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:1.0:1780298644.266681:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.266715:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609574272 00000100:00100000:3.0:1780298644.266720:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609574272 00002000:00010000:3.0:1780298644.266735:0:19950:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00002000:00100000:3.0:1780298644.266740:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f8b2a00 with x1866777609574272 ext(0->18446744073709551615) 00002000:00010000:3.0:1780298644.266744:0:19950:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f8b2a00. 00000100:00100000:3.0:1780298644.266763:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6281 00000100:00100000:3.0:1780298644.266769:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609574272:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:0.0:1780298644.266814:0:16972:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 3633916 00002000:00010000:0.0:1780298644.266817:0:16972:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298644.266819:0:16972:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298644.266821:0:16972:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298644.266823:0:16972:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 328 -> 328 00000100:00100000:0.0:1780298644.266847:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609574144:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 210us (310us total) trans 4294969717 rc 0/0 00000100:00100000:0.0:1780298644.266853:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6280 00002000:00010000:0.0:1780298644.266856:0:16972:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (3633916->18446744073709551615) 00002000:00100000:0.0:1780298644.266858:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fb480 with x1866777609574144 ext(3633916->18446744073709551615) 00002000:00010000:0.0:1780298644.266862:0:16972:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481fb480. 00002000:00010000:3.0:1780298644.266924:0:19950:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4113282 -> 0 00002000:00010000:3.0:1780298644.266927:0:19950:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:3.0:1780298644.266928:0:19950:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:3.0:1780298644.266960:0:19950:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:3.0:1780298644.266961:0:19950:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 48 -> 0 00000100:00100000:3.0:1780298644.266984:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609574272:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 216us (306us total) trans 4294969268 rc 0/0 00000100:00100000:3.0:1780298644.266989:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6281 00002000:00010000:3.0:1780298644.266991:0:19950:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00002000:00100000:3.0:1780298644.266994:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f8b2a00 with x1866777609574272 ext(0->18446744073709551615) 00002000:00010000:3.0:1780298644.266999:0:19950:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f8b2a00. 00000100:00100000:1.0:1780298644.267766:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.267792:0:17578:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609574400 00000100:00100000:3.0:1780298644.267796:0:17578:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609574400 00000100:00100000:3.0:1780298644.267822:0:17578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62024 00000100:00100000:3.0:1780298644.267826:0:17578:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+129:29360:x1866777609574400:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.267837:0:17578:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.267846:0:17578:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156000/0xaf35132e63034d74 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 89 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3d5d expref: -99 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.267895:0:17578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156e00/0xaf35132e63034d7b lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 90 type: IBT flags: 0x40000000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.267904:0:17578:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156e00/0xaf35132e63034d7b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 90 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.267911:0:17578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156e00/0xaf35132e63034d7b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 90 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1780298644.267926:0:17578:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 3633916 00000004:00200000:3.0:1780298644.267929:0:17578:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1780298644.267939:0:17578:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156e00/0xaf35132e63034d7b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 90 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.267954:0:17578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156000/0xaf35132e63034d74 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 90 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d5d expref: 131 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.267963:0:17578:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156e00/0xaf35132e63034d7b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 89 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d5d expref: 130 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.267968:0:17578:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298644.267970:0:17578:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1780298644.267972:0:17578:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156e00, rc 0) 00000100:00100000:3.0:1780298644.267993:0:17578:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+130:29360:x1866777609574400:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 167us (227us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298644.267999:0:17578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62024 00000100:00100000:2.0:1780298644.269238:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.269343:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609574528 00000100:00100000:0.0:1780298644.269348:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609574528 00000100:00100000:0.0:1780298644.269383:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62025 00000100:00100000:0.0:1780298644.269390:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+169:29360:x1866777609574528:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.269401:0:6477:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.269410:0:6477:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf400/0xaf35132e63034d82 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 90 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3d64 expref: -99 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.269452:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae600/0xaf35132e63034d89 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 91 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.269462:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae600/0xaf35132e63034d89 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 91 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.269470:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae600/0xaf35132e63034d89 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 91 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1780298644.269485:0:6477:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 3633916 00000004:00200000:0.0:1780298644.269488:0:6477:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1780298644.269498:0:6477:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae600/0xaf35132e63034d89 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 91 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.269514:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf400/0xaf35132e63034d82 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 91 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d64 expref: 171 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.269533:0:6477:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae600/0xaf35132e63034d89 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 90 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d64 expref: 170 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.269540:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298644.269542:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1780298644.269544:0:6477:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae600, rc 0) 00000100:00100000:0.0:1780298644.269574:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+170:29360:x1866777609574528:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 184us (336us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.269581:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62025 00000100:00100000:1.0:1780298644.270426:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.270467:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609574656 00000100:00100000:0.0:1780298644.270472:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609574656 00000100:00100000:0.0:1780298644.270505:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9793 00000100:00100000:0.0:1780298644.270513:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2400:x1866777609574656:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.270522:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.270532:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e63034d90 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3d6b expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.270570:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36d4b6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605009664:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.270618:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393b40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.270619:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.270669:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609574784 00000100:00100000:3.0:1780298644.270674:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609574784 00000100:00100000:1.0:1780298644.270729:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9794 00000100:00100000:1.0:1780298644.270741:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609574784:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.270750:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.270759:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f4922155/0xaf35132e63034d97 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3d72 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.270786:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605009792:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.270823:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.271298:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36d4b6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605009664:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.271316:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e63034d90 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d6b expref: 16 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.271332:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e63034d90 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d6b expref: 16 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.271351:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae400, rc 0) 00000100:00100000:0.0:1780298644.271388:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2400:x1866777609574656:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 876us (962us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.271395:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9793 00000100:00100000:1.0:1780298644.271679:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8b3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605009792:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.271693:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f4922155/0xaf35132e63034d97 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d72 expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.271706:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f4922155/0xaf35132e63034d97 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d72 expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.271715:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278600, rc 0) 00000100:00100000:1.0:1780298644.271743:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609574784:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1008us (1127us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.271748:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9794 00000100:00100000:2.0:1780298644.272514:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.272559:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609574912 00000100:00100000:1.0:1780298644.272564:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609574912 00000100:00100000:1.0:1780298644.272612:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9795 00000100:00100000:1.0:1780298644.272623:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2398:x1866777609574912:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.272632:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298644.272640:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:1.0:1780298644.272641:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000488d1c2a/0xaf35132e63034d9e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3d79 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.272672:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605009920:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.272678:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609575040 00000100:00100000:0.0:1780298644.272682:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609575040 00000100:00100000:0.0:1780298644.272707:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9796 00000100:00100000:0.0:1780298644.272715:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609575040:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.272722:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298644.272727:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00010000:00010000:0.0:1780298644.272730:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e63034da5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3d80 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.272758:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36d4b6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605010048:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.272791:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393b40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.273366:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8b3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605009920:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.273381:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36d4b6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605010048:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.273382:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000488d1c2a/0xaf35132e63034d9e lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d79 expref: 16 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.273399:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000488d1c2a/0xaf35132e63034d9e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d79 expref: 16 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.273405:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e63034da5 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d80 expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.273408:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279400, rc 0) 00010000:00010000:0.0:1780298644.273419:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e63034da5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d80 expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.273428:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf600, rc 0) 00000100:00100000:1.0:1780298644.273441:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2398:x1866777609574912:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 822us (926us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.273446:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9795 00000100:00100000:0.0:1780298644.273451:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609575040:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 735us (808us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.273457:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9796 00000100:00100000:2.0:1780298644.274398:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.274416:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.274431:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609575168 00000100:00100000:3.0:1780298644.274438:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609575168 00000100:00100000:0.0:1780298644.274448:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609575296 00000100:00100000:0.0:1780298644.274453:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609575296 00000100:00100000:3.0:1780298644.274472:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9797 00000100:00100000:3.0:1780298644.274483:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2400:x1866777609575168:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298644.274488:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9798 00010000:00010000:3.0:1780298644.274495:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298644.274501:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609575296:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.274506:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e63034dac lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3d87 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.274509:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.274520:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d4d8e3d7/0xaf35132e63034db3 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3d8e expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.274540:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241a81880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605010176:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.274548:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36d4b6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605010304:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.274593:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.274593:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393b40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.275321:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241a81880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605010176:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.275324:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36d4b6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605010304:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.275347:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e63034dac lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d87 expref: 16 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.275352:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d4d8e3d7/0xaf35132e63034db3 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d8e expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.275372:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e63034dac lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d87 expref: 16 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.275373:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d4d8e3d7/0xaf35132e63034db3 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d8e expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.275394:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf000, rc 0) 00010000:00010000:1.0:1780298644.275397:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156a00, rc 0) 00000100:00100000:0.0:1780298644.275420:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609575296:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 924us (1004us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.275425:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9798 00000100:00100000:1.0:1780298644.275432:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2400:x1866777609575168:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 953us (1033us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.275443:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9797 00000100:00100000:2.0:1780298644.276344:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.276381:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609575424 00000100:00100000:1.0:1780298644.276386:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609575424 00000100:00100000:1.0:1780298644.276419:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9799 00000100:00100000:1.0:1780298644.276428:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481faa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:29360:x1866777609575424:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.276436:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.276446:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63034dba lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3d95 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.276465:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000002dc12793/0xaf35132e63034d66 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d4f expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.276474:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(10 169) 00010000:00010000:1.0:1780298644.276477:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(11 169) 00010000:00010000:1.0:1780298644.276482:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63034dba lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2031616->2035711] (req 2031616->2035711) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d95 expref: 16 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.276505:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000002dc12793/0xaf35132e63034d66 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d4f expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.276520:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000002dc12793/0xaf35132e63034d66 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d4f expref: 9 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.276530:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605010432:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298644.276568:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.277139:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8b3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605010432:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298644.277151:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63034dba lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2031616->2035711] (req 2031616->2035711) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d95 expref: 16 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.277159:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278c00, rc 0) 00000100:00100000:1.0:1780298644.277180:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481faa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:29360:x1866777609575424:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 756us (837us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.277185:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9799 00000100:00100000:3.0:1780298644.277388:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.277421:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609575552 00000100:00100000:1.0:1780298644.277426:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609575552 00010000:00010000:1.0:1780298644.277442:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000002dc12793/0xaf35132e63034d66 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d4f expref: 10 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.277462:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15175 00000100:00100000:1.0:1780298644.277467:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a82a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609575552:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298644.277475:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298644.277483:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000002dc12793/0xaf35132e63034d66 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d4f expref: 10 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.277491:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000002dc12793/0xaf35132e63034d66 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d4f expref: 10 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.277503:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002dc12793/0xaf35132e63034d66 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d4f expref: 10 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.277509:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(11 169) 00010000:00010000:1.0:1780298644.277512:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(11 169) 00010000:00010000:1.0:1780298644.277517:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63034dba lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2031616->2035711) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d95 expref: 15 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.277523:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63034dba lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2031616->2035711) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d95 expref: 15 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.277537:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63034dba lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2031616->2035711) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d95 expref: 15 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.277544:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b3480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777605010560:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298644.277570:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.278157:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8b3480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777605010560:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298644.278165:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298644.278185:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a82a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609575552:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 718us (803us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.278192:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15175 00000100:00100000:2.0:1780298644.278406:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.278436:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609575680 00000100:00100000:1.0:1780298644.278439:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609575680 00002000:00100000:1.0:1780298644.278459:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (2031616->2035711) req@ffff9cf37f734e00 x1866777609575680/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.278466:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f734e00 with x1866777609575680 ext(2031616->2035711) 00002000:00010000:1.0:1780298644.278469:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f734e00 00000100:00100000:1.0:1780298644.278485:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6282 00000100:00100000:1.0:1780298644.278490:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f734e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609575680:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298644.278611:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f734e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609575680:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 121us (206us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298644.278615:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6282 00002000:00100000:1.0:1780298644.278620:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (2031616->2035711) req@ffff9cf37f734e00 x1866777609575680/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/4536 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.278626:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f734e00 with x1866777609575680 ext(2031616->2035711) 00002000:00010000:1.0:1780298644.278628:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f734e00 00000100:00100000:3.0:1780298644.279559:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.279598:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609575808 00000100:00100000:0.0:1780298644.279602:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609575808 00000100:00100000:0.0:1780298644.279628:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9800 00000100:00100000:0.0:1780298644.279634:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a82680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609575808:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.279640:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.279651:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000441cddf1/0xaf35132e63034dc1 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3d9c expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.279663:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(11 169) 00010000:00010000:0.0:1780298644.279666:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(11 169) 00010000:00010000:0.0:1780298644.279669:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000441cddf1/0xaf35132e63034dc1 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d9c expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.279677:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000441cddf1/0xaf35132e63034dc1 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d9c expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.279689:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf400, rc 0) 00000100:00100000:0.0:1780298644.279707:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a82680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609575808:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 75us (149us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.279712:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9800 00000100:00100000:2.0:1780298644.281324:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.281354:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609575936 00000100:00100000:1.0:1780298644.281359:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609575936 00000100:00100000:1.0:1780298644.281382:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9801 00000100:00100000:1.0:1780298644.281388:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f736680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609575936:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.281394:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.281401:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f4922155/0xaf35132e63034dc8 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3da3 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.281427:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605010688:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.281461:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.282018:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8b3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605010688:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.282031:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f4922155/0xaf35132e63034dc8 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3da3 expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.282041:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f4922155/0xaf35132e63034dc8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3da3 expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.282051:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278600, rc 0) 00000100:00100000:1.0:1780298644.282073:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f736680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609575936:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 686us (752us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.282078:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9801 00000100:00100000:3.0:1780298644.282748:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.282780:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609576064 00000100:00100000:1.0:1780298644.282784:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609576064 00000100:00100000:1.0:1780298644.282812:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9802 00000100:00100000:1.0:1780298644.282820:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a81880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609576064:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.282828:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.282836:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e63034dcf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3daa expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.282852:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e63034d6d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d56 expref: 7 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.282859:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(12 122) 00010000:00010000:1.0:1780298644.282863:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(13 122) 00010000:00010000:1.0:1780298644.282866:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e63034dcf lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3584000->3588095] (req 3584000->3588095) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3daa expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.282883:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e63034d6d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d56 expref: 7 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.282894:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e63034d6d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d56 expref: 7 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.282906:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605010816:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298644.282942:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.283479:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8b3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605010816:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298644.283488:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e63034dcf lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3584000->3588095] (req 3584000->3588095) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3daa expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.283496:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279a00, rc 0) 00000100:00100000:1.0:1780298644.283519:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a81880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:29360:x1866777609576064:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 700us (770us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.283525:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9802 00000100:00100000:2.0:1780298644.283711:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.283751:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609576192 00000100:00100000:1.0:1780298644.283757:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609576192 00010000:00010000:1.0:1780298644.283775:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e63034d6d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d56 expref: 8 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.283804:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15176 00000100:00100000:1.0:1780298644.283813:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f737b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609576192:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298644.283822:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298644.283832:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e63034d6d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d56 expref: 8 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.283840:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e63034d6d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d56 expref: 8 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.283854:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e63034d6d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d56 expref: 8 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.283861:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(13 122) 00010000:00010000:1.0:1780298644.283863:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(13 122) 00010000:00010000:1.0:1780298644.283867:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e63034dcf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3584000->37138431] (req 3584000->3588095) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3daa expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.283872:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e63034dcf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3584000->37138431] (req 3584000->3588095) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3daa expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.283886:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e63034dcf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3584000->37138431] (req 3584000->3588095) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3daa expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.283892:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b3480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:8374:1866777605010944:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298644.283921:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.284600:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8b3480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:8374:1866777605010944:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298644.284611:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298644.284640:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f737b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609576192:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 828us (929us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.284647:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15176 00000100:00100000:2.0:1780298644.284781:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.284804:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609576320 00000100:00100000:3.0:1780298644.284807:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609576320 00000100:00100000:3.0:1780298644.284833:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6283 00000100:00100000:3.0:1780298644.284838:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f736300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609576320:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:3.0:1780298644.284965:0:19951:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:3.0:1780298644.284968:0:19951:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:3.0:1780298644.284969:0:19951:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:3.0:1780298644.284971:0:19951:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:3.0:1780298644.284972:0:19951:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 0 -> 8 00000100:00100000:3.0:1780298644.284996:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f736300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2399:x1866777609576320:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 157us (217us total) trans 4294969269 rc 0/0 00000100:00100000:3.0:1780298644.285003:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6283 00000100:00100000:3.0:1780298644.285952:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.285983:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609576448 00000100:00100000:1.0:1780298644.285986:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609576448 00000100:00100000:1.0:1780298644.286009:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9803 00000100:00100000:1.0:1780298644.286013:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a83b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609576448:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.286019:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.286026:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034dd6 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3db1 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.286047:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605011072:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.286083:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.286736:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8b3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605011072:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.286747:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034dd6 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3db1 expref: 13 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.286756:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034dd6 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3db1 expref: 13 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.286762:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279400, rc 0) 00000100:00100000:1.0:1780298644.286783:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a83b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609576448:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 770us (833us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.286787:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9803 00000100:00100000:2.0:1780298644.287666:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.287693:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609576576 00000100:00100000:0.0:1780298644.287697:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609576576 00000100:00100000:0.0:1780298644.287720:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9804 00000100:00100000:0.0:1780298644.287725:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243844700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609576576:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.287731:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.287738:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e63034ddd lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3db8 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.287762:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36d4b6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605011200:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.287790:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393b40 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.288443:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36d4b6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605011200:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.288461:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e63034ddd lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3db8 expref: 13 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.288470:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e63034ddd lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3db8 expref: 13 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.288477:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf600, rc 0) 00000100:00100000:0.0:1780298644.288498:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243844700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609576576:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 773us (833us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.288502:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9804 00000100:00100000:3.0:1780298644.289307:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.289349:0:7411:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609576832 00000100:00100000:2.0:1780298644.289354:0:7411:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609576832 00000100:00100000:2.0:1780298644.289388:0:7411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62026 00000100:00100000:2.0:1780298644.289397:0:7411:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a80380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+130:29360:x1866777609576832:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:2.0:1780298644.289435:0:7411:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298644.289452:0:7411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156e00/0xaf35132e63034d7b lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 91 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d5d expref: 130 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.289466:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298644.289470:0:7411:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298644.289598:0:7411:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c800/0xaf35132e63034de4 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 90 type: IBT flags: 0x40000000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.289612:0:7411:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae600/0xaf35132e63034d89 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 90 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d64 expref: 169 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.289633:0:7411:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c800/0xaf35132e63034de4 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 90 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.289653:0:7411:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae600/0xaf35132e63034d89 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 90 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d64 expref: 169 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.289666:0:7411:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae600/0xaf35132e63034d89 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 90 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d64 expref: 169 pid: 6477 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.289676:0:7411:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243844a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7411:1866777605011328:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298644.289713:0:7411:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.290297:0:7411:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243844a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7411:1866777605011328:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298644.290314:0:7411:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c800/0xaf35132e63034de4 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 90 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.290418:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.290445:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609576960 00000100:00100000:2.0:1780298644.290449:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609576960 00010000:00010000:2.0:1780298644.290465:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae600/0xaf35132e63034d89 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 90 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d64 expref: 170 pid: 6477 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.290484:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15177 00000100:00100000:2.0:1780298644.290490:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+170:2400:x1866777609576960:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298644.290496:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298644.290500:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae600/0xaf35132e63034d89 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 91 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d64 expref: 170 pid: 6477 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.290507:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae600/0xaf35132e63034d89 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 91 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d64 expref: 170 pid: 6477 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.290518:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae600/0xaf35132e63034d89 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 91 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d64 expref: 170 pid: 6477 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.290526:0:20253:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298644.290529:0:20253:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c800/0xaf35132e63034de4 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 90 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.290537:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c800/0xaf35132e63034de4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 90 type: IBT flags: 0x40290000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.290542:0:20253:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c800/0xaf35132e63034de4 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 90 type: IBT flags: 0x40290000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.290556:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298644.290574:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+169:2400:x1866777609576960:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 85us (158us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.290580:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15177 00010000:00010000:1.0:1780298644.290583:0:7411:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c800/0xaf35132e63034de4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 89 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.290593:0:7411:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c800/0xaf35132e63034de4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 89 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1780298644.290646:0:7411:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991082 00000004:00010000:1.0:1780298644.290668:0:7411:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000d5a128d3 reply state 00000000c986cdad transno 4294991083 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c800/0xaf35132e63034de4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 89 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.290679:0:7411:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c800/0xaf35132e63034de4 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 89 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.290685:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.290688:0:7411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1780298644.290691:0:7411:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1780298644.290748:0:7411:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a80380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+132:29360:x1866777609576832:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1352us (1440us total) trans 4294991083 rc 0/0 00000100:00100000:1.0:1780298644.290755:0:7411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62026 00000020:00080000:1.0:1780298644.290759:0:7411:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1780298644.291368:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.291403:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609577088 00000100:00100000:1.0:1780298644.291407:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609577088 00000100:00100000:1.0:1780298644.291437:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9805 00000100:00100000:1.0:1780298644.291445:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e4f2450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:29360:x1866777609577088:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.291453:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.291462:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63034deb lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3dbf expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.291479:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63034dba lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2031616->2035711) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d95 expref: 15 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.291489:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000441cddf1/0xaf35132e63034dc1 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d9c expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.291508:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(11 169) 00010000:00010000:1.0:1780298644.291512:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(11 169) 00010000:00010000:1.0:1780298644.291517:0:20252:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63034deb lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3dbf expref: 11 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.291534:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000441cddf1/0xaf35132e63034dc1 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d9c expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.291544:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000441cddf1/0xaf35132e63034dc1 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d9c expref: 11 pid: 8358 timeout: 933 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.291555:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605011456:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298644.291596:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63034dba lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2031616->2035711) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d95 expref: 15 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.291607:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63034dba lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2031616->2035711) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d95 expref: 15 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.291618:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605011584:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298644.291642:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.292172:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8b3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605011456:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298644.292470:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.292485:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8b1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605011584:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298644.292506:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:1.0:1780298644.292511:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63034deb lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3dbf expref: 11 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.292519:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609577216 00000100:00100000:2.0:1780298644.292525:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609577216 00010000:00010000:1.0:1780298644.292525:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279c00, rc 0) 00000100:00100000:0.0:1780298644.292539:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609577344 00000100:00100000:0.0:1780298644.292543:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609577344 00010000:00010000:2.0:1780298644.292548:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000441cddf1/0xaf35132e63034dc1 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d9c expref: 12 pid: 8358 timeout: 933 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.292560:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e4f2450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:29360:x1866777609577088:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1114us (1190us total) trans 0 rc 0/0 00010000:00010000:0.0:1780298644.292560:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63034dba lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2031616->2035711) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d95 expref: 16 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.292577:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9805 00000100:00100000:2.0:1780298644.292584:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15178 00000100:00100000:2.0:1780298644.292594:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609577216:12345-192.168.203.25@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1780298644.292601:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15179 00010000:00010000:2.0:1780298644.292604:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:0.0:1780298644.292613:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609577344:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298644.292620:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000441cddf1/0xaf35132e63034dc1 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d9c expref: 11 pid: 8358 timeout: 933 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.292634:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298644.292639:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000441cddf1/0xaf35132e63034dc1 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d9c expref: 11 pid: 8358 timeout: 933 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.292651:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63034dba lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2031616->2035711) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d95 expref: 16 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.292656:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000441cddf1/0xaf35132e63034dc1 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d9c expref: 11 pid: 8358 timeout: 933 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.292684:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63034dba lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2031616->2035711) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d95 expref: 16 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.292692:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1780298644.292699:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63034dba lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2031616->2035711) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3d95 expref: 16 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.292709:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(11 169) 00010000:00010000:0.0:1780298644.292712:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(11 169) 00010000:00010000:0.0:1780298644.292716:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63034deb lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3dbf expref: 10 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.292725:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63034deb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3dbf expref: 10 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.292726:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609577216:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 132us (255us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.292740:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15178 00010000:00010000:0.0:1780298644.292751:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63034deb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3dbf expref: 9 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.292760:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36d4b6300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777605011712:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298644.292798:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393b40 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.293351:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36d4b6300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777605011712:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298644.293359:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298644.293371:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.293386:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2400:x1866777609577344:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 776us (882us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.293398:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15179 00000100:00100000:1.0:1780298644.293407:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609577472 00000100:00100000:1.0:1780298644.293412:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609577472 00000100:00100000:1.0:1780298644.293441:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9806 00000100:00100000:1.0:1780298644.293449:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609577472:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.293457:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.293466:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f4922155/0xaf35132e63034df2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3dc6 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.293482:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e63034dcf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3584000->37138431] (req 3584000->3588095) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3daa expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.293489:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(13 122) 00010000:00010000:1.0:1780298644.293492:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(14 122) 00010000:00010000:1.0:1780298644.293496:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f4922155/0xaf35132e63034df2 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3dc6 expref: 8 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.293511:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e63034dcf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3584000->37138431] (req 3584000->3588095) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3daa expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.293527:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e63034dcf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3584000->37138431] (req 3584000->3588095) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3daa expref: 11 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.293536:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605011840:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298644.293570:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.294044:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8b3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605011840:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298644.294058:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f4922155/0xaf35132e63034df2 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3dc6 expref: 8 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.294069:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278600, rc 0) 00000100:00100000:1.0:1780298644.294098:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:29360:x1866777609577472:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 650us (727us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.294104:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9806 00000100:00100000:3.0:1780298644.294220:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.294265:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609577600 00000100:00100000:0.0:1780298644.294269:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609577600 00010000:00010000:0.0:1780298644.294290:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e63034dcf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3584000->37138431] (req 3584000->3588095) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3daa expref: 12 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.294308:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15180 00000100:00100000:0.0:1780298644.294313:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2399:x1866777609577600:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298644.294319:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298644.294326:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e63034dcf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3584000->37138431] (req 3584000->3588095) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3daa expref: 12 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.294333:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e63034dcf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3584000->37138431] (req 3584000->3588095) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3daa expref: 12 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.294342:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e63034dcf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3584000->37138431] (req 3584000->3588095) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3daa expref: 12 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.294349:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(14 122) 00010000:00010000:0.0:1780298644.294351:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(14 122) 00010000:00010000:0.0:1780298644.294354:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000f4922155/0xaf35132e63034df2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3dc6 expref: 7 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.294360:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f4922155/0xaf35132e63034df2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3dc6 expref: 7 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.294375:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000f4922155/0xaf35132e63034df2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3dc6 expref: 7 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.294381:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36d4b6300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:7400:1866777605011968:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298644.294413:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393b40 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.294996:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36d4b6300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:7400:1866777605011968:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298644.295007:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298644.295032:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609577600:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 719us (813us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.295040:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15180 00000100:00100000:3.0:1780298644.295173:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.295208:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609577728 00000100:00100000:0.0:1780298644.295213:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609577728 00002000:00010000:0.0:1780298644.295232:0:19943:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (1725856->18446744073709551615) 00002000:00100000:0.0:1780298644.295237:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24339ea00 with x1866777609577728 ext(1725856->18446744073709551615) 00000100:00100000:2.0:1780298644.295243:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00002000:00010000:0.0:1780298644.295258:0:19943:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24339ea00. 00000100:00100000:3.0:1780298644.295290:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609577856 00000100:00100000:3.0:1780298644.295295:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609577856 00000100:00100000:0.0:1780298644.295296:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6284 00000100:00100000:0.0:1780298644.295308:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609577728:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:3.0:1780298644.295313:0:19949:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00002000:00100000:3.0:1780298644.295317:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243846a00 with x1866777609577856 ext(0->18446744073709551615) 00002000:00010000:3.0:1780298644.295323:0:19949:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf243846a00. 00000100:00100000:3.0:1780298644.295346:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6285 00000100:00100000:3.0:1780298644.295354:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243846a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609577856:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:3.0:1780298644.295434:0:19949:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:3.0:1780298644.295438:0:19949:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:3.0:1780298644.295441:0:19949:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:3.0:1780298644.295444:0:19949:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:3.0:1780298644.295446:0:19949:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 8 -> 8 00000100:00100000:3.0:1780298644.295479:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243846a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609577856:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 126us (238us total) trans 4294969270 rc 0/0 00002000:00010000:0.0:1780298644.295484:0:19943:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 3633916 -> 1725856 00000100:00100000:3.0:1780298644.295489:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6285 00002000:00010000:0.0:1780298644.295489:0:19943:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298644.295493:0:19943:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:3.0:1780298644.295494:0:19949:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00002000:00010000:0.0:1780298644.295495:0:19943:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00100000:3.0:1780298644.295499:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243846a00 with x1866777609577856 ext(0->18446744073709551615) 00002000:00010000:0.0:1780298644.295500:0:19943:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 328 -> 328 00002000:00010000:3.0:1780298644.295506:0:19949:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf243846a00. 00000100:00100000:0.0:1780298644.295524:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609577728:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 219us (401us total) trans 4294969718 rc 0/0 00000100:00100000:0.0:1780298644.295531:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6284 00002000:00010000:0.0:1780298644.295534:0:19943:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (1725856->18446744073709551615) 00002000:00100000:0.0:1780298644.295536:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24339ea00 with x1866777609577728 ext(1725856->18446744073709551615) 00002000:00010000:0.0:1780298644.295540:0:19943:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24339ea00. 00000100:00100000:1.0:1780298644.296216:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.296250:0:6476:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609577984 00000100:00100000:2.0:1780298644.296255:0:6476:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609577984 00000100:00100000:2.0:1780298644.296301:0:6476:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62027 00000100:00100000:2.0:1780298644.296310:0:6476:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+132:29360:x1866777609577984:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.296323:0:6476:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.296332:0:6476:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c600/0xaf35132e63034df9 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 90 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3dcd expref: -99 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.296388:0:6476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830d200/0xaf35132e63034e00 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 91 type: IBT flags: 0x40000000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.296399:0:6476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830d200/0xaf35132e63034e00 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 91 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.296406:0:6476:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830d200/0xaf35132e63034e00 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 91 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298644.296426:0:6476:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 1725856 00000004:00200000:2.0:1780298644.296430:0:6476:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298644.296444:0:6476:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830d200/0xaf35132e63034e00 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 91 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.296465:0:6476:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c600/0xaf35132e63034df9 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 91 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3dcd expref: 134 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.296479:0:6476:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830d200/0xaf35132e63034e00 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 90 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3dcd expref: 133 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.296485:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298644.296488:0:6476:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298644.296491:0:6476:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf34830d200, rc 0) 00000100:00100000:2.0:1780298644.296521:0:6476:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+133:29360:x1866777609577984:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 213us (305us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.296528:0:6476:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62027 00000100:00100000:2.0:1780298644.297476:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.297517:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609578112 00000100:00100000:3.0:1780298644.297522:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609578112 00000100:00100000:3.0:1780298644.297548:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6286 00000100:00100000:3.0:1780298644.297553:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243847480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609578112:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:3.0:1780298644.297614:0:19950:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 1725856 -> 1725856 00002000:00010000:3.0:1780298644.297617:0:19950:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:3.0:1780298644.297618:0:19950:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:3.0:1780298644.297620:0:19950:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:3.0:1780298644.297621:0:19950:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 328 -> 328 00000100:00100000:3.0:1780298644.297643:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243847480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609578112:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 90us (169us total) trans 4294969719 rc 0/0 00000100:00100000:3.0:1780298644.297647:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6286 00000100:00100000:0.0:1780298644.301230:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.301289:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609578240 00000100:00100000:1.0:1780298644.301295:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609578240 00002000:00100000:1.0:1780298644.301322:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (0->1728511) req@ffff9cf36d4b6300 x1866777609578240/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.301331:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36d4b6300 with x1866777609578240 ext(0->1728511) 00002000:00010000:1.0:1780298644.301336:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf36d4b6300 00000100:00100000:1.0:1780298644.301356:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6287 00000100:00100000:1.0:1780298644.301364:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609578240:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298644.306773:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609578240:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 5398us (5533us total) trans 0 rc 1728512/1728512 00000100:00100000:0.0:1780298644.306786:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6287 00002000:00100000:0.0:1780298644.306802:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (0->1728511) req@ffff9cf36d4b6300 x1866777609578240/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/440 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 1728512/1728512 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298644.306813:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36d4b6300 with x1866777609578240 ext(0->1728511) 00002000:00010000:0.0:1780298644.306819:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf36d4b6300 00000100:00100000:1.0:1780298644.309906:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.309949:0:17578:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609578496 00000100:00100000:3.0:1780298644.309960:0:17578:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609578496 00000100:00100000:3.0:1780298644.310019:0:17578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62028 00000100:00100000:3.0:1780298644.310031:0:17578:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+169:29360:x1866777609578496:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:3.0:1780298644.310292:0:17578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156000/0xaf35132e63034e07 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 91 type: IBT flags: 0x40000000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.310312:0:17578:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830d200/0xaf35132e63034e00 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 91 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3dcd expref: 132 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.310334:0:17578:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156000/0xaf35132e63034e07 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 91 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.310369:0:17578:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830d200/0xaf35132e63034e00 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 91 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3dcd expref: 132 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.310390:0:17578:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830d200/0xaf35132e63034e00 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 91 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3dcd expref: 132 pid: 6476 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.310404:0:17578:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24339d180 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000_UUID:17578:1866777605012096:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298644.310489:0:17578:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.311141:0:17578:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24339d180 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000_UUID:17578:1866777605012096:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298644.311162:0:17578:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156000/0xaf35132e63034e07 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 91 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.311318:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.311352:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609578624 00000100:00100000:3.0:1780298644.311356:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609578624 00010000:00010000:3.0:1780298644.311412:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830d200/0xaf35132e63034e00 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 91 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3dcd expref: 133 pid: 6476 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.311436:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15181 00000100:00100000:3.0:1780298644.311444:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b5dc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+133:2400:x1866777609578624:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298644.311454:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298644.311461:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830d200/0xaf35132e63034e00 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 92 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3dcd expref: 133 pid: 6476 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.311470:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830d200/0xaf35132e63034e00 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 92 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3dcd expref: 133 pid: 6476 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.311485:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830d200/0xaf35132e63034e00 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 92 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3dcd expref: 133 pid: 6476 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.311499:0:6461:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298644.311502:0:6461:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156000/0xaf35132e63034e07 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 91 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.311511:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156000/0xaf35132e63034e07 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 91 type: IBT flags: 0x40290000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.311516:0:6461:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156000/0xaf35132e63034e07 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 91 type: IBT flags: 0x40290000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.311532:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298644.311554:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b5dc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+132:2400:x1866777609578624:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 113us (236us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298644.311559:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15181 00010000:00010000:1.0:1780298644.311562:0:17578:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156000/0xaf35132e63034e07 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 90 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.311571:0:17578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156000/0xaf35132e63034e07 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 90 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1780298644.311624:0:17578:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991084 00000004:00010000:1.0:1780298644.311646:0:17578:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000b61037b4 reply state 000000001b8ab475 transno 4294991085 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156000/0xaf35132e63034e07 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 90 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.311657:0:17578:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156000/0xaf35132e63034e07 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 90 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.311661:0:17578:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.311663:0:17578:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1780298644.311665:0:17578:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1780298644.311719:0:17578:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+172:29360:x1866777609578496:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1690us (1821us total) trans 4294991085 rc 0/0 00000100:00100000:1.0:1780298644.311727:0:17578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62028 00000020:00080000:1.0:1780298644.311731:0:17578:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1780298644.312372:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.312426:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609578752 00000100:00100000:2.0:1780298644.312431:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609578752 00000100:00100000:2.0:1780298644.312467:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9807 00000100:00100000:2.0:1780298644.312475:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:29360:x1866777609578752:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.312486:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.312500:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63034e0e lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3dd4 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.312534:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63034deb lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3dbf expref: 11 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.312544:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(11 169) 00010000:00010000:2.0:1780298644.312548:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(12 169) 00010000:00010000:2.0:1780298644.312552:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63034e0e lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3dd4 expref: 16 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.312572:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63034deb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3dbf expref: 11 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.312584:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63034deb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3dbf expref: 11 pid: 20252 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.312594:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243845c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605012224:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298644.312635:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.313316:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243845c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605012224:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298644.313335:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63034e0e lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3dd4 expref: 16 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.313349:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3000, rc 0) 00000100:00100000:1.0:1780298644.313379:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:29360:x1866777609578752:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 905us (1008us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.313390:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9807 00000100:00100000:3.0:1780298644.314942:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.314985:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609578880 00000100:00100000:2.0:1780298644.314990:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609578880 00010000:00010000:2.0:1780298644.315009:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63034deb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3dbf expref: 12 pid: 20252 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.315030:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15182 00000100:00100000:2.0:1780298644.315036:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609578880:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298644.315044:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298644.315058:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63034deb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3dbf expref: 12 pid: 20252 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.315071:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63034deb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3dbf expref: 12 pid: 20252 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.315084:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63034deb lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3dbf expref: 12 pid: 20252 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.315094:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(12 169) 00010000:00010000:2.0:1780298644.315096:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(12 169) 00010000:00010000:2.0:1780298644.315101:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63034e0e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3dd4 expref: 15 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.315107:0:6463:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63034e0e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3dd4 expref: 15 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.315139:0:6463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63034e0e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3dd4 expref: 15 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.315148:0:6463:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243845c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6463:1866777605012352:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298644.315182:0:6463:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb16c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.315918:0:6463:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243845c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6463:1866777605012352:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298644.315935:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298644.315963:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609578880:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 926us (1021us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.315975:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15182 00000100:00100000:3.0:1780298644.316053:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.316084:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609579008 00000100:00100000:2.0:1780298644.316089:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609579008 00000100:00100000:2.0:1780298644.316150:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9808 00000100:00100000:2.0:1780298644.316157:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:29360:x1866777609579008:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.316167:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.316178:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e63034e15 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3ddb expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.316193:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000f4922155/0xaf35132e63034df2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3dc6 expref: 8 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.316199:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(14 122) 00010000:00010000:2.0:1780298644.316202:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(15 122) 00010000:00010000:2.0:1780298644.316206:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e63034e15 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ddb expref: 12 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.316219:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000f4922155/0xaf35132e63034df2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3dc6 expref: 8 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.316228:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000f4922155/0xaf35132e63034df2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3dc6 expref: 8 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.316235:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243845c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605012480:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298644.316293:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb16c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.316910:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243845c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605012480:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298644.316925:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e63034e15 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ddb expref: 12 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.316938:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2600, rc 0) 00000100:00100000:2.0:1780298644.316964:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:29360:x1866777609579008:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 809us (914us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.316969:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9808 00000100:00100000:0.0:1780298644.317092:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.317194:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609579136 00000100:00100000:3.0:1780298644.317199:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609579136 00010000:00010000:2.0:1780298644.317238:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000f4922155/0xaf35132e63034df2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3dc6 expref: 9 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.317265:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15183 00000100:00100000:2.0:1780298644.317272:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b6680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609579136:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298644.317292:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298644.317306:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000f4922155/0xaf35132e63034df2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3dc6 expref: 9 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.317317:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000f4922155/0xaf35132e63034df2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3dc6 expref: 9 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.317334:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f4922155/0xaf35132e63034df2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3dc6 expref: 9 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.317345:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(15 122) 00010000:00010000:2.0:1780298644.317349:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(15 122) 00010000:00010000:2.0:1780298644.317356:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e63034e15 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ddb expref: 11 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.317365:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e63034e15 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ddb expref: 11 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.317385:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e63034e15 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ddb expref: 11 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.317395:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243845c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:20253:1866777605012608:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298644.317428:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb16c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.318236:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243845c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:20253:1866777605012608:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298644.318250:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298644.318289:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.318296:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b6680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609579136:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1022us (1204us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.318302:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15183 00000100:00100000:1.0:1780298644.318452:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609579264 00000100:00100000:1.0:1780298644.318457:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609579264 00000100:00100000:3.0:1780298644.318476:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00002000:00010000:1.0:1780298644.318483:0:19951:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4709602->18446744073709551615) 00002000:00100000:1.0:1780298644.318489:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24339f100 with x1866777609579264 ext(4709602->18446744073709551615) 00002000:00010000:1.0:1780298644.318497:0:19951:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24339f100. 00000100:00100000:1.0:1780298644.318527:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609579392 00000100:00100000:1.0:1780298644.318531:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609579392 00000100:00100000:0.0:1780298644.318542:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6288 00002000:00010000:1.0:1780298644.318545:0:19951:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4194304->18446744073709551615) 00002000:00100000:1.0:1780298644.318549:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24339f480 with x1866777609579392 ext(4194304->18446744073709551615) 00002000:00010000:1.0:1780298644.318553:0:19951:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24339f480. 00000100:00100000:0.0:1780298644.318555:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609579264:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:1.0:1780298644.318576:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6289 00000100:00100000:1.0:1780298644.318587:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609579392:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:1.0:1780298644.318715:0:19951:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 0 -> 4194304 00002000:00010000:1.0:1780298644.318719:0:19951:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298644.318720:0:8362:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 1725856 -> 4709602 00002000:00010000:1.0:1780298644.318722:0:19951:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298644.318726:0:8362:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298644.318727:0:19951:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298644.318734:0:19951:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 8 -> 8 00002000:00010000:0.0:1780298644.318734:0:8362:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298644.318741:0:8362:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298644.318743:0:8362:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 328 -> 328 00000100:00100000:1.0:1780298644.318775:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609579392:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 188us (297us total) trans 4294969271 rc 0/0 00000100:00100000:0.0:1780298644.318775:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609579264:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 223us (498us total) trans 4294969720 rc 0/0 00000100:00100000:1.0:1780298644.318791:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6289 00002000:00010000:1.0:1780298644.318795:0:19951:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4194304->18446744073709551615) 00002000:00100000:1.0:1780298644.318798:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24339f480 with x1866777609579392 ext(4194304->18446744073709551615) 00000100:00100000:0.0:1780298644.318803:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6288 00002000:00010000:1.0:1780298644.318805:0:19951:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24339f480. 00002000:00010000:0.0:1780298644.318808:0:8362:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4709602->18446744073709551615) 00002000:00100000:0.0:1780298644.318812:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24339f100 with x1866777609579264 ext(4709602->18446744073709551615) 00002000:00010000:0.0:1780298644.318818:0:8362:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24339f100. 00000100:00100000:0.0:1780298644.319644:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.319684:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609579520 00000100:00100000:2.0:1780298644.319689:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609579520 00000100:00100000:2.0:1780298644.319718:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62029 00000100:00100000:2.0:1780298644.319723:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+172:29360:x1866777609579520:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.319735:0:6477:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.319747:0:6477:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3600/0xaf35132e63034e1c lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 91 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3de2 expref: -99 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.319805:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3e00/0xaf35132e63034e23 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 92 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.319819:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3e00/0xaf35132e63034e23 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 92 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.319826:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3e00/0xaf35132e63034e23 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 92 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298644.319843:0:6477:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 8903906 00000004:00200000:2.0:1780298644.319846:0:6477:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298644.319858:0:6477:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3e00/0xaf35132e63034e23 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 92 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.319873:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3600/0xaf35132e63034e1c lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 92 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3de2 expref: 174 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.319884:0:6477:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3e00/0xaf35132e63034e23 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 91 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3de2 expref: 173 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.319890:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298644.319892:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298644.319893:0:6477:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3e00, rc 0) 00000100:00100000:2.0:1780298644.319915:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+173:29360:x1866777609579520:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 192us (273us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.319919:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62029 00000100:00100000:2.0:1780298644.321840:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.321885:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609579648 00000100:00100000:0.0:1780298644.321890:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609579648 00000100:00100000:0.0:1780298644.321929:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6290 00000100:00100000:0.0:1780298644.321938:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243845c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609579648:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298644.322082:0:19943:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4709602 -> 4709602 00002000:00010000:0.0:1780298644.322085:0:19943:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298644.322087:0:19943:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298644.322089:0:19943:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298644.322090:0:19943:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 328 -> 344 00000100:00100000:0.0:1780298644.322144:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243845c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609579648:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 186us (285us total) trans 4294969721 rc 0/0 00000100:00100000:0.0:1780298644.322151:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6290 00000100:00100000:3.0:1780298644.322920:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.322969:0:7411:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609579776 00000100:00100000:1.0:1780298644.322974:0:7411:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609579776 00000100:00100000:1.0:1780298644.323015:0:7411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62030 00000100:00100000:1.0:1780298644.323025:0:7411:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+132:29360:x1866777609579776:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.323037:0:7411:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.323052:0:7411:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279400/0xaf35132e63034e2a lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 92 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3de9 expref: -99 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.323115:0:7411:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278400/0xaf35132e63034e31 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 93 type: IBT flags: 0x40000000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.323147:0:7411:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278400/0xaf35132e63034e31 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 93 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.323158:0:7411:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278400/0xaf35132e63034e31 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 93 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1780298644.323181:0:7411:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 8903906 00000004:00200000:1.0:1780298644.323185:0:7411:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1780298644.323200:0:7411:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278400/0xaf35132e63034e31 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 93 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.323226:0:7411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279400/0xaf35132e63034e2a lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 93 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3de9 expref: 134 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.323242:0:7411:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278400/0xaf35132e63034e31 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 92 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3de9 expref: 133 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.323250:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.323253:0:7411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1780298644.323256:0:7411:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278400, rc 0) 00000100:00100000:1.0:1780298644.323301:0:7411:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+133:29360:x1866777609579776:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 278us (380us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.323308:0:7411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62030 00000100:00100000:2.0:1780298644.324146:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.324189:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.324208:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609579904 00000100:00100000:1.0:1780298644.324214:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609579904 00000100:00100000:0.0:1780298644.324235:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609580032 00000100:00100000:0.0:1780298644.324240:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609580032 00000100:00100000:1.0:1780298644.324253:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9809 00000100:00100000:1.0:1780298644.324266:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243844700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609579904:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.324276:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298644.324292:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9810 00010000:00010000:1.0:1780298644.324298:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e63034e38 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3df0 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.324304:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243844a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609580032:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.324318:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.324329:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63034e3f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3df7 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.324355:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605012736:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.324371:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36d4b5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605012864:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.324416:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a540 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.324416:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393b40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.325247:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8b1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605012736:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.325249:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36d4b5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605012864:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.325288:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63034e3f lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3df7 expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.325294:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e63034e38 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3df0 expref: 12 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.325309:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63034e3f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3df7 expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.325314:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e63034e38 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3df0 expref: 12 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.325320:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae000, rc 0) 00010000:00010000:1.0:1780298644.325330:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279800, rc 0) 00000100:00100000:1.0:1780298644.325356:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243844700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609579904:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1090us (1209us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.325356:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243844a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609580032:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1054us (1164us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.325369:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9809 00000100:00100000:0.0:1780298644.325380:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9810 00000100:00100000:1.0:1780298644.326316:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.326337:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.326352:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609580160 00000100:00100000:2.0:1780298644.326356:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609580160 00000100:00100000:2.0:1780298644.326383:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609580288 00000100:00100000:2.0:1780298644.326385:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609580288 00000100:00100000:3.0:1780298644.326397:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9811 00000100:00100000:3.0:1780298644.326410:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609580160:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298644.326419:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9812 00010000:00010000:3.0:1780298644.326420:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298644.326431:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609580288:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.326433:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000042c16ab0/0xaf35132e63034e46 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3dfe expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.326442:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.326453:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63034e4d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3e05 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.326478:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24339c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605012992:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.326492:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605013120:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.326532:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.326533:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.327333:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24339c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605012992:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.327338:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605013120:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.327360:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000042c16ab0/0xaf35132e63034e46 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3dfe expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.327364:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63034e4d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e05 expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.327386:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63034e4d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e05 expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.327392:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000042c16ab0/0xaf35132e63034e46 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3dfe expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.327394:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2800, rc 0) 00010000:00010000:3.0:1780298644.327408:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156800, rc 0) 00000100:00100000:1.0:1780298644.327424:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609580288:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 997us (1085us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.327429:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9812 00000100:00100000:3.0:1780298644.327436:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609580160:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1028us (1118us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.327445:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9811 00000100:00100000:0.0:1780298644.328448:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.328491:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609580416 00000100:00100000:3.0:1780298644.328498:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609580416 00000100:00100000:2.0:1780298644.328530:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.328535:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9813 00000100:00100000:3.0:1780298644.328546:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609580416:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298644.328553:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609580544 00010000:00010000:3.0:1780298644.328557:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298644.328559:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609580544 00010000:00010000:3.0:1780298644.328569:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034e54 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3e0c expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.328595:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9814 00000100:00100000:1.0:1780298644.328604:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609580544:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.328612:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.328621:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63034e5b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3e13 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.328650:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24339c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605013248:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.328658:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605013376:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.328706:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.328707:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a540 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.329370:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24339c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605013248:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.329388:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034e54 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e0c expref: 12 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.329404:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034e54 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e0c expref: 12 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.329416:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157c00, rc 0) 00000100:00100000:1.0:1780298644.329443:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8b1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605013376:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.329454:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63034e5b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e13 expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.329479:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63034e5b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e13 expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.329493:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278a00, rc 0) 00000100:00100000:3.0:1780298644.329497:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609580416:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 950us (1047us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.329505:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9813 00000100:00100000:1.0:1780298644.329520:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609580544:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 916us (989us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.329527:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9814 00000100:00100000:0.0:1780298644.330452:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.330488:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609580672 00000100:00100000:2.0:1780298644.330493:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609580672 00000100:00100000:2.0:1780298644.330521:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9815 00000100:00100000:2.0:1780298644.330530:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:29360:x1866777609580672:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.330538:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.330546:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e63034e62 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3e1a expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.330566:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e63034e15 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ddb expref: 12 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.330575:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(15 122) 00010000:00010000:2.0:1780298644.330580:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(16 122) 00010000:00010000:2.0:1780298644.330584:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e63034e62 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1478656->1482751] (req 1478656->1482751) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e1a expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.330612:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e63034e15 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ddb expref: 12 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.330626:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e63034e15 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ddb expref: 12 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.330636:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605013504:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298644.330675:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb16c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.331294:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605013504:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298644.331308:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e63034e62 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1478656->1482751] (req 1478656->1482751) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e1a expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.331317:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2e00, rc 0) 00000100:00100000:0.0:1780298644.331342:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:29360:x1866777609580672:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 815us (892us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.331347:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9815 00000100:00100000:0.0:1780298644.331550:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.331589:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609580800 00000100:00100000:1.0:1780298644.331594:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609580800 00010000:00010000:1.0:1780298644.331616:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e63034e15 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ddb expref: 13 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.331647:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15184 00000100:00100000:1.0:1780298644.331655:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609580800:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298644.331666:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298644.331679:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e63034e15 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ddb expref: 13 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.331692:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e63034e15 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ddb expref: 13 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.331709:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e63034e15 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ddb expref: 13 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.331726:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(16 122) 00010000:00010000:1.0:1780298644.331730:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(16 122) 00010000:00010000:1.0:1780298644.331738:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e63034e62 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1478656->1482751) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e1a expref: 8 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.331749:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e63034e62 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1478656->1482751) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e1a expref: 8 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.331773:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e63034e62 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1478656->1482751) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e1a expref: 8 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.331784:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b1880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:8374:1866777605013632:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298644.331824:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a540 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.332519:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8b1880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:8374:1866777605013632:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298644.332530:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298644.332558:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2399:x1866777609580800:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 905us (1008us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.332566:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15184 00000100:00100000:2.0:1780298644.332830:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.332858:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609580928 00000100:00100000:3.0:1780298644.332863:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609580928 00002000:00100000:3.0:1780298644.332891:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (1478656->1482751) req@ffff9cf34372e680 x1866777609580928/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298644.332900:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf34372e680 with x1866777609580928 ext(1478656->1482751) 00002000:00010000:3.0:1780298644.332905:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf34372e680 00000100:00100000:3.0:1780298644.332926:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6291 00000100:00100000:3.0:1780298644.332935:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609580928:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298644.333108:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609580928:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 175us (277us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1780298644.333115:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6291 00002000:00100000:3.0:1780298644.333143:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (1478656->1482751) req@ffff9cf34372e680 x1866777609580928/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/4536 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298644.333159:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf34372e680 with x1866777609580928 ext(1478656->1482751) 00002000:00010000:3.0:1780298644.333163:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf34372e680 00000100:00100000:0.0:1780298644.334181:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.334214:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609581056 00000100:00100000:1.0:1780298644.334219:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609581056 00002000:00100000:1.0:1780298644.334238:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x12:0x0] (1482752->1486847) req@ffff9cf36d4b4000 x1866777609581056/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.334247:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36d4b4000 with x1866777609581056 ext(1482752->1486847) 00002000:00010000:1.0:1780298644.334251:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf36d4b4000 00000100:00100000:1.0:1780298644.334272:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6292 00000100:00100000:1.0:1780298644.334289:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609581056:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298644.334387:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609581056:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 108us (207us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298644.334394:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6292 00002000:00100000:1.0:1780298644.334400:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x12:0x0] (1482752->1486847) req@ffff9cf36d4b4000 x1866777609581056/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/4536 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.334408:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36d4b4000 with x1866777609581056 ext(1482752->1486847) 00002000:00010000:1.0:1780298644.334412:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf36d4b4000 00000100:00100000:2.0:1780298644.335421:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.335456:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609581184 00000100:00100000:3.0:1780298644.335461:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609581184 00000100:00100000:3.0:1780298644.335487:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9816 00000100:00100000:3.0:1780298644.335492:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609581184:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.335499:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.335514:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034e69 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3e21 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.335558:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24339c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605013760:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.335596:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.336268:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24339c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605013760:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.336300:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034e69 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e21 expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.336314:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034e69 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e21 expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.336322:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157c00, rc 0) 00000100:00100000:3.0:1780298644.336346:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609581184:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 854us (926us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.336351:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9816 00000100:00100000:0.0:1780298644.337084:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.337141:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609581312 00000100:00100000:2.0:1780298644.337146:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609581312 00000100:00100000:2.0:1780298644.337179:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9817 00000100:00100000:2.0:1780298644.337189:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609581312:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.337198:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.337207:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e63034e70 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3e28 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.337240:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605013888:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.337294:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1c00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.337945:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605013888:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.337966:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e63034e70 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e28 expref: 12 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.337981:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e63034e70 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e28 expref: 12 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.337991:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3600, rc 0) 00000100:00100000:1.0:1780298644.338018:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609581312:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 832us (936us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.338024:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9817 00000100:00100000:1.0:1780298644.338912:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.338947:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609581440 00000100:00100000:2.0:1780298644.338952:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609581440 00000100:00100000:2.0:1780298644.338984:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9818 00000100:00100000:2.0:1780298644.338993:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609581440:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.339002:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.339009:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005c47b04a/0xaf35132e63034e77 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3e2f expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.339030:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605014016:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.339068:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb10c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.339696:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605014016:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.339713:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005c47b04a/0xaf35132e63034e77 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e2f expref: 12 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.339734:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005c47b04a/0xaf35132e63034e77 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e2f expref: 12 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.339745:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3400, rc 0) 00000100:00100000:2.0:1780298644.339776:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609581440:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 784us (863us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.339783:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9818 00000100:00100000:0.0:1780298644.340607:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.340645:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609581568 00000100:00100000:2.0:1780298644.340650:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609581568 00000100:00100000:2.0:1780298644.340678:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9819 00000100:00100000:2.0:1780298644.340683:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609581568:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.340692:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.340701:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b93eb96b/0xaf35132e63034e7e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3e36 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.340736:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605014144:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.340766:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb10c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.341455:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605014144:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.341467:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b93eb96b/0xaf35132e63034e7e lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e36 expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.341478:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b93eb96b/0xaf35132e63034e7e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e36 expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.341484:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3800, rc 0) 00000100:00100000:2.0:1780298644.341508:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609581568:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 825us (903us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.341512:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9819 00000100:00100000:0.0:1780298644.342424:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.342490:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609581696 00000100:00100000:3.0:1780298644.342496:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609581696 00000100:00100000:3.0:1780298644.342529:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9820 00000100:00100000:3.0:1780298644.342538:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:29360:x1866777609581696:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.342547:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.342556:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034e85 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3e3d expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.342576:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63034e0e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3dd4 expref: 17 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.342585:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(12 169) 00010000:00010000:3.0:1780298644.342589:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(13 169) 00010000:00010000:3.0:1780298644.342594:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034e85 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1236992->1241087] (req 1236992->1241087) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e3d expref: 12 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.342616:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63034e0e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3dd4 expref: 17 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.342630:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63034e0e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3dd4 expref: 17 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.342641:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24339c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605014272:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298644.342684:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.343389:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24339c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605014272:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298644.343407:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034e85 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1236992->1241087] (req 1236992->1241087) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e3d expref: 12 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.343427:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157c00, rc 0) 00000100:00100000:3.0:1780298644.343453:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:29360:x1866777609581696:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 917us (1036us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298644.343458:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9820 00000100:00100000:1.0:1780298644.343576:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.343619:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609581824 00000100:00100000:0.0:1780298644.343624:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609581824 00010000:00010000:0.0:1780298644.343645:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63034e0e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3dd4 expref: 18 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.343674:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15185 00000100:00100000:0.0:1780298644.343682:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b3100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609581824:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298644.343692:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298644.343705:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63034e0e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3dd4 expref: 18 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.343718:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63034e0e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3dd4 expref: 18 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.343735:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63034e0e lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3dd4 expref: 18 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.343746:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(13 169) 00010000:00010000:0.0:1780298644.343750:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(13 169) 00010000:00010000:0.0:1780298644.343758:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034e85 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1236992->34791423] (req 1236992->1241087) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e3d expref: 11 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.343766:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034e85 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1236992->34791423] (req 1236992->1241087) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e3d expref: 11 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.343796:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034e85 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1236992->34791423] (req 1236992->1241087) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e3d expref: 11 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.343807:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36d4b7b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777605014400:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298644.343843:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393b40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.344567:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36d4b7b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777605014400:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298644.344579:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298644.344609:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b3100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609581824:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 928us (1034us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.344617:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15185 00000100:00100000:0.0:1780298644.344715:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.344796:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609581952 00000100:00100000:3.0:1780298644.344800:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609581952 00000100:00100000:3.0:1780298644.344827:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6293 00000100:00100000:3.0:1780298644.344832:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609581952:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:3.0:1780298644.345003:0:19949:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4709602 -> 4709602 00002000:00010000:3.0:1780298644.345007:0:19949:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:3.0:1780298644.345008:0:19949:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:3.0:1780298644.345010:0:19949:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:3.0:1780298644.345012:0:19949:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 344 -> 352 00000100:00100000:3.0:1780298644.345044:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609581952:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 210us (329us total) trans 4294969722 rc 0/0 00000100:00100000:3.0:1780298644.345051:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6293 00000100:00100000:2.0:1780298644.345881:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.345916:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609582080 00000100:00100000:1.0:1780298644.345920:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609582080 00000100:00100000:1.0:1780298644.345945:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9821 00000100:00100000:1.0:1780298644.345952:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2397:x1866777609582080:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.345961:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.345972:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000488d1c2a/0xaf35132e63034e8c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3e44 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.346009:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605014528:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.346053:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.346751:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605014528:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.346768:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000488d1c2a/0xaf35132e63034e8c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e44 expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.346779:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000488d1c2a/0xaf35132e63034e8c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e44 expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.346786:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279400, rc 0) 00000100:00100000:1.0:1780298644.346809:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2397:x1866777609582080:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 858us (929us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.346813:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9821 00000100:00100000:0.0:1780298644.347642:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.347677:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609582208 00000100:00100000:2.0:1780298644.347682:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609582208 00000100:00100000:2.0:1780298644.347706:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9822 00000100:00100000:2.0:1780298644.347711:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2398:x1866777609582208:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.347718:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.347725:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e63034e93 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3e4b expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.347746:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605014656:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.347780:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb10c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.348444:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605014656:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.348459:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e63034e93 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e4b expref: 14 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.348469:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e63034e93 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e4b expref: 14 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.348476:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3c00, rc 0) 00000100:00100000:1.0:1780298644.348507:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2398:x1866777609582208:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 795us (864us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.348514:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9822 00000100:00100000:0.0:1780298644.349429:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.349460:0:6476:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609582464 00000100:00100000:2.0:1780298644.349464:0:6476:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609582464 00000100:00100000:2.0:1780298644.349501:0:6476:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62031 00000100:00100000:2.0:1780298644.349510:0:6476:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+133:29360:x1866777609582464:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:2.0:1780298644.349545:0:6476:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298644.349561:0:6476:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278400/0xaf35132e63034e31 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 93 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3de9 expref: 133 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.349571:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298644.349574:0:6476:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298644.349705:0:6476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3400/0xaf35132e63034e9a lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 92 type: IBT flags: 0x40000000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.349721:0:6476:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3e00/0xaf35132e63034e23 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 92 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3de2 expref: 172 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.349748:0:6476:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3400/0xaf35132e63034e9a lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 92 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.349767:0:6476:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3e00/0xaf35132e63034e23 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 92 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3de2 expref: 172 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.349777:0:6476:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3e00/0xaf35132e63034e23 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 92 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3de2 expref: 172 pid: 6477 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.349785:0:6476:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372ce00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6476:1866777605014784:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298644.349819:0:6476:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1c00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.350493:0:6476:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372ce00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6476:1866777605014784:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298644.350508:0:6476:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3400/0xaf35132e63034e9a lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 92 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.350698:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.350740:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609582592 00000100:00100000:1.0:1780298644.350745:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609582592 00010000:00010000:1.0:1780298644.350768:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3e00/0xaf35132e63034e23 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 92 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3de2 expref: 173 pid: 6477 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.350796:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15186 00000100:00100000:1.0:1780298644.350805:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+173:2399:x1866777609582592:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298644.350815:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298644.350820:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3e00/0xaf35132e63034e23 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 93 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3de2 expref: 173 pid: 6477 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.350831:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3e00/0xaf35132e63034e23 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 93 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3de2 expref: 173 pid: 6477 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.350844:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3e00/0xaf35132e63034e23 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 93 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3de2 expref: 173 pid: 6477 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.350856:0:7400:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.350861:0:7400:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3400/0xaf35132e63034e9a lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 92 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.350873:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3400/0xaf35132e63034e9a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 92 type: IBT flags: 0x40290000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.350880:0:7400:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3400/0xaf35132e63034e9a lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 92 type: IBT flags: 0x40290000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.350900:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298644.350918:0:6476:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3400/0xaf35132e63034e9a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 91 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.350927:0:6476:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3400/0xaf35132e63034e9a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 91 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.350936:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+172:2399:x1866777609582592:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 132us (237us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.350943:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15186 00000001:00080000:2.0:1780298644.350986:0:6476:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991086 00000004:00010000:2.0:1780298644.351012:0:6476:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 000000005280b303 reply state 000000002420be3a transno 4294991087 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3400/0xaf35132e63034e9a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 91 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.351025:0:6476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3400/0xaf35132e63034e9a lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 91 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.351032:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298644.351035:0:6476:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1780298644.351037:0:6476:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1780298644.351107:0:6476:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+135:29360:x1866777609582464:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1598us (1679us total) trans 4294991087 rc 0/0 00000100:00100000:2.0:1780298644.351114:0:6476:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62031 00000020:00080000:2.0:1780298644.351119:0:6476:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1780298644.351845:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.351875:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609582720 00000100:00100000:2.0:1780298644.351879:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609582720 00000100:00100000:2.0:1780298644.351901:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9823 00000100:00100000:2.0:1780298644.351906:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:29360:x1866777609582720:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.351914:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.351921:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e63034ea1 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3e52 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.351937:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034e85 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1236992->34791423] (req 1236992->1241087) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e3d expref: 14 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.351944:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(13 169) 00010000:00010000:2.0:1780298644.351947:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(14 169) 00010000:00010000:2.0:1780298644.351950:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e63034ea1 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e52 expref: 14 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.351963:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034e85 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1236992->34791423] (req 1236992->1241087) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e3d expref: 14 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.351974:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034e85 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1236992->34791423] (req 1236992->1241087) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e3d expref: 14 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.351983:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605014912:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298644.352020:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb10c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.352670:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605014912:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298644.352687:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e63034ea1 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e52 expref: 14 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.352701:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2800, rc 0) 00000100:00100000:2.0:1780298644.352733:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:29360:x1866777609582720:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 825us (888us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.352743:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9823 00000100:00100000:3.0:1780298644.352875:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.352908:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609582848 00000100:00100000:0.0:1780298644.352912:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609582848 00010000:00010000:0.0:1780298644.352926:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034e85 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1236992->34791423] (req 1236992->1241087) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e3d expref: 14 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.352945:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15187 00000100:00100000:0.0:1780298644.352950:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2399:x1866777609582848:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298644.352956:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298644.352968:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034e85 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1236992->34791423] (req 1236992->1241087) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e3d expref: 14 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.352978:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034e85 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1236992->34791423] (req 1236992->1241087) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e3d expref: 14 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.352995:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034e85 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1236992->34791423] (req 1236992->1241087) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e3d expref: 14 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.353002:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(14 169) 00010000:00010000:0.0:1780298644.353004:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(14 169) 00010000:00010000:0.0:1780298644.353008:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e63034ea1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e52 expref: 13 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.353013:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e63034ea1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e52 expref: 13 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.353028:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e63034ea1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e52 expref: 13 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.353035:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255488a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777605015040:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298644.353063:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393d80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.353776:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255488a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777605015040:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298644.353786:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298644.353814:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2399:x1866777609582848:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 863us (940us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.353821:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15187 00000100:00100000:2.0:1780298644.353842:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.353883:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609582976 00000100:00100000:1.0:1780298644.353888:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609582976 00000100:00100000:1.0:1780298644.353923:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9824 00000100:00100000:1.0:1780298644.353931:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:29360:x1866777609582976:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.353939:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.353948:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63034ea8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3e59 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.353965:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e63034e62 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1478656->1482751) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e1a expref: 10 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.353971:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(16 122) 00010000:00010000:1.0:1780298644.353973:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(16 122) 00010000:00010000:1.0:1780298644.353976:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63034ea8 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e59 expref: 10 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.353989:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e63034e62 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1478656->1482751) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e1a expref: 10 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.354009:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e63034e62 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1478656->1482751) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e1a expref: 10 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.354018:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605015168:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298644.354059:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.354633:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605015168:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298644.354645:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63034ea8 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e59 expref: 10 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.354657:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278a00, rc 0) 00000100:00100000:1.0:1780298644.354687:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:29360:x1866777609582976:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 757us (846us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.354697:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9824 00000100:00100000:0.0:1780298644.354895:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.354932:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609583104 00000100:00100000:2.0:1780298644.354937:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609583104 00010000:00010000:2.0:1780298644.354953:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e63034e62 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1478656->1482751) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e1a expref: 10 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.354981:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15188 00000100:00100000:2.0:1780298644.354988:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf360fe9850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609583104:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298644.354998:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298644.355012:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e63034e62 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1478656->1482751) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e1a expref: 10 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.355033:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e63034e62 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1478656->1482751) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e1a expref: 10 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.355048:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e63034e62 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1478656->1482751) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e1a expref: 10 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.355060:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(16 122) 00010000:00010000:2.0:1780298644.355063:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(16 122) 00010000:00010000:2.0:1780298644.355069:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63034ea8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e59 expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.355078:0:6463:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63034ea8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e59 expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.355099:0:6463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63034ea8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e59 expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.355110:0:6463:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372dc00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777605015296:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298644.355173:0:6463:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb10c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.355902:0:6463:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372dc00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777605015296:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298644.355915:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298644.355953:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf360fe9850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609583104:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 965us (1058us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.355961:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15188 00000100:00100000:0.0:1780298644.356059:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.356098:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609583232 00000100:00100000:1.0:1780298644.356103:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609583232 00002000:00010000:1.0:1780298644.356146:0:8362:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (3609904->18446744073709551615) 00002000:00100000:1.0:1780298644.356151:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25548aa00 with x1866777609583232 ext(3609904->18446744073709551615) 00002000:00010000:1.0:1780298644.356159:0:8362:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf25548aa00. 00000100:00100000:1.0:1780298644.356189:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6294 00000100:00100000:1.0:1780298644.356201:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609583232:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:2.0:1780298644.356202:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.356237:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609583360 00000100:00100000:0.0:1780298644.356241:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609583360 00002000:00010000:0.0:1780298644.356264:0:19943:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00002000:00100000:0.0:1780298644.356267:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf34372dc00 with x1866777609583360 ext(0->18446744073709551615) 00002000:00010000:0.0:1780298644.356273:0:19943:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf34372dc00. 00000100:00100000:0.0:1780298644.356304:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6295 00000100:00100000:0.0:1780298644.356313:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609583360:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:1.0:1780298644.356467:0:8362:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4709602 -> 3609904 00002000:00010000:0.0:1780298644.356476:0:19943:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4194304 -> 0 00002000:00010000:0.0:1780298644.356480:0:19943:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298644.356486:0:19943:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298644.356487:0:8362:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298644.356491:0:19943:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298644.356493:0:8362:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298644.356495:0:19943:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 8 -> 0 00002000:00010000:1.0:1780298644.356498:0:8362:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298644.356502:0:8362:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 352 -> 336 00000100:00100000:0.0:1780298644.356527:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609583360:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 216us (329us total) trans 4294969272 rc 0/0 00000100:00100000:1.0:1780298644.356533:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609583232:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 335us (473us total) trans 4294969723 rc 0/0 00000100:00100000:0.0:1780298644.356540:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6295 00002000:00010000:0.0:1780298644.356545:0:19943:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00000100:00100000:1.0:1780298644.356549:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6294 00002000:00100000:0.0:1780298644.356550:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf34372dc00 with x1866777609583360 ext(0->18446744073709551615) 00002000:00010000:1.0:1780298644.356554:0:8362:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (3609904->18446744073709551615) 00002000:00010000:0.0:1780298644.356556:0:19943:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf34372dc00. 00002000:00100000:1.0:1780298644.356560:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25548aa00 with x1866777609583232 ext(3609904->18446744073709551615) 00002000:00010000:1.0:1780298644.356566:0:8362:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf25548aa00. 00000100:00100000:0.0:1780298644.357406:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.357442:0:17578:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609583488 00000100:00100000:1.0:1780298644.357447:0:17578:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609583488 00000100:00100000:1.0:1780298644.357481:0:17578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62032 00000100:00100000:1.0:1780298644.357489:0:17578:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+172:29360:x1866777609583488:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.357501:0:17578:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.357515:0:17578:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278600/0xaf35132e63034eaf lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 92 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3e60 expref: -99 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.357587:0:17578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279c00/0xaf35132e63034eb6 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 93 type: IBT flags: 0x40000000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.357601:0:17578:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279c00/0xaf35132e63034eb6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 93 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.357612:0:17578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279c00/0xaf35132e63034eb6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 93 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1780298644.357635:0:17578:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 3609904 00000004:00200000:1.0:1780298644.357638:0:17578:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1780298644.357653:0:17578:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279c00/0xaf35132e63034eb6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 93 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.357675:0:17578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278600/0xaf35132e63034eaf lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 93 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e60 expref: 174 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.357688:0:17578:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279c00/0xaf35132e63034eb6 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 92 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e60 expref: 173 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.357696:0:17578:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.357698:0:17578:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1780298644.357702:0:17578:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279c00, rc 0) 00000100:00100000:1.0:1780298644.357732:0:17578:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+173:29360:x1866777609583488:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 244us (327us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.357739:0:17578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62032 00000100:00100000:2.0:1780298644.358651:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.358697:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609583616 00000100:00100000:1.0:1780298644.358703:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609583616 00000100:00100000:1.0:1780298644.358739:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9825 00000100:00100000:1.0:1780298644.358748:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2399:x1866777609583616:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.358758:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.358768:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63034ebd lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3e67 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.358780:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.358813:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605015424:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.358856:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609583744 00000100:00100000:2.0:1780298644.358861:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609583744 00000100:00100000:1.0:1780298644.358868:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837ae40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.358894:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9826 00000100:00100000:2.0:1780298644.358905:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2399:x1866777609583744:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.358913:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.358921:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e63034ec4 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3e6e expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.358949:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605015552:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.358983:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb16c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.359580:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605015424:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.359605:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63034ebd lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e67 expref: 18 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.359623:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63034ebd lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e67 expref: 18 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.359633:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279a00, rc 0) 00000100:00100000:1.0:1780298644.359661:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2399:x1866777609583616:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 915us (1011us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.359667:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9825 00000100:00100000:2.0:1780298644.359809:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605015552:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.359828:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e63034ec4 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e6e expref: 13 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.359842:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e63034ec4 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e6e expref: 13 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.359858:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2000, rc 0) 00000100:00100000:2.0:1780298644.359890:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2399:x1866777609583744:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 987us (1112us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.359897:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9826 00000100:00100000:0.0:1780298644.360883:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.360915:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.360936:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609583872 00000100:00100000:2.0:1780298644.360942:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609583872 00000100:00100000:1.0:1780298644.360957:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609584000 00000100:00100000:1.0:1780298644.360963:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609584000 00000100:00100000:2.0:1780298644.360979:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9827 00000100:00100000:2.0:1780298644.360989:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255488a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2397:x1866777609583872:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.360999:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.361012:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b93eb96b/0xaf35132e63034ecb lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3e75 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.361014:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9828 00000100:00100000:3.0:1780298644.361030:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609584000:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.361040:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.361050:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042c16ab0/0xaf35132e63034ed2 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3e7c expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.361051:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605015680:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.361092:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24339c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605015808:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.361111:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb16c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.361161:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.362059:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605015680:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.362062:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24339c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605015808:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.362087:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000042c16ab0/0xaf35132e63034ed2 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e7c expref: 13 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.362091:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b93eb96b/0xaf35132e63034ecb lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e75 expref: 18 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.362103:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042c16ab0/0xaf35132e63034ed2 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e7c expref: 13 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.362113:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156800, rc 0) 00010000:00010000:2.0:1780298644.362115:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b93eb96b/0xaf35132e63034ecb lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e75 expref: 18 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.362153:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3800, rc 0) 00000100:00100000:1.0:1780298644.362164:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609584000:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1140us (1247us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.362169:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9828 00000100:00100000:2.0:1780298644.362182:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255488a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2397:x1866777609583872:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1195us (1297us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.362191:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9827 00000100:00100000:0.0:1780298644.363742:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.363784:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609584128 00000100:00100000:3.0:1780298644.363788:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609584128 00000100:00100000:3.0:1780298644.363822:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6296 00000100:00100000:3.0:1780298644.363828:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2398:x1866777609584128:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:3.0:1780298644.363944:0:19950:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 3609904 -> 3609904 00002000:00010000:3.0:1780298644.363947:0:19950:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:3.0:1780298644.363948:0:19950:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:3.0:1780298644.363950:0:19950:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:3.0:1780298644.363951:0:19950:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 336 -> 336 00000100:00100000:3.0:1780298644.363987:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2398:x1866777609584128:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 159us (251us total) trans 4294969724 rc 0/0 00000100:00100000:3.0:1780298644.363991:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6296 00000100:00100000:0.0:1780298644.364799:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.364838:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609584256 00000100:00100000:2.0:1780298644.364845:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609584256 00000100:00100000:2.0:1780298644.364883:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62033 00000100:00100000:2.0:1780298644.364894:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ada680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+135:29360:x1866777609584256:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.364910:0:6477:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.364925:0:6477:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2600/0xaf35132e63034ed9 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 93 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3e83 expref: -99 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.364996:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3600/0xaf35132e63034ee0 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 94 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.365011:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3600/0xaf35132e63034ee0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 94 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.365023:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3600/0xaf35132e63034ee0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 94 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298644.365048:0:6477:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 3609904 00000004:00200000:2.0:1780298644.365052:0:6477:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298644.365068:0:6477:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3600/0xaf35132e63034ee0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 94 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.365090:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2600/0xaf35132e63034ed9 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 94 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e83 expref: 137 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.365108:0:6477:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3600/0xaf35132e63034ee0 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 93 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e83 expref: 136 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.365121:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298644.365148:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298644.365152:0:6477:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3600, rc 0) 00000100:00100000:2.0:1780298644.365187:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ada680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+136:29360:x1866777609584256:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 296us (387us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.365195:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62033 00000100:00100000:2.0:1780298644.366767:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.366799:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609584384 00000100:00100000:1.0:1780298644.366804:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609584384 00000100:00100000:1.0:1780298644.366834:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9829 00000100:00100000:1.0:1780298644.366839:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:29360:x1866777609584384:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.366848:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.366858:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034ee7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3e8a expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.366875:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63034ea8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e59 expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.366882:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(16 122) 00010000:00010000:1.0:1780298644.366886:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(17 122) 00010000:00010000:1.0:1780298644.366889:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034ee7 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [606208->614399] (req 606208->614399) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e8a expref: 13 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.366911:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63034ea8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e59 expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.366921:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63034ea8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e59 expref: 9 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.366932:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605015936:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298644.366979:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837ae40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.367689:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605015936:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298644.367706:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034ee7 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [606208->614399] (req 606208->614399) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e8a expref: 13 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.367717:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279400, rc 0) 00000100:00100000:1.0:1780298644.367742:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:29360:x1866777609584384:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 904us (979us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.367747:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9829 00000100:00100000:0.0:1780298644.367939:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.367969:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609584512 00000100:00100000:2.0:1780298644.367973:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609584512 00010000:00010000:2.0:1780298644.367990:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63034ea8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e59 expref: 10 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.368010:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15189 00000100:00100000:2.0:1780298644.368015:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adbb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609584512:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298644.368023:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298644.368036:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63034ea8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e59 expref: 10 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.368045:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63034ea8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e59 expref: 10 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.368056:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63034ea8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e59 expref: 10 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.368068:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(17 122) 00010000:00010000:2.0:1780298644.368070:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(17 122) 00010000:00010000:2.0:1780298644.368075:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034ee7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [606208->34160639] (req 606208->614399) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e8a expref: 12 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.368081:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034ee7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [606208->34160639] (req 606208->614399) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e8a expref: 12 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.368097:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034ee7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [606208->34160639] (req 606208->614399) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e8a expref: 12 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.368104:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372d880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:20253:1866777605016064:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298644.368150:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb16c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.368841:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372d880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:20253:1866777605016064:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298644.368854:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298644.368885:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adbb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609584512:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 868us (945us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.368894:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15189 00000100:00100000:0.0:1780298644.369518:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.369549:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609584640 00000100:00100000:2.0:1780298644.369553:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609584640 00000100:00100000:2.0:1780298644.369583:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9830 00000100:00100000:2.0:1780298644.369592:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2399:x1866777609584640:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.369603:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.369616:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63034eee lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3e91 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.369654:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605016192:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.369662:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.369704:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1480 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.369706:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609584768 00000100:00100000:0.0:1780298644.369712:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609584768 00000100:00100000:0.0:1780298644.369750:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9831 00000100:00100000:0.0:1780298644.369761:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609584768:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.369770:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.369781:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e63034ef5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3e98 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.369810:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244adad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605016320:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.369849:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.370397:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605016192:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.370412:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63034eee lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e91 expref: 18 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.370426:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63034eee lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e91 expref: 18 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.370436:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3000, rc 0) 00000100:00100000:2.0:1780298644.370468:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2399:x1866777609584640:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 878us (951us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.370474:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9830 00002000:00010000:2.0:1780298644.370664:0:19992:0:(ofd_lvb.c:259:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size: 0 -> 612875 00002000:00010000:2.0:1780298644.370668:0:19992:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks: 0 -> 1 00000100:00100000:2.0:1780298644.370677:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244adad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605016320:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.370693:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e63034ef5 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e98 expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.370706:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e63034ef5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e98 expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.370716:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf200, rc 0) 00000100:00100000:2.0:1780298644.370744:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609584768:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 987us (1088us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.370750:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9831 00000100:00100000:0.0:1780298644.371619:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.371643:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.371660:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609584896 00000100:00100000:2.0:1780298644.371664:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609584896 00000100:00100000:1.0:1780298644.371676:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609585024 00000100:00100000:1.0:1780298644.371681:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609585024 00000100:00100000:2.0:1780298644.371699:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9832 00000100:00100000:2.0:1780298644.371710:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2398:x1866777609584896:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298644.371714:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9833 00010000:00010000:2.0:1780298644.371723:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298644.371727:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609585024:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.371735:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.371739:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e63034efc lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3e9f expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.371743:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f4922155/0xaf35132e63034f03 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3ea6 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.371791:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605016448:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.371791:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605016576:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.371835:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837ae40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.371836:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1480 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.372564:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f3b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605016576:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.372582:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f4922155/0xaf35132e63034f03 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ea6 expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.372596:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f4922155/0xaf35132e63034f03 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ea6 expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.372597:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605016448:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.372610:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278600, rc 0) 00010000:00010000:0.0:1780298644.372625:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e63034efc lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e9f expref: 18 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.372638:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609585024:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 914us (991us total) trans 0 rc 301/301 00010000:00010000:0.0:1780298644.372642:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e63034efc lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e9f expref: 18 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.372653:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9833 00010000:00010000:0.0:1780298644.372661:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2e00, rc 0) 00000100:00100000:0.0:1780298644.372685:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2398:x1866777609584896:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 979us (1065us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.372692:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9832 00000100:00100000:3.0:1780298644.373762:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.373802:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609585152 00000100:00100000:1.0:1780298644.373808:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609585152 00000100:00100000:1.0:1780298644.373843:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9834 00000100:00100000:1.0:1780298644.373851:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:29360:x1866777609585152:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.373860:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.373869:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63034f0a lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3ead expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.373890:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e63034ea1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e52 expref: 14 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.373899:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(14 169) 00010000:00010000:1.0:1780298644.373903:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(15 169) 00010000:00010000:1.0:1780298644.373908:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63034f0a lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1101824->1110015] (req 1101824->1110015) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ead expref: 18 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.373930:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e63034ea1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e52 expref: 14 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.373944:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e63034ea1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e52 expref: 14 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.373961:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f2680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605016704:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298644.374006:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837af00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.374703:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f2680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605016704:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298644.374720:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63034f0a lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1101824->1110015] (req 1101824->1110015) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ead expref: 18 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.374732:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278c00, rc 0) 00000100:00100000:1.0:1780298644.374765:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:29360:x1866777609585152:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 916us (1002us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.374772:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9834 00000100:00100000:2.0:1780298644.374941:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.374983:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609585280 00000100:00100000:1.0:1780298644.374987:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609585280 00010000:00010000:1.0:1780298644.375007:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e63034ea1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e52 expref: 15 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.375035:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15190 00000100:00100000:1.0:1780298644.375042:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2400:x1866777609585280:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298644.375052:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298644.375066:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e63034ea1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e52 expref: 15 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.375077:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e63034ea1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e52 expref: 15 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.375098:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e63034ea1 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e52 expref: 15 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.375107:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(15 169) 00010000:00010000:1.0:1780298644.375111:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(15 169) 00010000:00010000:1.0:1780298644.375117:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63034f0a lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1101824->1110015) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ead expref: 17 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.375155:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63034f0a lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1101824->1110015) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ead expref: 17 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.375179:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63034f0a lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1101824->1110015) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ead expref: 17 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.375190:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f2680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777605016832:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298644.375225:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837af00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.375930:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f2680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777605016832:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298644.375941:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298644.375968:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609585280:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 927us (1029us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.375979:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15190 00000100:00100000:3.0:1780298644.376290:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.376327:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609585408 00000100:00100000:1.0:1780298644.376332:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609585408 00002000:00100000:1.0:1780298644.376361:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (1101824->1110015) req@ffff9cf24339e300 x1866777609585408/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.376372:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24339e300 with x1866777609585408 ext(1101824->1110015) 00002000:00010000:1.0:1780298644.376376:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24339e300 00000100:00100000:1.0:1780298644.376405:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6297 00000100:00100000:1.0:1780298644.376413:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609585408:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298644.376562:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609585408:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 149us (282us total) trans 0 rc 8192/8192 00000100:00100000:1.0:1780298644.376570:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6297 00002000:00100000:1.0:1780298644.376577:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (1101824->1110015) req@ffff9cf24339e300 x1866777609585408/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/8632 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 8192/8192 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.376583:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24339e300 with x1866777609585408 ext(1101824->1110015) 00002000:00010000:1.0:1780298644.376586:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24339e300 00000100:00100000:2.0:1780298644.378073:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.378111:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.378143:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609585536 00000100:00100000:0.0:1780298644.378150:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609585536 00000100:00100000:1.0:1780298644.378166:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609585664 00000100:00100000:1.0:1780298644.378170:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609585664 00000100:00100000:0.0:1780298644.378188:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9835 00000100:00100000:0.0:1780298644.378199:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2399:x1866777609585536:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298644.378202:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9836 00000100:00100000:3.0:1780298644.378211:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609585664:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.378212:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.378218:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.378225:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e63034f11 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3eb4 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.378226:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bed20494/0xaf35132e63034f18 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3ebb expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.378242:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(15 169) 00010000:00010000:0.0:1780298644.378246:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(15 169) 00010000:00010000:0.0:1780298644.378252:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e63034f11 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3eb4 expref: 15 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.378262:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24339d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605016960:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.378264:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e63034f11 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3eb4 expref: 15 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.378274:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf600, rc 0) 00000100:00100000:0.0:1780298644.378314:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2399:x1866777609585536:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 119us (243us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.378321:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9835 00000100:00100000:3.0:1780298644.378322:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.378965:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24339d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605016960:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.378982:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bed20494/0xaf35132e63034f18 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ebb expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.378995:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bed20494/0xaf35132e63034f18 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ebb expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.379006:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157000, rc 0) 00000100:00100000:1.0:1780298644.379035:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609585664:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 826us (924us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.379045:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9836 00000100:00100000:1.0:1780298644.379827:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.379858:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609585792 00000100:00100000:0.0:1780298644.379862:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609585792 00000100:00100000:0.0:1780298644.379887:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9837 00000100:00100000:0.0:1780298644.379891:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609585792:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.379898:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.379906:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63034f1f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3ec2 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.379931:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605017088:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.379964:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733936c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.380615:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605017088:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.380628:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63034f1f lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ec2 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.380637:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63034f1f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ec2 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.380643:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf400, rc 0) 00000100:00100000:0.0:1780298644.380665:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609585792:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 774us (840us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.380669:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9837 00000100:00100000:3.0:1780298644.381682:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.381721:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609585920 00000100:00100000:1.0:1780298644.381725:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609585920 00000100:00100000:1.0:1780298644.381750:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9838 00000100:00100000:1.0:1780298644.381754:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609585920:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.381761:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.381768:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63034f26 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3ec9 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.381793:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605017216:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.381824:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837af00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.382528:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605017216:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.382542:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63034f26 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ec9 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.382557:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63034f26 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ec9 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.382564:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279200, rc 0) 00000100:00100000:2.0:1780298644.382588:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609585920:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 832us (907us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.382595:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9838 00000100:00100000:2.0:1780298644.383428:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.383470:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609586048 00000100:00100000:3.0:1780298644.383476:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609586048 00002000:00100000:3.0:1780298644.383505:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (4067328->4071423) req@ffff9cf242b41880 x1866777609586048/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298644.383515:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b41880 with x1866777609586048 ext(4067328->4071423) 00002000:00010000:3.0:1780298644.383520:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf242b41880 00000100:00100000:3.0:1780298644.383543:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6298 00000100:00100000:3.0:1780298644.383551:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b41880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2399:x1866777609586048:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298644.383689:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b41880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2399:x1866777609586048:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 139us (260us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298644.383695:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6298 00002000:00100000:3.0:1780298644.383700:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (4067328->4071423) req@ffff9cf242b41880 x1866777609586048/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/440 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298644.383705:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b41880 with x1866777609586048 ext(4067328->4071423) 00002000:00010000:3.0:1780298644.383708:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf242b41880 00000100:00100000:3.0:1780298644.384475:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.384508:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609586176 00000100:00100000:2.0:1780298644.384512:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609586176 00002000:00100000:2.0:1780298644.384529:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (4071424->4075519) req@ffff9cf24339fb80 x1866777609586176/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.384536:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24339fb80 with x1866777609586176 ext(4071424->4075519) 00002000:00010000:2.0:1780298644.384538:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24339fb80 00000100:00100000:2.0:1780298644.384556:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6299 00000100:00100000:2.0:1780298644.384561:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2400:x1866777609586176:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298644.384624:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2400:x1866777609586176:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 62us (153us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.384629:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6299 00002000:00100000:2.0:1780298644.384634:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (4071424->4075519) req@ffff9cf24339fb80 x1866777609586176/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/440 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.384640:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24339fb80 with x1866777609586176 ext(4071424->4075519) 00002000:00010000:2.0:1780298644.384642:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24339fb80 00000100:00100000:1.0:1780298644.385322:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.385349:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609586304 00000100:00100000:0.0:1780298644.385353:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609586304 00002000:00100000:0.0:1780298644.385368:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (4075520->4079615) req@ffff9cf2558f3800 x1866777609586304/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298644.385373:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2558f3800 with x1866777609586304 ext(4075520->4079615) 00002000:00010000:0.0:1780298644.385376:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2558f3800 00000100:00100000:0.0:1780298644.385391:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6300 00000100:00100000:0.0:1780298644.385395:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2399:x1866777609586304:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298644.385453:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2399:x1866777609586304:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 57us (131us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.385458:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6300 00002000:00100000:0.0:1780298644.385461:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (4075520->4079615) req@ffff9cf2558f3800 x1866777609586304/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/440 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298644.385467:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2558f3800 with x1866777609586304 ext(4075520->4079615) 00002000:00010000:0.0:1780298644.385470:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2558f3800 00000100:00100000:1.0:1780298644.386484:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.386514:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609586432 00000100:00100000:2.0:1780298644.386519:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609586432 00000100:00100000:2.0:1780298644.386545:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9839 00000100:00100000:2.0:1780298644.386550:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609586432:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.386558:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.386567:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e63034f2d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3ed0 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.386603:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b40000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605017344:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.386645:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.387422:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b40000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605017344:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.387439:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e63034f2d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ed0 expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.387450:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e63034f2d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ed0 expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.387458:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2000, rc 0) 00000100:00100000:2.0:1780298644.387482:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609586432:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 931us (998us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.387489:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9839 00000100:00100000:1.0:1780298644.388316:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.388350:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609586560 00000100:00100000:3.0:1780298644.388354:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609586560 00000100:00100000:3.0:1780298644.388375:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9840 00000100:00100000:3.0:1780298644.388379:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609586560:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.388385:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.388392:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004ba675d2/0xaf35132e63034f34 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3ed7 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.388422:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24339d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605017472:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.388455:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.389038:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24339d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605017472:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.389059:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004ba675d2/0xaf35132e63034f34 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ed7 expref: 14 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.389068:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004ba675d2/0xaf35132e63034f34 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ed7 expref: 14 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.389075:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156e00, rc 0) 00000100:00100000:3.0:1780298644.389096:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609586560:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 716us (782us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.389100:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9840 00000100:00100000:0.0:1780298644.390030:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.390064:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609586688 00000100:00100000:2.0:1780298644.390069:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609586688 00000100:00100000:2.0:1780298644.390098:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9841 00000100:00100000:2.0:1780298644.390106:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609586688:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.390116:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.390161:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e63034f3b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3ede expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.390190:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b40000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605017600:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.390232:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.390858:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b40000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605017600:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.390871:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e63034f3b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ede expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.390880:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e63034f3b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ede expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.390887:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2600, rc 0) 00000100:00100000:2.0:1780298644.390911:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609586688:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 806us (880us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.390918:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9841 00000100:00100000:1.0:1780298644.391699:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.391729:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609586816 00000100:00100000:2.0:1780298644.391733:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609586816 00000100:00100000:2.0:1780298644.391758:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9842 00000100:00100000:2.0:1780298644.391763:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:29360:x1866777609586816:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.391770:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.391776:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63034f42 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3ee5 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.391793:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(17 122) 00010000:00010000:2.0:1780298644.391796:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(17 122) 00010000:00010000:2.0:1780298644.391804:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63034f42 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [401408->606207] (req 401408->405503) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ee5 expref: 10 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.391815:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63034f42 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [401408->606207] (req 401408->405503) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ee5 expref: 10 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.391820:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2800, rc 0) 00000100:00100000:2.0:1780298644.391842:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:29360:x1866777609586816:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 79us (145us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.391846:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9842 00000100:00100000:0.0:1780298644.392596:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.392629:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609586944 00000100:00100000:3.0:1780298644.392635:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609586944 00002000:00100000:3.0:1780298644.392660:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (401408->405503) req@ffff9cf244adb800 x1866777609586944/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298644.392670:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244adb800 with x1866777609586944 ext(401408->405503) 00002000:00010000:3.0:1780298644.392675:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf244adb800 00000100:00100000:3.0:1780298644.392703:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6301 00000100:00100000:3.0:1780298644.392712:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609586944:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298644.392829:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609586944:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 119us (234us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298644.392841:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6301 00002000:00100000:3.0:1780298644.392848:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (401408->405503) req@ffff9cf244adb800 x1866777609586944/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/440 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298644.392857:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244adb800 with x1866777609586944 ext(401408->405503) 00002000:00010000:3.0:1780298644.392861:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf244adb800 00000100:00100000:1.0:1780298644.393941:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.393977:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609587072 00000100:00100000:0.0:1780298644.393982:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609587072 00002000:00100000:0.0:1780298644.394013:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x12:0x0] (405504->409599) req@ffff9cf36d4b6680 x1866777609587072/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298644.394021:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36d4b6680 with x1866777609587072 ext(405504->409599) 00002000:00010000:0.0:1780298644.394025:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf36d4b6680 00000100:00100000:0.0:1780298644.394043:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6302 00000100:00100000:0.0:1780298644.394051:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609587072:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298644.394154:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609587072:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 105us (213us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.394160:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6302 00002000:00100000:0.0:1780298644.394165:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x12:0x0] (405504->409599) req@ffff9cf36d4b6680 x1866777609587072/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/440 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298644.394171:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36d4b6680 with x1866777609587072 ext(405504->409599) 00002000:00010000:0.0:1780298644.394173:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf36d4b6680 00000100:00100000:1.0:1780298644.395077:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.395108:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609587200 00000100:00100000:3.0:1780298644.395113:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609587200 00002000:00100000:3.0:1780298644.395162:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x12:0x0] (409600->413695) req@ffff9cf36d4b7100 x1866777609587200/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298644.395173:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36d4b7100 with x1866777609587200 ext(409600->413695) 00002000:00010000:3.0:1780298644.395177:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf36d4b7100 00000100:00100000:0.0:1780298644.395206:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6303 00000100:00100000:0.0:1780298644.395214:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609587200:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298644.395274:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609587200:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 62us (197us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.395294:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6303 00002000:00100000:0.0:1780298644.395299:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x12:0x0] (409600->413695) req@ffff9cf36d4b7100 x1866777609587200/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/440 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298644.395304:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36d4b7100 with x1866777609587200 ext(409600->413695) 00002000:00010000:0.0:1780298644.395306:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf36d4b7100 00000100:00100000:3.0:1780298644.396441:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.396475:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609587328 00000100:00100000:2.0:1780298644.396480:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609587328 00002000:00100000:2.0:1780298644.396501:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (401408->413695) req@ffff9cf24339d500 x1866777609587328/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 12776/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.396512:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24339d500 with x1866777609587328 ext(401408->413695) 00002000:00010000:2.0:1780298644.396516:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24339d500 00000100:00100000:2.0:1780298644.396538:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6304 00000100:00100000:2.0:1780298644.396544:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609587328:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:2.0:1780298644.396995:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609587328:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 450us (553us total) trans 4294969273 rc 0/0 00000100:00100000:2.0:1780298644.397003:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6304 00002000:00100000:2.0:1780298644.397010:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (401408->413695) req@ffff9cf24339d500 x1866777609587328/t4294969273(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 12776/448 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.397020:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24339d500 with x1866777609587328 ext(401408->413695) 00002000:00010000:2.0:1780298644.397024:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24339d500 00000100:00100000:1.0:1780298644.397738:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.397772:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609587456 00000100:00100000:2.0:1780298644.397776:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609587456 00000100:00100000:2.0:1780298644.397806:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9843 00000100:00100000:2.0:1780298644.397815:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609587456:12345-192.168.203.25@tcp:16:fsx.0 00000001:00080000:1.0:1780298644.398566:0:8351:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0000: transno 4294969273 is committed 00000001:00080000:1.0:1780298644.398574:0:8351:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0000: transno 4294969271 is committed 00000100:00100000:2.0:1780298644.398647:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609587456:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 833us (908us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.398655:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9843 00000100:00100000:1.0:1780298644.399657:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.399682:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609587584 00000100:00100000:3.0:1780298644.399687:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609587584 00000100:00100000:3.0:1780298644.399711:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9844 00000100:00100000:3.0:1780298644.399716:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609587584:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.399724:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.399738:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004ba675d2/0xaf35132e63034f49 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3eec expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.399780:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24339c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605017728:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.399846:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856cc0 going to sleep for 11 seconds 00002000:00010000:1.0:1780298644.400506:0:9128:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 1 -> 24 00000100:00100000:1.0:1780298644.400513:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24339c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605017728:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.400553:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004ba675d2/0xaf35132e63034f49 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3eec expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.400574:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004ba675d2/0xaf35132e63034f49 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3eec expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.400581:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156e00, rc 0) 00000100:00100000:1.0:1780298644.400608:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609587584:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 891us (951us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.400615:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9844 00000100:00100000:1.0:1780298644.401576:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.401616:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609587712 00000100:00100000:2.0:1780298644.401620:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609587712 00000100:00100000:2.0:1780298644.401652:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9845 00000100:00100000:2.0:1780298644.401660:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609587712:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.401670:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.401679:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e63034f50 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3ef3 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.401712:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b40000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605017856:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.401746:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.402371:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b40000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605017856:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.402394:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e63034f50 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ef3 expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.402405:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e63034f50 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ef3 expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.402411:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2000, rc 0) 00000100:00100000:0.0:1780298644.402434:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609587712:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 777us (860us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.402438:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9845 00000100:00100000:3.0:1780298644.403233:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.403268:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609587840 00000100:00100000:2.0:1780298644.403271:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609587840 00000100:00100000:2.0:1780298644.403307:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9846 00000100:00100000:2.0:1780298644.403315:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:29360:x1866777609587840:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.403325:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.403336:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e63034f57 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3efa expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.403357:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63034f0a lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1101824->1110015) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ead expref: 17 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.403366:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e63034f11 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3eb4 expref: 16 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.403372:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(15 169) 00010000:00010000:2.0:1780298644.403375:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(15 169) 00010000:00010000:2.0:1780298644.403378:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e63034f57 lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2371584->2379775] (req 2371584->2379775) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3efa expref: 16 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.403396:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e63034f11 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3eb4 expref: 16 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.403405:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e63034f11 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3eb4 expref: 16 pid: 19992 timeout: 933 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.403412:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b42a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605017984:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298644.403452:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63034f0a lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1101824->1110015) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ead expref: 17 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.403461:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63034f0a lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1101824->1110015) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ead expref: 17 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.403477:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b40380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605018112:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298644.403502:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.404052:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b42a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605017984:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298644.404202:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b40380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605018112:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298644.404220:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e63034f57 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2371584->2379775] (req 2371584->2379775) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3efa expref: 16 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.404233:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2e00, rc 0) 00000100:00100000:2.0:1780298644.404266:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:29360:x1866777609587840:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 952us (1032us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.404274:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9846 00000100:00100000:1.0:1780298644.404330:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.404363:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609587968 00000100:00100000:3.0:1780298644.404368:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609587968 00010000:00010000:2.0:1780298644.404413:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e63034f11 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3eb4 expref: 16 pid: 19992 timeout: 933 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.404443:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15191 00000100:00100000:2.0:1780298644.404451:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609587968:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298644.404460:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298644.404476:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e63034f11 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3eb4 expref: 16 pid: 19992 timeout: 933 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.404490:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e63034f11 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3eb4 expref: 16 pid: 19992 timeout: 933 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.404514:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e63034f11 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3eb4 expref: 16 pid: 19992 timeout: 933 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.404527:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298644.404532:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.404553:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2400:x1866777609587968:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 104us (231us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.404560:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15191 00000100:00100000:1.0:1780298644.404560:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609588096 00000100:00100000:1.0:1780298644.404565:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609588096 00010000:00010000:1.0:1780298644.404582:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63034f0a lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1101824->1110015) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ead expref: 18 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.404606:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15192 00000100:00100000:1.0:1780298644.404613:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adb100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2399:x1866777609588096:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298644.404619:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298644.404627:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63034f0a lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1101824->1110015) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ead expref: 18 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.404637:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63034f0a lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1101824->1110015) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ead expref: 18 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.404651:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63034f0a lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1101824->1110015) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ead expref: 18 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.404663:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(15 169) 00010000:00010000:1.0:1780298644.404667:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(15 169) 00010000:00010000:1.0:1780298644.404676:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e63034f57 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2371584->35926015] (req 2371584->2379775) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3efa expref: 14 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.404685:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e63034f57 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2371584->35926015] (req 2371584->2379775) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3efa expref: 14 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.404708:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e63034f57 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2371584->35926015] (req 2371584->2379775) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3efa expref: 14 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.404718:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304c380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777605018240:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298644.404746:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837ae40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.405397:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304c380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777605018240:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298644.405406:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298644.405425:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adb100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2399:x1866777609588096:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 815us (895us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.405430:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15192 00000100:00100000:0.0:1780298644.405831:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.405864:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609588224 00000100:00100000:1.0:1780298644.405868:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609588224 00002000:00100000:1.0:1780298644.405895:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (2371584->2375679) req@ffff9cf244ada300 x1866777609588224/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.405904:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244ada300 with x1866777609588224 ext(2371584->2375679) 00002000:00010000:1.0:1780298644.405908:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf244ada300 00000100:00100000:1.0:1780298644.405926:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6305 00000100:00100000:1.0:1780298644.405932:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ada300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2399:x1866777609588224:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298644.406059:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ada300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2399:x1866777609588224:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 127us (227us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298644.406063:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6305 00002000:00100000:1.0:1780298644.406068:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (2371584->2375679) req@ffff9cf244ada300 x1866777609588224/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/4536 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.406073:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244ada300 with x1866777609588224 ext(2371584->2375679) 00002000:00010000:1.0:1780298644.406075:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf244ada300 00000100:00100000:2.0:1780298644.406867:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.406902:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609588352 00000100:00100000:3.0:1780298644.406907:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609588352 00002000:00100000:3.0:1780298644.406929:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (2375680->2379775) req@ffff9cf242b40380 x1866777609588352/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298644.406939:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b40380 with x1866777609588352 ext(2375680->2379775) 00002000:00010000:3.0:1780298644.406943:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf242b40380 00000100:00100000:3.0:1780298644.406962:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6306 00000100:00100000:3.0:1780298644.406969:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2400:x1866777609588352:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298644.407046:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2400:x1866777609588352:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 77us (178us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1780298644.407053:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6306 00002000:00100000:3.0:1780298644.407060:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (2375680->2379775) req@ffff9cf242b40380 x1866777609588352/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/4536 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298644.407070:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b40380 with x1866777609588352 ext(2375680->2379775) 00002000:00010000:3.0:1780298644.407074:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf242b40380 00000100:00100000:0.0:1780298644.407934:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.407969:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609588480 00000100:00100000:2.0:1780298644.407975:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609588480 00000100:00100000:0.0:1780298644.407993:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.408009:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609588608 00000100:00100000:2.0:1780298644.408013:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609588608 00000100:00100000:1.0:1780298644.408028:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9847 00000100:00100000:1.0:1780298644.408038:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2397:x1866777609588480:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298644.408045:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9848 00010000:00010000:1.0:1780298644.408047:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298644.408053:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609588608:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.408060:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63034f5e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3f01 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.408061:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.408071:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e63034f65 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3f08 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.408100:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605018368:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.408102:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b42a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605018496:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.408165:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.408166:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.408909:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605018368:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.408931:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b42a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605018496:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.408933:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63034f5e lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f01 expref: 16 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.408953:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63034f5e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f01 expref: 16 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.408956:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e63034f65 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f08 expref: 8 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.408984:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279a00, rc 0) 00010000:00010000:2.0:1780298644.408995:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e63034f65 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f08 expref: 8 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.409005:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3800, rc 0) 00000100:00100000:1.0:1780298644.409017:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2397:x1866777609588480:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 981us (1083us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.409025:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9847 00000100:00100000:2.0:1780298644.409032:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609588608:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 980us (1039us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.409041:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9848 00000100:00100000:3.0:1780298644.409920:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.409965:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609588736 00000100:00100000:0.0:1780298644.409970:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609588736 00000100:00100000:3.0:1780298644.409983:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.410004:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609588864 00000100:00100000:0.0:1780298644.410008:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609588864 00000100:00100000:1.0:1780298644.410018:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9849 00000100:00100000:2.0:1780298644.410028:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9850 00000100:00100000:1.0:1780298644.410028:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2399:x1866777609588736:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.410036:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298644.410040:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609588864:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.410044:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e63034f6c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3f0f expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.410047:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.410057:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63034f73 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3f16 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.410070:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605018624:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.410083:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b42a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605018752:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.410105:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.410108:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.410741:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b42a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605018752:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.410756:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63034f73 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f16 expref: 8 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.410760:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605018624:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.410775:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63034f73 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f16 expref: 8 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.410781:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3c00, rc 0) 00010000:00010000:1.0:1780298644.410786:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e63034f6c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f0f expref: 16 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.410800:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e63034f6c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f0f expref: 16 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.410808:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609588864:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 771us (824us total) trans 0 rc 301/301 00010000:00010000:1.0:1780298644.410808:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278400, rc 0) 00000100:00100000:2.0:1780298644.410816:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9850 00000100:00100000:1.0:1780298644.410832:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2399:x1866777609588736:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 807us (913us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.410837:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9849 00000100:00100000:0.0:1780298644.411958:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.411995:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609588992 00000100:00100000:3.0:1780298644.411999:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609588992 00000100:00100000:3.0:1780298644.412022:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9851 00000100:00100000:3.0:1780298644.412027:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:29360:x1866777609588992:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.412035:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.412041:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034f7a lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3f1d expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.412057:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(15 169) 00010000:00010000:3.0:1780298644.412060:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(15 169) 00010000:00010000:3.0:1780298644.412066:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034f7a lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [942080->2371583] (req 942080->954367) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f1d expref: 16 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.412076:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034f7a lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [942080->2371583] (req 942080->954367) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f1d expref: 16 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.412082:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157c00, rc 0) 00000100:00100000:3.0:1780298644.412103:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:29360:x1866777609588992:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 76us (148us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298644.412107:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9851 00000100:00100000:3.0:1780298644.412973:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.413011:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609589120 00000100:00100000:0.0:1780298644.413015:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609589120 00002000:00100000:0.0:1780298644.413036:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (942080->954367) req@ffff9cf242861180 x1866777609589120/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298644.413042:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242861180 with x1866777609589120 ext(942080->954367) 00002000:00010000:0.0:1780298644.413046:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf242861180 00000100:00100000:0.0:1780298644.413064:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6307 00000100:00100000:0.0:1780298644.413069:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242861180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609589120:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298644.413217:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242861180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609589120:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 146us (242us total) trans 0 rc 12288/12288 00000100:00100000:0.0:1780298644.413224:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6307 00002000:00100000:0.0:1780298644.413231:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (942080->954367) req@ffff9cf242861180 x1866777609589120/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/12728 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 12288/12288 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298644.413239:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242861180 with x1866777609589120 ext(942080->954367) 00002000:00010000:0.0:1780298644.413250:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf242861180 00000100:00100000:0.0:1780298644.414220:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.414254:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.414267:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609589248 00000100:00100000:2.0:1780298644.414272:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609589248 00000100:00100000:3.0:1780298644.414296:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609589376 00000100:00100000:3.0:1780298644.414300:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609589376 00000100:00100000:2.0:1780298644.414321:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9852 00000100:00100000:3.0:1780298644.414333:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9853 00000100:00100000:2.0:1780298644.414340:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e4f2050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2399:x1866777609589248:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298644.414346:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609589376:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.414351:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.414356:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.414363:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e63034f81 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3f24 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.414368:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a58d48e9/0xaf35132e63034f88 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3f2b expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.414414:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242861c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605018880:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.414415:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b42a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605019008:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.414471:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.414471:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.415191:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b42a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605019008:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.415217:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e63034f81 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f24 expref: 17 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.415234:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242861c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605018880:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.415236:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e63034f81 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f24 expref: 17 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.415251:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3e00, rc 0) 00010000:00010000:3.0:1780298644.415271:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a58d48e9/0xaf35132e63034f88 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f2b expref: 8 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.415290:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e4f2050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2399:x1866777609589248:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 950us (1061us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.415299:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9852 00010000:00010000:3.0:1780298644.415303:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a58d48e9/0xaf35132e63034f88 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f2b expref: 8 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.415311:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156a00, rc 0) 00000100:00100000:3.0:1780298644.415337:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609589376:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 996us (1083us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.415345:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9853 00000100:00100000:0.0:1780298644.416251:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.416293:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.416309:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609589504 00000100:00100000:1.0:1780298644.416314:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609589504 00000100:00100000:2.0:1780298644.416335:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609589632 00000100:00100000:2.0:1780298644.416341:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609589632 00000100:00100000:1.0:1780298644.416349:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9854 00000100:00100000:1.0:1780298644.416356:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2397:x1866777609589504:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.416363:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.416371:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63034f8f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3f32 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.416381:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9855 00000100:00100000:2.0:1780298644.416392:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609589632:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298644.416396:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605019136:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.416399:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.416409:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e63034f96 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3f39 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.416433:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.416442:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b42300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605019264:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.416473:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.417159:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605019136:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.417181:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63034f8f lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f32 expref: 17 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.417192:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63034f8f lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f32 expref: 17 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.417200:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279a00, rc 0) 00000100:00100000:1.0:1780298644.417224:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2397:x1866777609589504:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 869us (974us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.417232:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9854 00000100:00100000:2.0:1780298644.417247:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b42300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605019264:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.417287:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e63034f96 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f39 expref: 8 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.417302:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e63034f96 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f39 expref: 8 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.417313:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2400, rc 0) 00000100:00100000:2.0:1780298644.417344:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609589632:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 955us (1052us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.417351:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9855 00000100:00100000:0.0:1780298644.418257:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.418343:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609589760 00000100:00100000:1.0:1780298644.418348:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609589760 00000100:00100000:1.0:1780298644.418376:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9856 00000100:00100000:1.0:1780298644.418384:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:29360:x1866777609589760:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.418397:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.418405:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63034f9d lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3f40 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.418421:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(15 169) 00010000:00010000:1.0:1780298644.418426:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(15 169) 00010000:00010000:1.0:1780298644.418436:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63034f9d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [69632->942079] (req 69632->81919) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f40 expref: 17 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.418452:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63034f9d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [69632->942079] (req 69632->81919) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f40 expref: 17 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.418461:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279200, rc 0) 00000100:00100000:1.0:1780298644.418513:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:29360:x1866777609589760:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 131us (258us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.418518:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9856 00000100:00100000:0.0:1780298644.419205:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.419259:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609589888 00000100:00100000:2.0:1780298644.419264:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609589888 00000100:00100000:2.0:1780298644.419312:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6308 00000100:00100000:2.0:1780298644.419321:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2400:x1866777609589888:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298644.419531:0:19943:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 3609904 -> 3609904 00002000:00010000:2.0:1780298644.419535:0:19943:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298644.419538:0:19943:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298644.419540:0:19943:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298644.419542:0:19943:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 336 -> 328 00000100:00100000:2.0:1780298644.419574:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2400:x1866777609589888:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 255us (368us total) trans 4294969725 rc 0/0 00000100:00100000:2.0:1780298644.419583:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6308 00000100:00100000:0.0:1780298644.420512:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.420550:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609590016 00000100:00100000:2.0:1780298644.420557:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609590016 00000100:00100000:2.0:1780298644.420596:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9857 00000100:00100000:2.0:1780298644.420607:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2400:x1866777609590016:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298644.420613:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:2.0:1780298644.420616:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.420629:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63034fa4 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3f47 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.420637:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609590144 00000100:00100000:3.0:1780298644.420643:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609590144 00000100:00100000:2.0:1780298644.420673:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b42300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605019392:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.420692:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9858 00000100:00100000:1.0:1780298644.420712:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609590144:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.420720:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.420732:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017dab243/0xaf35132e63034fab lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3f4e expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.420744:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.420762:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605019520:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.420785:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.421410:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b42300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605019392:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.421428:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63034fa4 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f47 expref: 19 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.421440:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63034fa4 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f47 expref: 19 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.421449:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2000, rc 0) 00000100:00100000:2.0:1780298644.421473:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2400:x1866777609590016:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 871us (962us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.421478:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9857 00000100:00100000:1.0:1780298644.421600:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605019520:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.421614:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000017dab243/0xaf35132e63034fab lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f4e expref: 8 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.421630:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017dab243/0xaf35132e63034fab lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f4e expref: 8 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.421637:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279800, rc 0) 00000100:00100000:1.0:1780298644.421657:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609590144:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 954us (1046us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.421662:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9858 00000100:00100000:0.0:1780298644.422420:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.422454:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609590272 00000100:00100000:2.0:1780298644.422458:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609590272 00000100:00100000:0.0:1780298644.422463:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.422482:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609590400 00000100:00100000:2.0:1780298644.422485:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609590400 00000100:00100000:3.0:1780298644.422495:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9859 00000100:00100000:1.0:1780298644.422536:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9860 00000100:00100000:3.0:1780298644.422541:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2400:x1866777609590272:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298644.422548:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609590400:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.422553:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.422554:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.422564:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e63034fb2 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3f5c expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.422566:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e63034fb9 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3f55 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.422592:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605019648:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.422610:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242861c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605019776:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.422634:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.422649:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.423383:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242861c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605019776:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.423387:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605019648:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.423411:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e63034fb2 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f5c expref: 8 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.423415:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e63034fb9 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f55 expref: 19 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.423437:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e63034fb2 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f5c expref: 8 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.423442:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e63034fb9 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f55 expref: 19 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.423467:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157200, rc 0) 00010000:00010000:1.0:1780298644.423469:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278400, rc 0) 00000100:00100000:3.0:1780298644.423502:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2400:x1866777609590272:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 965us (1082us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.423504:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609590400:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 960us (1039us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.423516:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9860 00000100:00100000:3.0:1780298644.423525:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9859 00000100:00100000:0.0:1780298644.424600:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.424633:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.424644:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609590528 00000100:00100000:3.0:1780298644.424651:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609590528 00000100:00100000:2.0:1780298644.424670:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609590656 00000100:00100000:2.0:1780298644.424675:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609590656 00000100:00100000:3.0:1780298644.424689:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9861 00000100:00100000:3.0:1780298644.424699:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2400:x1866777609590528:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.424708:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.424719:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009e0b8e7b/0xaf35132e63034fc0 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3f63 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.424725:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9862 00000100:00100000:1.0:1780298644.424738:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609590656:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.424744:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.424752:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6416acb/0xaf35132e63034fc7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3f6a expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.424761:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242861c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605019904:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.424776:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605020032:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.424813:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.424814:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.425577:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242861c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605019904:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.425601:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009e0b8e7b/0xaf35132e63034fc0 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f63 expref: 19 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.425607:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605020032:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.425615:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009e0b8e7b/0xaf35132e63034fc0 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f63 expref: 19 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.425624:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156c00, rc 0) 00010000:00010000:1.0:1780298644.425628:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a6416acb/0xaf35132e63034fc7 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f6a expref: 8 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.425644:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6416acb/0xaf35132e63034fc7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f6a expref: 8 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.425650:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2400:x1866777609590528:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 953us (1050us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.425662:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9861 00010000:00010000:1.0:1780298644.425666:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278c00, rc 0) 00000100:00100000:1.0:1780298644.425693:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609590656:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 958us (1061us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.425700:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9862 00000100:00100000:0.0:1780298644.426704:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.426731:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609590784 00000100:00100000:0.0:1780298644.426736:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.426737:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609590784 00000100:00100000:3.0:1780298644.426769:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609590912 00000100:00100000:3.0:1780298644.426773:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609590912 00000100:00100000:1.0:1780298644.426782:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9863 00000100:00100000:1.0:1780298644.426793:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2399:x1866777609590784:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298644.426800:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9864 00010000:00010000:1.0:1780298644.426802:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.426811:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63034fce lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3f71 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.426813:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b00000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609590912:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.426827:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.426838:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e63034fd5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3f78 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.426850:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605020160:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.426876:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b03b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605020288:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.426898:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.426910:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733936c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.427781:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b03b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605020288:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.427792:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605020160:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.427800:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e63034fd5 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f78 expref: 8 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.427814:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e63034fd5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f78 expref: 8 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.427814:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63034fce lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f71 expref: 19 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.427825:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baec00, rc 0) 00010000:00010000:1.0:1780298644.427842:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63034fce lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f71 expref: 19 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.427852:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b00000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609590912:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1045us (1118us total) trans 0 rc 301/301 00010000:00010000:1.0:1780298644.427854:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278a00, rc 0) 00000100:00100000:3.0:1780298644.427859:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9864 00000100:00100000:1.0:1780298644.427883:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2399:x1866777609590784:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1091us (1178us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.427890:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9863 00000100:00100000:3.0:1780298644.428913:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.428958:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609591040 00000100:00100000:0.0:1780298644.428971:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609591040 00000100:00100000:0.0:1780298644.429008:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9865 00000100:00100000:0.0:1780298644.429015:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242861c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:29360:x1866777609591040:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.429026:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.429038:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63034fdc lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3f7f expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.429068:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e63034f57 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2371584->35926015] (req 2371584->2379775) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3efa expref: 17 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.429077:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(15 169) 00010000:00010000:0.0:1780298644.429080:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(16 169) 00010000:00010000:0.0:1780298644.429084:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63034fdc lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3448832->3457023] (req 3448832->3457023) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f7f expref: 18 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.429104:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e63034f57 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2371584->35926015] (req 2371584->2379775) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3efa expref: 17 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.429115:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e63034f57 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2371584->35926015] (req 2371584->2379775) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3efa expref: 17 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.429155:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b03b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605020416:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298644.429204:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.429821:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b03b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605020416:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298644.429835:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63034fdc lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3448832->3457023] (req 3448832->3457023) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f7f expref: 18 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.429844:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baee00, rc 0) 00000100:00100000:2.0:1780298644.429868:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242861c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:29360:x1866777609591040:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 855us (956us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.429872:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9865 00000100:00100000:2.0:1780298644.430225:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.430272:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609591168 00000100:00100000:0.0:1780298644.430276:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609591168 00002000:00100000:0.0:1780298644.430311:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (2371584->2379775) req@ffff9cf242b42300 x1866777609591168/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 8680/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298644.430318:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b42300 with x1866777609591168 ext(2371584->2379775) 00010000:00010000:0.0:1780298644.430324:0:8364:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e63034f57 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2371584->35926015] (req 2371584->2379775) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3efa expref: 18 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.430334:0:8364:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e63034f57 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2371584->35926015] (req 2371584->2379775) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3efa expref: 18 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1780298644.430340:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf242b42300 00000100:00100000:0.0:1780298644.430355:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 204 00000100:00100000:0.0:1780298644.430360:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2399:x1866777609591168:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:0.0:1780298644.430806:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2399:x1866777609591168:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 445us (582us total) trans 4294969726 rc 0/0 00000100:00100000:0.0:1780298644.430811:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 204 00002000:00100000:0.0:1780298644.430816:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (2371584->2379775) req@ffff9cf242b42300 x1866777609591168/t4294969726(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 8680/448 e 0 to 0 dl 1780298655 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298644.430822:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b42300 with x1866777609591168 ext(2371584->2379775) 00010000:00010000:0.0:1780298644.430826:0:8364:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e63034f57 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2371584->35926015] (req 2371584->2379775) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3efa expref: 20 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.430834:0:8364:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e63034f57 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2371584->35926015] (req 2371584->2379775) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3efa expref: 20 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1780298644.430839:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf242b42300 00000100:00100000:3.0:1780298644.431570:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.431613:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609591296 00000100:00100000:0.0:1780298644.431617:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609591296 00010000:00010000:0.0:1780298644.431633:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e63034f57 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2371584->35926015] (req 2371584->2379775) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3efa expref: 20 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.431656:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15193 00000100:00100000:0.0:1780298644.431663:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242862d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2399:x1866777609591296:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298644.431674:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:0.0:1780298644.431683:0:6461:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 328 -> 344 00010000:00010000:0.0:1780298644.431689:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e63034f57 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [2371584->35926015] (req 2371584->2379775) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3efa expref: 20 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.431709:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e63034f57 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [2371584->35926015] (req 2371584->2379775) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3efa expref: 20 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.431735:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e63034f57 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [2371584->35926015] (req 2371584->2379775) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3efa expref: 20 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.431745:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(16 169) 00010000:00010000:0.0:1780298644.431747:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(16 169) 00010000:00010000:0.0:1780298644.431753:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63034fdc lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3448832->18446744073709551615] (req 3448832->3457023) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f7f expref: 17 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.431759:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63034fdc lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3448832->18446744073709551615] (req 3448832->3457023) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f7f expref: 17 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.431788:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63034fdc lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3448832->18446744073709551615] (req 3448832->3457023) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f7f expref: 17 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.431800:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b03b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777605020544:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298644.431852:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393d80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.432526:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b03b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777605020544:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298644.432536:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298644.432557:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242862d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+19:2399:x1866777609591296:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 896us (991us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.432562:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15193 00000100:00100000:2.0:1780298644.432844:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.432880:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609591424 00000100:00100000:1.0:1780298644.432885:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609591424 00002000:00100000:1.0:1780298644.432904:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (3448832->3457023) req@ffff9cf242b42a00 x1866777609591424/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.432911:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b42a00 with x1866777609591424 ext(3448832->3457023) 00002000:00010000:1.0:1780298644.432913:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf242b42a00 00000100:00100000:1.0:1780298644.432930:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6309 00000100:00100000:1.0:1780298644.432936:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2399:x1866777609591424:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298644.433050:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2399:x1866777609591424:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 114us (206us total) trans 0 rc 8192/8192 00000100:00100000:1.0:1780298644.433054:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6309 00002000:00100000:1.0:1780298644.433058:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (3448832->3457023) req@ffff9cf242b42a00 x1866777609591424/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/8632 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 8192/8192 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.433063:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b42a00 with x1866777609591424 ext(3448832->3457023) 00002000:00010000:1.0:1780298644.433066:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf242b42a00 00000100:00100000:3.0:1780298644.434226:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.434255:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.434271:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609591552 00000100:00100000:1.0:1780298644.434287:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609591552 00000100:00100000:0.0:1780298644.434292:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609591680 00000100:00100000:0.0:1780298644.434295:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609591680 00000100:00100000:0.0:1780298644.434323:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9866 00000100:00100000:1.0:1780298644.434331:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9867 00000100:00100000:0.0:1780298644.434331:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242863b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2399:x1866777609591552:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298644.434337:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609591680:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.434341:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.434344:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.434354:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6416acb/0xaf35132e63034fea lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3f8d expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.434354:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e63034fe3 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3f86 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.434379:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e63034fe3 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f86 expref: 19 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.434391:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605020672:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.434393:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e63034fe3 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f86 expref: 19 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.434403:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae600, rc 0) 00000100:00100000:0.0:1780298644.434428:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242863b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2399:x1866777609591552:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 98us (205us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.434435:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9866 00000100:00100000:1.0:1780298644.434439:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a540 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.435154:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605020672:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.435173:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a6416acb/0xaf35132e63034fea lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f8d expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.435186:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6416acb/0xaf35132e63034fea lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f8d expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.435193:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278c00, rc 0) 00000100:00100000:1.0:1780298644.435214:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609591680:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 878us (960us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.435219:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9867 00000100:00100000:2.0:1780298644.435994:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.436021:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.436036:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609591808 00000100:00100000:1.0:1780298644.436040:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609591808 00000100:00100000:3.0:1780298644.436048:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609591936 00000100:00100000:3.0:1780298644.436053:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609591936 00000100:00100000:1.0:1780298644.436074:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9868 00000100:00100000:1.0:1780298644.436085:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b42680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2397:x1866777609591808:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298644.436086:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9869 00000100:00100000:3.0:1780298644.436098:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609591936:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.436099:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.436107:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.436111:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e63034ff1 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3f94 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.436115:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e0b8e7b/0xaf35132e63034ff8 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3f9b expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.436157:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e63034ff1 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f94 expref: 19 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.436167:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605020800:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.436170:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e63034ff1 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f94 expref: 19 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.436178:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278400, rc 0) 00000100:00100000:1.0:1780298644.436198:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b42680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2397:x1866777609591808:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 118us (205us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.436204:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.436214:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9868 00000100:00100000:0.0:1780298644.436891:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605020800:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.436927:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009e0b8e7b/0xaf35132e63034ff8 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f9b expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.436942:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e0b8e7b/0xaf35132e63034ff8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f9b expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.436951:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156c00, rc 0) 00000100:00100000:0.0:1780298644.436983:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609591936:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 888us (960us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.436994:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9869 00000100:00100000:0.0:1780298644.437834:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.437876:0:7411:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609592192 00000100:00100000:1.0:1780298644.437881:0:7411:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609592192 00000100:00100000:1.0:1780298644.437919:0:7411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62034 00000100:00100000:1.0:1780298644.437928:0:7411:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b03b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+136:29360:x1866777609592192:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:1.0:1780298644.437979:0:7411:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298644.437998:0:7411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3600/0xaf35132e63034ee0 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 94 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e83 expref: 136 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.438013:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.438018:0:7411:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298644.438210:0:7411:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278a00/0xaf35132e63034fff lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 93 type: IBT flags: 0x40000000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.438225:0:7411:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279c00/0xaf35132e63034eb6 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 93 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e60 expref: 172 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.438248:0:7411:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278a00/0xaf35132e63034fff lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 93 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.438297:0:7411:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279c00/0xaf35132e63034eb6 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 93 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e60 expref: 172 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.438311:0:7411:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279c00/0xaf35132e63034eb6 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 93 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e60 expref: 172 pid: 17578 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.438323:0:7411:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304c380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7411:1866777605020928:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298644.438368:0:7411:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.438941:0:7411:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304c380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7411:1866777605020928:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298644.438956:0:7411:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278a00/0xaf35132e63034fff lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 93 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.439095:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.439152:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609592320 00000100:00100000:1.0:1780298644.439158:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609592320 00010000:00010000:1.0:1780298644.439177:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279c00/0xaf35132e63034eb6 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 93 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e60 expref: 173 pid: 17578 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.439205:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15194 00000100:00100000:1.0:1780298644.439213:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348004e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+173:2399:x1866777609592320:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298644.439222:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298644.439228:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279c00/0xaf35132e63034eb6 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 94 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e60 expref: 173 pid: 17578 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.439239:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279c00/0xaf35132e63034eb6 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 94 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e60 expref: 173 pid: 17578 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.439253:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279c00/0xaf35132e63034eb6 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 94 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e60 expref: 173 pid: 17578 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.439263:0:6463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.439267:0:6463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278a00/0xaf35132e63034fff lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 93 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.439292:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278a00/0xaf35132e63034fff lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 93 type: IBT flags: 0x40290000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.439301:0:6463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278a00/0xaf35132e63034fff lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 93 type: IBT flags: 0x40290000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.439321:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298644.439350:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348004e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+172:2399:x1866777609592320:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 137us (253us total) trans 0 rc 0/0 00010000:00010000:2.0:1780298644.439354:0:7411:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278a00/0xaf35132e63034fff lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 92 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.439359:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15194 00010000:00010000:2.0:1780298644.439368:0:7411:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278a00/0xaf35132e63034fff lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 92 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1780298644.439415:0:7411:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991088 00000004:00010000:2.0:1780298644.439440:0:7411:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 000000006b0e3bc0 reply state 000000002d035628 transno 4294991089 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278a00/0xaf35132e63034fff lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 92 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.439461:0:7411:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278a00/0xaf35132e63034fff lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 92 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.439466:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298644.439468:0:7411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1780298644.439470:0:7411:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1780298644.439520:0:7411:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b03b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+138:29360:x1866777609592192:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1595us (1687us total) trans 4294991089 rc 0/0 00000100:00100000:2.0:1780298644.439525:0:7411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62034 00000020:00080000:2.0:1780298644.439528:0:7411:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1780298644.440050:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.440080:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609592448 00000100:00100000:3.0:1780298644.440086:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609592448 00000100:00100000:3.0:1780298644.440115:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9870 00000100:00100000:3.0:1780298644.440140:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348007b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+19:29360:x1866777609592448:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.440149:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.440159:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e63035006 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3fa2 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.440177:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63034f9d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [69632->942079] (req 69632->81919) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f40 expref: 20 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.440185:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034f7a lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [942080->2371583] (req 942080->954367) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f1d expref: 20 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.440193:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63034fdc lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3448832->18446744073709551615] (req 3448832->3457023) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f7f expref: 17 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.440200:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 3 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(16 169) 00010000:00010000:3.0:1780298644.440204:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(19 169) 00010000:00010000:3.0:1780298644.440208:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e63035006 lrc: 6/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fa2 expref: 20 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.440233:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63034fdc lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3448832->18446744073709551615] (req 3448832->3457023) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f7f expref: 17 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.440243:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63034fdc lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3448832->18446744073709551615] (req 3448832->3457023) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f7f expref: 17 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.440253:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5dbb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605021056:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298644.440307:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034f7a lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [942080->2371583] (req 942080->954367) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f1d expref: 20 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.440317:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034f7a lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [942080->2371583] (req 942080->954367) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f1d expref: 20 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.440328:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605021184:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298644.440366:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63034f9d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [69632->942079] (req 69632->81919) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f40 expref: 20 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.440377:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63034f9d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [69632->942079] (req 69632->81919) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f40 expref: 20 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.440449:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5db480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605021312:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298644.440479:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.440872:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5dbb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605021056:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298644.441156:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605021184:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298644.441268:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5db480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605021312:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298644.441306:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:3.0:1780298644.441314:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e63035006 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fa2 expref: 20 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.441334:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.441350:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609592704 00000100:00100000:2.0:1780298644.441355:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609592704 00000100:00100000:0.0:1780298644.441369:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.441372:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609592576 00010000:00010000:2.0:1780298644.441378:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63034f9d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [69632->942079] (req 69632->81919) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f40 expref: 21 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.441378:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609592576 00010000:00010000:3.0:1780298644.441388:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157200, rc 0) 00010000:00010000:1.0:1780298644.441400:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63034fdc lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3448832->18446744073709551615] (req 3448832->3457023) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f7f expref: 18 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.441407:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609592832 00000100:00100000:2.0:1780298644.441413:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609592832 00000100:00100000:0.0:1780298644.441414:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15195 00000100:00100000:3.0:1780298644.441423:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348007b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:29360:x1866777609592448:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1300us (1371us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.441428:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348af7050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2399:x1866777609592704:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298644.441433:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034f7a lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [942080->2371583] (req 942080->954367) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f1d expref: 22 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.441437:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9870 00010000:00010000:0.0:1780298644.441439:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1780298644.441445:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15196 00010000:00010000:0.0:1780298644.441457:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63034f9d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [69632->942079] (req 69632->81919) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f40 expref: 21 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.441462:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348007100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609592576:12345-192.168.203.25@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1780298644.441474:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15197 00010000:00010000:0.0:1780298644.441474:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63034f9d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [69632->942079] (req 69632->81919) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f40 expref: 21 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.441478:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1780298644.441489:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f755f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2400:x1866777609592832:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298644.441499:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298644.441499:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63034f9d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 8 type: EXT [69632->942079] (req 69632->81919) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f40 expref: 21 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.441509:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63034fdc lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 9 type: EXT [3448832->18446744073709551615] (req 3448832->3457023) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f7f expref: 18 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.441512:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034f7a lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 8 type: EXT [942080->2371583] (req 942080->954367) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f1d expref: 20 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.441515:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298644.441538:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63034fdc lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [3448832->18446744073709551615] (req 3448832->3457023) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f7f expref: 18 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.441545:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348af7050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2399:x1866777609592704:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 121us (240us total) trans 0 rc 0/0 00010000:00010000:2.0:1780298644.441554:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034f7a lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [942080->2371583] (req 942080->954367) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f1d expref: 20 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.441557:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15195 00010000:00010000:1.0:1780298644.441577:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63034fdc lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [3448832->18446744073709551615] (req 3448832->3457023) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f7f expref: 18 pid: 8359 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.441588:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63034f7a lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [942080->2371583] (req 942080->954367) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3f1d expref: 19 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.441590:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(19 169) 00010000:00010000:1.0:1780298644.441598:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(19 169) 00010000:00010000:1.0:1780298644.441607:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e63035006 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fa2 expref: 18 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.441616:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e63035006 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fa2 expref: 18 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.441629:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298644.441643:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e63035006 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fa2 expref: 18 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.441654:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304c380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777605021440:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298644.441662:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f755f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2400:x1866777609592832:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 176us (293us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.441670:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15197 00000100:00100000:1.0:1780298644.441694:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.442340:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304c380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777605021440:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298644.442351:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298644.442376:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348007100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609592576:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 922us (1043us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.442384:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15196 00000100:00100000:3.0:1780298644.442456:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.442496:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609592960 00000100:00100000:1.0:1780298644.442502:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609592960 00000100:00100000:1.0:1780298644.442533:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9871 00000100:00100000:1.0:1780298644.442540:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609592960:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.442548:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.442556:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e6303500d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3fa9 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.442569:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63034f42 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [401408->606207] (req 401408->405503) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ee5 expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.442574:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034ee7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [606208->34160639] (req 606208->614399) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e8a expref: 9 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.442582:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 2 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(17 122) 00010000:00010000:1.0:1780298644.442585:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(19 122) 00010000:00010000:1.0:1780298644.442587:0:20252:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e6303500d lrc: 5/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fa9 expref: 8 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.442599:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034ee7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [606208->34160639] (req 606208->614399) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e8a expref: 9 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.442606:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034ee7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [606208->34160639] (req 606208->614399) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e8a expref: 9 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.442612:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605021568:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298644.442646:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63034f42 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [401408->606207] (req 401408->405503) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ee5 expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.442653:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63034f42 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [401408->606207] (req 401408->405503) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ee5 expref: 8 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.442660:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605021696:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298644.442674:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.443212:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605021568:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298644.443295:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605021696:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298644.443310:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e6303500d lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fa9 expref: 8 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.443323:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279a00, rc 0) 00000100:00100000:1.0:1780298644.443346:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:29360:x1866777609592960:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 808us (892us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.443353:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9871 00000100:00100000:0.0:1780298644.443586:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.443608:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.443621:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609593216 00000100:00100000:2.0:1780298644.443626:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609593216 00010000:00010000:2.0:1780298644.443643:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63034f42 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [401408->606207] (req 401408->405503) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ee5 expref: 8 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.443644:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609593088 00000100:00100000:1.0:1780298644.443649:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609593088 00000100:00100000:2.0:1780298644.443671:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15198 00002000:00100000:1.0:1780298644.443677:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (606208->612874) req@ffff9cf36e5da300 x1866777609593088/t0(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 7160/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00000100:00100000:2.0:1780298644.443683:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f757100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609593216:12345-192.168.203.25@tcp:103:ldlm_bl.0 00002000:00100000:1.0:1780298644.443695:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e5da300 with x1866777609593088 ext(606208->612874) 00010000:00010000:2.0:1780298644.443697:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298644.443705:0:19950:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034ee7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [606208->34160639] (req 606208->614399) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e8a expref: 10 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.443712:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63034f42 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [401408->606207] (req 401408->405503) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ee5 expref: 8 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.443722:0:19950:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034ee7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [606208->34160639] (req 606208->614399) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e8a expref: 10 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.443728:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63034f42 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [401408->606207] (req 401408->405503) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ee5 expref: 8 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1780298644.443735:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9cf36e5da300 00010000:00010000:2.0:1780298644.443750:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63034f42 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [401408->606207] (req 401408->405503) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ee5 expref: 8 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.443761:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 205 00010000:00010000:2.0:1780298644.443763:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298644.443770:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5da300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609593088:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:2.0:1780298644.443791:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f757100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609593216:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 112us (205us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.443798:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15198 00000100:00100000:1.0:1780298644.444239:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5da300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2399:x1866777609593088:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 471us (663us total) trans 4294969274 rc 0/0 00000100:00100000:1.0:1780298644.444245:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 205 00002000:00100000:1.0:1780298644.444249:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (606208->612874) req@ffff9cf36e5da300 x1866777609593088/t4294969274(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 7160/448 e 0 to 0 dl 1780298655 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.444260:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e5da300 with x1866777609593088 ext(606208->612874) 00010000:00010000:1.0:1780298644.444264:0:19950:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034ee7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [606208->34160639] (req 606208->614399) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e8a expref: 12 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.444274:0:19950:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034ee7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [606208->34160639] (req 606208->614399) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e8a expref: 12 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1780298644.444290:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9cf36e5da300 00000100:00100000:3.0:1780298644.444939:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.444984:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609593344 00000100:00100000:0.0:1780298644.444990:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609593344 00010000:00010000:0.0:1780298644.445010:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034ee7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [606208->34160639] (req 606208->614399) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e8a expref: 12 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.445039:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15199 00000100:00100000:0.0:1780298644.445047:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5dbb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609593344:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298644.445057:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:0.0:1780298644.445064:0:6461:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 24 -> 40 00010000:00010000:0.0:1780298644.445070:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034ee7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [606208->34160639] (req 606208->614399) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e8a expref: 12 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.445082:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034ee7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [606208->34160639] (req 606208->614399) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e8a expref: 12 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.445096:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e63034ee7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [606208->34160639] (req 606208->614399) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3e8a expref: 12 pid: 9128 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.445108:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(19 122) 00010000:00010000:0.0:1780298644.445112:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(19 122) 00010000:00010000:0.0:1780298644.445118:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e6303500d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fa9 expref: 6 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.445148:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e6303500d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fa9 expref: 6 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.445175:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e6303500d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fa9 expref: 6 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.445188:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f754a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6461:1866777605021824:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298644.445229:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393b40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.445901:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f754a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6461:1866777605021824:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298644.445911:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298644.445931:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5dbb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609593344:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 887us (993us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.445938:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15199 00000100:00100000:3.0:1780298644.446020:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.446254:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609593472 00000100:00100000:1.0:1780298644.446260:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609593472 00002000:00010000:1.0:1780298644.446291:0:8363:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (360462->18446744073709551615) 00002000:00100000:1.0:1780298644.446296:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e5d9c00 with x1866777609593472 ext(360462->18446744073709551615) 00002000:00010000:1.0:1780298644.446304:0:8363:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf36e5d9c00. 00000100:00100000:2.0:1780298644.446321:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.446332:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609593600 00000100:00100000:1.0:1780298644.446344:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609593600 00000100:00100000:3.0:1780298644.446356:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6310 00002000:00010000:1.0:1780298644.446359:0:8363:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00002000:00100000:1.0:1780298644.446363:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b43800 with x1866777609593600 ext(0->18446744073709551615) 00002000:00010000:1.0:1780298644.446368:0:8363:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf242b43800. 00000100:00100000:2.0:1780298644.446384:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2399:x1866777609593472:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:1.0:1780298644.446394:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6311 00000100:00100000:1.0:1780298644.446404:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b43800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609593600:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:1.0:1780298644.446568:0:8363:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 612875 -> 0 00002000:00010000:1.0:1780298644.446571:0:8363:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298644.446573:0:8363:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298644.446575:0:8363:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298644.446577:0:8363:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 40 -> 0 00000100:00100000:1.0:1780298644.446601:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609593600:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 201us (281us total) trans 4294969275 rc 0/0 00000100:00100000:1.0:1780298644.446607:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6311 00002000:00010000:1.0:1780298644.446609:0:8363:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00002000:00100000:1.0:1780298644.446611:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b43800 with x1866777609593600 ext(0->18446744073709551615) 00002000:00010000:1.0:1780298644.446615:0:8363:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf242b43800. 00002000:00010000:2.0:1780298644.447267:0:19943:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 3609904 -> 360462 00002000:00010000:2.0:1780298644.447270:0:19943:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298644.447272:0:19943:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298644.447273:0:19943:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298644.447275:0:19943:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 344 -> 296 00000100:00100000:2.0:1780298644.447311:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+19:2399:x1866777609593472:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 949us (1292us total) trans 4294969727 rc 0/0 00000100:00100000:2.0:1780298644.447316:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6310 00002000:00010000:2.0:1780298644.447319:0:19943:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (360462->18446744073709551615) 00002000:00100000:2.0:1780298644.447321:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e5d9c00 with x1866777609593472 ext(360462->18446744073709551615) 00002000:00010000:2.0:1780298644.447325:0:19943:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf36e5d9c00. 00000100:00100000:2.0:1780298644.448031:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.448064:0:6476:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609593728 00000100:00100000:1.0:1780298644.448068:0:6476:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609593728 00000100:00100000:1.0:1780298644.448095:0:6476:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62035 00000100:00100000:1.0:1780298644.448100:0:6476:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b41f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+138:29360:x1866777609593728:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.448114:0:6476:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.448161:0:6476:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278000/0xaf35132e63035014 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 93 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3fb0 expref: -99 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.448221:0:6476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278600/0xaf35132e6303501b lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 94 type: IBT flags: 0x40000000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.448231:0:6476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278600/0xaf35132e6303501b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 94 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.448239:0:6476:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278600/0xaf35132e6303501b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 94 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1780298644.448258:0:6476:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 360462 00000004:00200000:1.0:1780298644.448261:0:6476:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1780298644.448275:0:6476:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278600/0xaf35132e6303501b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 94 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.448309:0:6476:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278000/0xaf35132e63035014 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 94 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fb0 expref: 140 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.448323:0:6476:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278600/0xaf35132e6303501b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 93 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fb0 expref: 139 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.448328:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.448330:0:6476:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1780298644.448332:0:6476:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278600, rc 0) 00000100:00100000:1.0:1780298644.448356:0:6476:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b41f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+139:29360:x1866777609593728:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 257us (325us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.448360:0:6476:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62035 00000100:00100000:0.0:1780298644.449701:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.449741:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609593856 00000100:00100000:2.0:1780298644.449746:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609593856 00000100:00100000:2.0:1780298644.449776:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9872 00000100:00100000:2.0:1780298644.449782:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f755180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2399:x1866777609593856:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.449789:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.449798:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e63035022 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3fb7 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.449830:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b40000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605021952:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.449839:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.449878:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609593984 00000100:00100000:2.0:1780298644.449882:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb13c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.449883:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609593984 00000100:00100000:0.0:1780298644.449922:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9873 00000100:00100000:0.0:1780298644.449929:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609593984:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.449936:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.449943:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63035029 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3fbe expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.449969:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f757b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605022080:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.450003:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393b40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.450676:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b40000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605021952:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.450684:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f757b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605022080:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.450704:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e63035022 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fb7 expref: 18 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.450709:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63035029 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fbe expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.450724:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e63035022 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fb7 expref: 18 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.450731:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63035029 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fbe expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.450734:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3c00, rc 0) 00010000:00010000:0.0:1780298644.450745:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf400, rc 0) 00000100:00100000:2.0:1780298644.450763:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f755180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2399:x1866777609593856:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 980us (1063us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.450767:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609593984:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 837us (930us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.450775:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9872 00000100:00100000:0.0:1780298644.450784:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9873 00000100:00100000:3.0:1780298644.451676:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.451710:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609594112 00000100:00100000:0.0:1780298644.451715:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609594112 00000100:00100000:0.0:1780298644.451744:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9874 00000100:00100000:0.0:1780298644.451749:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:29360:x1866777609594112:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.451756:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.451763:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e63035030 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3fc5 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.451776:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e63035006 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fa2 expref: 18 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.451782:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(19 169) 00010000:00010000:0.0:1780298644.451784:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(20 169) 00010000:00010000:0.0:1780298644.451787:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e63035030 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [114688->118783] (req 114688->118783) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fc5 expref: 18 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.451801:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e63035006 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fa2 expref: 18 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.451810:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e63035006 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fa2 expref: 18 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.451816:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f757b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605022208:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298644.451844:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393b40 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.452485:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f757b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605022208:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298644.452501:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e63035030 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [114688->118783] (req 114688->118783) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fc5 expref: 18 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.452510:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae600, rc 0) 00000100:00100000:0.0:1780298644.452537:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:29360:x1866777609594112:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 788us (862us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.452541:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9874 00000100:00100000:3.0:1780298644.452643:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.452683:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609594240 00000100:00100000:1.0:1780298644.452688:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609594240 00010000:00010000:1.0:1780298644.452714:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e63035006 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fa2 expref: 19 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.452743:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15200 00000100:00100000:1.0:1780298644.452752:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+19:2400:x1866777609594240:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298644.452761:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298644.452772:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e63035006 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fa2 expref: 19 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.452785:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e63035006 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fa2 expref: 19 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.452799:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e63035006 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fa2 expref: 19 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.452809:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(20 169) 00010000:00010000:1.0:1780298644.452812:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(20 169) 00010000:00010000:1.0:1780298644.452820:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e63035030 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 114688->118783) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fc5 expref: 17 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.452828:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e63035030 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 114688->118783) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fc5 expref: 17 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.452850:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e63035030 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 114688->118783) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fc5 expref: 17 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.452860:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304ea00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777605022336:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298644.452898:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.453575:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304ea00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777605022336:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298644.453586:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298644.453615:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2400:x1866777609594240:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 865us (972us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.453626:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15200 00000100:00100000:0.0:1780298644.453914:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.453955:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609594368 00000100:00100000:2.0:1780298644.453960:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609594368 00002000:00100000:2.0:1780298644.453989:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (114688->118783) req@ffff9cf37f757b80 x1866777609594368/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.454000:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f757b80 with x1866777609594368 ext(114688->118783) 00002000:00010000:2.0:1780298644.454004:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f757b80 00000100:00100000:2.0:1780298644.454025:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6312 00000100:00100000:2.0:1780298644.454032:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f757b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2399:x1866777609594368:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298644.454157:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f757b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2399:x1866777609594368:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 126us (244us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298644.454161:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6312 00002000:00100000:2.0:1780298644.454167:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (114688->118783) req@ffff9cf37f757b80 x1866777609594368/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/4536 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.454173:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f757b80 with x1866777609594368 ext(114688->118783) 00002000:00010000:2.0:1780298644.454175:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f757b80 00000100:00100000:3.0:1780298644.455084:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.455152:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609594496 00000100:00100000:1.0:1780298644.455157:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609594496 00000100:00100000:1.0:1780298644.455193:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9875 00000100:00100000:1.0:1780298644.455200:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2399:x1866777609594496:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.455208:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.455220:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e63035037 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3fcc expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.455238:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(20 169) 00010000:00010000:1.0:1780298644.455242:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(20 169) 00010000:00010000:1.0:1780298644.455248:0:20252:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e63035037 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fcc expref: 19 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.455260:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e63035037 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fcc expref: 19 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.455270:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278400, rc 0) 00000100:00100000:1.0:1780298644.455306:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+19:2399:x1866777609594496:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 107us (224us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.455313:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9875 00000100:00100000:0.0:1780298644.456290:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.456327:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609594624 00000100:00100000:2.0:1780298644.456332:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609594624 00000100:00100000:2.0:1780298644.456359:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9876 00000100:00100000:2.0:1780298644.456367:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f754380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609594624:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.456375:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.456384:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e6303503e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3fd3 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.456418:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b40000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605022464:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.456462:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb13c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.457192:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b40000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605022464:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.457214:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e6303503e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fd3 expref: 12 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.457231:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e6303503e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fd3 expref: 12 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.457240:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2800, rc 0) 00000100:00100000:2.0:1780298644.457268:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f754380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609594624:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 902us (990us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.457274:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9876 00000100:00100000:3.0:1780298644.458313:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.458357:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609594752 00000100:00100000:1.0:1780298644.458368:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609594752 00002000:00100000:1.0:1780298644.458410:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (77824->81919) req@ffff9cf24310fb80 x1866777609594752/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.458422:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24310fb80 with x1866777609594752 ext(77824->81919) 00002000:00010000:1.0:1780298644.458427:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24310fb80 00000100:00100000:1.0:1780298644.458453:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6313 00000100:00100000:1.0:1780298644.458462:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2397:x1866777609594752:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298644.458635:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2397:x1866777609594752:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 174us (320us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298644.458643:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6313 00002000:00100000:1.0:1780298644.458652:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (77824->81919) req@ffff9cf24310fb80 x1866777609594752/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/4536 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.458661:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24310fb80 with x1866777609594752 ext(77824->81919) 00002000:00010000:1.0:1780298644.458665:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24310fb80 00000100:00100000:0.0:1780298644.460057:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.460111:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609594880 00000100:00100000:1.0:1780298644.460119:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609594880 00002000:00100000:1.0:1780298644.460182:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (53248->114687) req@ffff9cf37f755c00 x1866777609594880/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 504/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.460192:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f755c00 with x1866777609594880 ext(53248->114687) 00002000:00010000:1.0:1780298644.460196:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f755c00 00000100:00100000:1.0:1780298644.460223:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6314 00000100:00100000:1.0:1780298644.460231:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f755c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2399:x1866777609594880:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298644.461151:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f755c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2399:x1866777609594880:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 920us (1096us total) trans 0 rc 57344/57344 00000100:00100000:1.0:1780298644.461158:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6314 00002000:00100000:1.0:1780298644.461165:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (53248->114687) req@ffff9cf37f755c00 x1866777609594880/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 504/440 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 57344/57344 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.461171:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f755c00 with x1866777609594880 ext(53248->114687) 00002000:00010000:1.0:1780298644.461174:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f755c00 00000100:00100000:0.0:1780298644.463864:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.463923:0:17578:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609595008 00000100:00100000:1.0:1780298644.463931:0:17578:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609595008 00000100:00100000:1.0:1780298644.463982:0:17578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62036 00000100:00100000:1.0:1780298644.463994:0:17578:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f755880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+172:29360:x1866777609595008:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.464009:0:17578:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.464030:0:17578:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279c00/0xaf35132e63035045 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 94 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3fda expref: -99 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.464154:0:17578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278c00/0xaf35132e6303504c lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 95 type: IBT flags: 0x40000000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.464173:0:17578:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278c00/0xaf35132e6303504c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 95 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.464186:0:17578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278c00/0xaf35132e6303504c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 95 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1780298644.464218:0:17578:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 360462 00000004:00200000:1.0:1780298644.464223:0:17578:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1780298644.464242:0:17578:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278c00/0xaf35132e6303504c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 95 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.464264:0:17578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279c00/0xaf35132e63035045 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 95 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fda expref: 174 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.464294:0:17578:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278c00/0xaf35132e6303504c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 94 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fda expref: 173 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.464301:0:17578:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.464303:0:17578:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1780298644.464306:0:17578:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278c00, rc 0) 00000100:00100000:1.0:1780298644.464335:0:17578:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f755880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+173:29360:x1866777609595008:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 347us (476us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.464340:0:17578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62036 00000100:00100000:3.0:1780298644.465223:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.465271:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609595136 00000100:00100000:2.0:1780298644.465290:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609595136 00000100:00100000:2.0:1780298644.465332:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9877 00000100:00100000:2.0:1780298644.465340:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255492300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609595136:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.465352:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.465364:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63035053 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3fe1 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.465410:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b40000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605022592:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.465478:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb13c0 going to sleep for 11 seconds 00002000:00010000:2.0:1780298644.466227:0:19992:0:(ofd_lvb.c:259:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size: 0 -> 2212497 00002000:00010000:2.0:1780298644.466231:0:19992:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks: 0 -> 1 00000100:00100000:2.0:1780298644.466245:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b40000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605022592:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.466272:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63035053 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fe1 expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.466303:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63035053 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fe1 expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.466314:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3c00, rc 0) 00000100:00100000:2.0:1780298644.466348:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255492300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609595136:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1008us (1128us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.466356:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9877 00000100:00100000:0.0:1780298644.467339:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.467380:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609595264 00000100:00100000:3.0:1780298644.467385:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609595264 00000100:00100000:3.0:1780298644.467420:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9878 00000100:00100000:3.0:1780298644.467429:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f754a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609595264:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.467439:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.467449:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eae09c9b/0xaf35132e6303505a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3fe8 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.467481:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255493480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605022720:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.467527:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.468319:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255493480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605022720:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.468340:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000eae09c9b/0xaf35132e6303505a lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fe8 expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.468356:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eae09c9b/0xaf35132e6303505a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fe8 expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.468365:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157a00, rc 0) 00000100:00100000:0.0:1780298644.468396:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f754a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609595264:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 969us (1056us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.468403:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9878 00000100:00100000:0.0:1780298644.469696:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.469743:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609595392 00000100:00100000:3.0:1780298644.469749:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609595392 00002000:00100000:3.0:1780298644.469780:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x2c0000401:0x11:0x0] (294912->303103) req@ffff9cf244adbb80 x1866777609595392/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298644.469791:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244adbb80 with x1866777609595392 ext(294912->303103) 00002000:00010000:3.0:1780298644.469796:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf244adbb80 00000100:00100000:3.0:1780298644.469819:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6315 00000100:00100000:3.0:1780298644.469829:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+19:2399:x1866777609595392:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298644.469979:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+19:2399:x1866777609595392:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 153us (284us total) trans 0 rc 8192/8192 00000100:00100000:3.0:1780298644.469987:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6315 00002000:00100000:3.0:1780298644.469993:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x2c0000401:0x11:0x0] (294912->303103) req@ffff9cf244adbb80 x1866777609595392/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/8632 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 8192/8192 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298644.470002:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244adbb80 with x1866777609595392 ext(294912->303103) 00002000:00010000:3.0:1780298644.470005:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf244adbb80 00000100:00100000:1.0:1780298644.471746:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.471787:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609595520 00000100:00100000:2.0:1780298644.471792:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609595520 00000100:00100000:2.0:1780298644.471826:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9879 00000100:00100000:2.0:1780298644.471835:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609595520:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.471847:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.471861:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63035061 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3fef expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.471901:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b40000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605022848:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.471957:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb13c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.472748:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b40000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605022848:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.472771:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63035061 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fef expref: 12 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.472784:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63035061 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fef expref: 12 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.472793:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2800, rc 0) 00000100:00100000:2.0:1780298644.472827:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609595520:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 993us (1090us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.472834:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9879 00000100:00100000:0.0:1780298644.474015:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.474061:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609595648 00000100:00100000:2.0:1780298644.474066:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609595648 00000100:00100000:2.0:1780298644.474103:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9880 00000100:00100000:2.0:1780298644.474113:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609595648:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.474178:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.474191:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e63035068 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3ff6 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.474236:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b40000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605022976:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.474298:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb13c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.475051:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b40000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605022976:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.475071:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e63035068 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ff6 expref: 12 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.475087:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e63035068 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ff6 expref: 12 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.475098:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3800, rc 0) 00000100:00100000:2.0:1780298644.475173:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609595648:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1061us (1157us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.475181:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9880 00000100:00100000:0.0:1780298644.476169:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.476228:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609595776 00000100:00100000:2.0:1780298644.476232:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609595776 00002000:00100000:2.0:1780298644.476256:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (1552384->1556479) req@ffff9cf242862d80 x1866777609595776/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.476264:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242862d80 with x1866777609595776 ext(1552384->1556479) 00002000:00010000:2.0:1780298644.476269:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf242862d80 00000100:00100000:2.0:1780298644.476297:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6316 00000100:00100000:2.0:1780298644.476303:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242862d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2399:x1866777609595776:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298644.476422:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242862d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2399:x1866777609595776:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 118us (253us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.476430:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6316 00002000:00100000:2.0:1780298644.476437:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (1552384->1556479) req@ffff9cf242862d80 x1866777609595776/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/440 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.476446:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242862d80 with x1866777609595776 ext(1552384->1556479) 00002000:00010000:2.0:1780298644.476450:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf242862d80 00000100:00100000:0.0:1780298644.477665:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.477705:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609595904 00000100:00100000:2.0:1780298644.477710:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609595904 00000100:00100000:2.0:1780298644.477739:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9881 00000100:00100000:2.0:1780298644.477747:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348004e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609595904:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.477758:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.477771:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e6303506f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d3ffd expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.477809:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b40000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605023104:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.477859:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb13c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.478676:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b40000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605023104:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.478698:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e6303506f lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ffd expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.478715:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e6303506f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3ffd expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.478726:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3c00, rc 0) 00000100:00100000:2.0:1780298644.478759:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348004e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609595904:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1013us (1094us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.478767:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9881 00000100:00100000:0.0:1780298644.479672:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.479715:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609596032 00000100:00100000:2.0:1780298644.479721:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609596032 00002000:00100000:2.0:1780298644.479748:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (1556480->1560575) req@ffff9cf347cd5880 x1866777609596032/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.479758:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf347cd5880 with x1866777609596032 ext(1556480->1560575) 00002000:00010000:2.0:1780298644.479763:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf347cd5880 00000100:00100000:2.0:1780298644.479785:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6317 00000100:00100000:2.0:1780298644.479795:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2399:x1866777609596032:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298644.479933:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2399:x1866777609596032:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 139us (260us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.479941:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6317 00002000:00100000:2.0:1780298644.479950:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (1556480->1560575) req@ffff9cf347cd5880 x1866777609596032/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/440 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.479960:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf347cd5880 with x1866777609596032 ext(1556480->1560575) 00002000:00010000:2.0:1780298644.479963:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf347cd5880 00000100:00100000:0.0:1780298644.481289:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.481338:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609596160 00000100:00100000:3.0:1780298644.481343:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609596160 00000100:00100000:3.0:1780298644.481374:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9882 00000100:00100000:3.0:1780298644.481384:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609596160:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.481393:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.481408:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000004b2c7ae/0xaf35132e63035076 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4004 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.481450:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255493800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605023232:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.481514:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.482300:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255493800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605023232:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.482318:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000004b2c7ae/0xaf35132e63035076 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4004 expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.482330:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000004b2c7ae/0xaf35132e63035076 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4004 expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.482338:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156400, rc 0) 00000100:00100000:3.0:1780298644.482367:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609596160:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 986us (1089us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.482376:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9882 00000100:00100000:0.0:1780298644.483317:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.483355:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609596288 00000100:00100000:2.0:1780298644.483360:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609596288 00000100:00100000:2.0:1780298644.483394:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9883 00000100:00100000:2.0:1780298644.483403:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:29360:x1866777609596288:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.483412:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.483422:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e6303507d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d400b expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.483446:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e6303500d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fa9 expref: 7 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.483456:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(19 122) 00010000:00010000:2.0:1780298644.483460:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(20 122) 00010000:00010000:2.0:1780298644.483465:0:20252:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e6303507d lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1470464->1478655] (req 1470464->1478655) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d400b expref: 12 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.483489:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e6303500d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fa9 expref: 7 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.483503:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e6303500d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fa9 expref: 7 pid: 20252 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.483514:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b40000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605023360:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298644.483559:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb13c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.484271:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b40000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605023360:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298644.484303:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e6303507d lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1470464->1478655] (req 1470464->1478655) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d400b expref: 12 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.484317:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3800, rc 0) 00000100:00100000:2.0:1780298644.484353:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:29360:x1866777609596288:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 952us (1036us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.484363:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9883 00000100:00100000:1.0:1780298644.484725:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.484767:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609596416 00000100:00100000:0.0:1780298644.484772:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609596416 00002000:00100000:0.0:1780298644.484793:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (2203648->2212496) req@ffff9cf375f55850 x1866777609596416/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 9344/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298644.484800:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf375f55850 with x1866777609596416 ext(2203648->2212496) 00010000:00010000:0.0:1780298644.484806:0:8362:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e6303500d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fa9 expref: 8 pid: 20252 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.484816:0:8362:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e6303500d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fa9 expref: 8 pid: 20252 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1780298644.484821:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9cf375f55850 00000100:00100000:0.0:1780298644.484838:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 206 00000100:00100000:0.0:1780298644.484842:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf375f55850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609596416:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:0.0:1780298644.485424:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf375f55850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609596416:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 580us (704us total) trans 4294969276 rc 0/0 00000100:00100000:0.0:1780298644.485431:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 206 00002000:00100000:0.0:1780298644.485438:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (2203648->2212496) req@ffff9cf375f55850 x1866777609596416/t4294969276(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 9344/448 e 0 to 0 dl 1780298655 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298644.485453:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf375f55850 with x1866777609596416 ext(2203648->2212496) 00010000:00010000:0.0:1780298644.485458:0:8362:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e6303500d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fa9 expref: 10 pid: 20252 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.485471:0:8362:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e6303500d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fa9 expref: 10 pid: 20252 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1780298644.485478:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9cf375f55850 00000100:00100000:0.0:1780298644.486305:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.486348:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609596544 00000100:00100000:3.0:1780298644.486354:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609596544 00010000:00010000:3.0:1780298644.486376:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e6303500d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fa9 expref: 10 pid: 20252 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.486400:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15201 00000100:00100000:3.0:1780298644.486408:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609596544:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298644.486418:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:3.0:1780298644.486427:0:6463:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 1 -> 24 00010000:00010000:3.0:1780298644.486433:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e6303500d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fa9 expref: 10 pid: 20252 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.486444:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e6303500d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fa9 expref: 10 pid: 20252 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.486463:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e6303500d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fa9 expref: 10 pid: 20252 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.486476:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(20 122) 00010000:00010000:3.0:1780298644.486479:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(20 122) 00010000:00010000:3.0:1780298644.486487:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e6303507d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1470464->35024895] (req 1470464->1478655) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d400b expref: 11 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.486497:0:6463:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e6303507d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1470464->35024895] (req 1470464->1478655) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d400b expref: 11 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.486521:0:6463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e6303507d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1470464->35024895] (req 1470464->1478655) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d400b expref: 11 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.486532:0:6463:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255493800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777605023488:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298644.486576:0:6463:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.487411:0:6463:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255493800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777605023488:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298644.487436:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298644.487512:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609596544:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1102us (1204us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.487524:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15201 00000100:00100000:0.0:1780298644.487542:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.487587:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609596672 00000100:00100000:3.0:1780298644.487594:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609596672 00000100:00100000:1.0:1780298644.487677:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6318 00000100:00100000:1.0:1780298644.487693:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609596672:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298644.487882:0:8363:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 2212497 -> 2212497 00002000:00010000:1.0:1780298644.487886:0:8363:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298644.487889:0:8363:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298644.487892:0:8363:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:1.0:1780298644.487894:0:8363:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 24 -> 40 00000100:00100000:1.0:1780298644.487929:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609596672:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 242us (389us total) trans 4294969277 rc 0/0 00000100:00100000:1.0:1780298644.487938:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6318 00000100:00100000:2.0:1780298644.488959:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.489014:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609596800 00000100:00100000:0.0:1780298644.489019:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609596800 00000100:00100000:0.0:1780298644.489053:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9884 00000100:00100000:0.0:1780298644.489062:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b40000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609596800:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.489074:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.489089:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e63035084 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4012 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.489155:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605023616:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.489207:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393b40 going to sleep for 11 seconds 00002000:00010000:0.0:1780298644.489849:0:9128:0:(ofd_lvb.c:277:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime: 1780298644 -> 1780298645 00000100:00100000:0.0:1780298644.489863:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605023616:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.489885:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e63035084 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4012 expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.489908:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e63035084 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4012 expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.489919:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf200, rc 0) 00000100:00100000:0.0:1780298644.489949:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b40000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609596800:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 889us (990us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.489957:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9884 00000100:00100000:3.0:1780298644.490826:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.490862:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609596928 00000100:00100000:2.0:1780298644.490872:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609596928 00000100:00100000:2.0:1780298644.490899:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9885 00000100:00100000:2.0:1780298644.490904:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255490380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609596928:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.490910:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.490917:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e6303508b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4019 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.490941:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b41180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605023744:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.490979:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb13c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.491633:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b41180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605023744:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.491650:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e6303508b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4019 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.491664:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e6303508b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4019 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.491673:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3000, rc 0) 00000100:00100000:0.0:1780298644.491703:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255490380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609596928:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 795us (874us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.491710:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9885 00000100:00100000:3.0:1780298644.492768:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.492838:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609597056 00000100:00100000:1.0:1780298644.492849:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609597056 00002000:00100000:1.0:1780298644.492894:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x2c0000401:0x11:0x0] (2502656->2506751) req@ffff9cf255491500 x1866777609597056/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.492906:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255491500 with x1866777609597056 ext(2502656->2506751) 00002000:00010000:1.0:1780298644.492912:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255491500 00000100:00100000:1.0:1780298644.492940:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6319 00000100:00100000:1.0:1780298644.492950:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255491500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2397:x1866777609597056:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298644.493107:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255491500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2397:x1866777609597056:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 160us (341us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.493112:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6319 00002000:00100000:1.0:1780298644.493117:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x2c0000401:0x11:0x0] (2502656->2506751) req@ffff9cf255491500 x1866777609597056/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/440 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.493166:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255491500 with x1866777609597056 ext(2502656->2506751) 00002000:00010000:1.0:1780298644.493170:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255491500 00000100:00100000:3.0:1780298644.493794:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.493842:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609597184 00000100:00100000:2.0:1780298644.493849:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609597184 00002000:00100000:2.0:1780298644.493879:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (2506752->2510847) req@ffff9cf255493800 x1866777609597184/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.493886:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255493800 with x1866777609597184 ext(2506752->2510847) 00002000:00010000:2.0:1780298644.493889:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255493800 00000100:00100000:2.0:1780298644.493907:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6320 00000100:00100000:2.0:1780298644.493914:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255493800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2398:x1866777609597184:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298644.494053:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255493800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2398:x1866777609597184:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 139us (258us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.494060:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6320 00002000:00100000:2.0:1780298644.494068:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (2506752->2510847) req@ffff9cf255493800 x1866777609597184/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/440 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.494077:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255493800 with x1866777609597184 ext(2506752->2510847) 00002000:00010000:2.0:1780298644.494080:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255493800 00000100:00100000:2.0:1780298644.495047:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.495071:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609597312 00000100:00100000:3.0:1780298644.495075:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609597312 00000100:00100000:3.0:1780298644.495100:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9886 00000100:00100000:3.0:1780298644.495106:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609597312:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.495114:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.495142:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c841b52/0xaf35132e63035092 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4020 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.495224:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255492a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605023872:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.495294:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856cc0 going to sleep for 11 seconds 00002000:00010000:2.0:1780298644.495929:0:8358:0:(ofd_lvb.c:271:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime: 1780298644 -> 1780298645 00000100:00100000:2.0:1780298644.495942:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255492a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605023872:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.495968:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009c841b52/0xaf35132e63035092 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4020 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.495989:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c841b52/0xaf35132e63035092 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4020 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.496001:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157600, rc 0) 00000100:00100000:2.0:1780298644.496036:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609597312:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 930us (989us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.496042:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9886 00000100:00100000:2.0:1780298644.497035:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.497081:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609597440 00000100:00100000:1.0:1780298644.497088:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609597440 00000100:00100000:1.0:1780298644.497137:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9887 00000100:00100000:1.0:1780298644.497147:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b43100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609597440:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.497159:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.497173:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63035099 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4027 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.497231:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605024000:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.497303:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837ae40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.498082:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605024000:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.498105:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63035099 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4027 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.498121:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63035099 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4027 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.498149:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279200, rc 0) 00000100:00100000:2.0:1780298644.498177:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609597440:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1034us (1143us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.498183:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9887 00000100:00100000:3.0:1780298644.499393:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.499436:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609597568 00000100:00100000:0.0:1780298644.499443:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609597568 00000100:00100000:0.0:1780298644.499502:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9888 00000100:00100000:0.0:1780298644.499512:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255490e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609597568:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.499527:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.499540:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e630350a0 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d402e expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.499585:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605024128:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.499648:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393b40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.500409:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605024128:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.500432:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e630350a0 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d402e expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.500448:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e630350a0 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d402e expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.500458:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baec00, rc 0) 00000100:00100000:2.0:1780298644.500495:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255490e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609597568:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 986us (1107us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.500501:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9888 00000100:00100000:1.0:1780298644.501408:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.501461:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609597696 00000100:00100000:0.0:1780298644.501469:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609597696 00000100:00100000:0.0:1780298644.501511:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9889 00000100:00100000:0.0:1780298644.501520:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+19:29360:x1866777609597696:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.501531:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.501544:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e630350a7 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4035 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.501567:0:19992:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e63035030 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 114688->118783) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fc5 expref: 17 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.501577:0:19992:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e63035037 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fcc expref: 20 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.501585:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(20 169) 00010000:00010000:0.0:1780298644.501590:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(20 169) 00010000:00010000:0.0:1780298644.501595:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e630350a7 lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [720896->729087] (req 720896->729087) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4035 expref: 20 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.501621:0:19992:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e63035037 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fcc expref: 20 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.501641:0:19992:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e63035037 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fcc expref: 20 pid: 20252 timeout: 933 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.501653:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605024256:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298644.501720:0:19992:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e63035030 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 114688->118783) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fc5 expref: 17 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.501732:0:19992:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e63035030 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 114688->118783) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fc5 expref: 17 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.501744:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605024384:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298644.501791:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.502414:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605024256:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298644.502661:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605024384:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298644.502689:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e630350a7 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [720896->729087] (req 720896->729087) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4035 expref: 20 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.502703:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae400, rc 0) 00000100:00100000:0.0:1780298644.502737:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:29360:x1866777609597696:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1219us (1331us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.502744:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9889 00000100:00100000:3.0:1780298644.502834:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.502877:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609597824 00000100:00100000:2.0:1780298644.502883:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609597824 00010000:00010000:2.0:1780298644.502922:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e63035037 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fcc expref: 20 pid: 20252 timeout: 933 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.502955:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15202 00000100:00100000:2.0:1780298644.502965:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348af4050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2400:x1866777609597824:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298644.502976:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298644.502990:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e63035037 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fcc expref: 20 pid: 20252 timeout: 933 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.503000:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e63035037 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fcc expref: 20 pid: 20252 timeout: 933 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.503015:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:2.0:1780298644.503020:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e63035037 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fcc expref: 20 pid: 20252 timeout: 933 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.503033:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298644.503063:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609597952 00000100:00100000:2.0:1780298644.503069:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348af4050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+19:2400:x1866777609597824:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 104us (236us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.503070:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609597952 00000100:00100000:2.0:1780298644.503079:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15202 00010000:00010000:1.0:1780298644.503097:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e63035030 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 114688->118783) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fc5 expref: 18 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.503164:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15203 00000100:00100000:1.0:1780298644.503176:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255491f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609597952:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298644.503186:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298644.503197:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e63035030 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 114688->118783) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fc5 expref: 18 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.503212:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e63035030 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 114688->118783) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fc5 expref: 18 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.503228:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e63035030 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 114688->118783) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fc5 expref: 18 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.503241:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(20 169) 00010000:00010000:1.0:1780298644.503246:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(20 169) 00010000:00010000:1.0:1780298644.503254:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e630350a7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [720896->34275327] (req 720896->729087) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4035 expref: 18 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.503270:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e630350a7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [720896->34275327] (req 720896->729087) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4035 expref: 18 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.503313:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e630350a7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [720896->34275327] (req 720896->729087) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4035 expref: 18 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.503326:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304ce00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777605024512:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298644.503382:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837af00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.504259:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304ce00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777605024512:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298644.504273:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298644.504314:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255491f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609597952:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1140us (1305us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.504322:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15203 00000100:00100000:3.0:1780298644.504384:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.504430:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609598080 00000100:00100000:2.0:1780298644.504437:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609598080 00000100:00100000:2.0:1780298644.504487:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6321 00000100:00100000:2.0:1780298644.504496:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255493480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+19:2400:x1866777609598080:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298644.504732:0:8364:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 360462 -> 360462 00002000:00010000:2.0:1780298644.504736:0:8364:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298644.504739:0:8364:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298644.504741:0:8364:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298644 -> 1780298644 00002000:00010000:2.0:1780298644.504743:0:8364:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 296 -> 312 00000100:00100000:2.0:1780298644.504773:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255493480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2400:x1866777609598080:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 279us (390us total) trans 4294969728 rc 0/0 00000100:00100000:2.0:1780298644.504783:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6321 00000100:00100000:0.0:1780298644.505822:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.505860:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.505887:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609598208 00000100:00100000:2.0:1780298644.505894:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609598208 00000100:00100000:2.0:1780298644.505931:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609598336 00000100:00100000:2.0:1780298644.505935:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609598336 00000100:00100000:3.0:1780298644.505941:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9890 00000100:00100000:3.0:1780298644.505954:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2399:x1866777609598208:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.505963:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.505974:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e630350ae lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d403c expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.505982:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9891 00000100:00100000:2.0:1780298644.505992:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609598336:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.506002:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298644.506013:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255492a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605024640:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.506015:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e630350b5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4043 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.506054:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b41c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605024768:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.506069:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.506094:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.507001:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b41c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605024768:192.168.203.25@tcp:106: 00002000:00010000:1.0:1780298644.507005:0:8356:0:(ofd_lvb.c:271:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime: 1780298644 -> 1780298645 00002000:00010000:1.0:1780298644.507010:0:8356:0:(ofd_lvb.c:277:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime: 1780298644 -> 1780298645 00000100:00100000:1.0:1780298644.507022:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255492a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605024640:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.507030:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e630350b5 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4043 expref: 14 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.507047:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e630350ae lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d403c expref: 18 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.507050:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e630350b5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4043 expref: 14 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.507077:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2600, rc 0) 00010000:00010000:1.0:1780298644.507079:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e630350ae lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d403c expref: 18 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.507093:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156400, rc 0) 00000100:00100000:2.0:1780298644.507117:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609598336:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1127us (1257us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.507147:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2399:x1866777609598208:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1171us (1305us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.507152:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9891 00000100:00100000:1.0:1780298644.507162:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9890 00000100:00100000:1.0:1780298644.508318:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.508355:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609598464 00000100:00100000:2.0:1780298644.508361:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609598464 00000100:00100000:2.0:1780298644.508402:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9892 00000100:00100000:2.0:1780298644.508411:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2398:x1866777609598464:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.508422:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.508432:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e630350bc lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d404a expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.508478:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b41c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605024896:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.508488:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.508533:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609598592 00000100:00100000:0.0:1780298644.508538:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609598592 00000100:00100000:2.0:1780298644.508541:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.508573:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9893 00000100:00100000:0.0:1780298644.508584:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255492a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609598592:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.508592:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.508602:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b35ce980/0xaf35132e630350c3 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4051 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.508633:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605025024:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.508666:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.510958:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b41c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605024896:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.510983:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e630350bc lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d404a expref: 18 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.511003:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e630350bc lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d404a expref: 18 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.511016:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2800, rc 0) 00000100:00100000:2.0:1780298644.511151:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2398:x1866777609598464:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 2741us (2834us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.511205:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9892 00000100:00100000:0.0:1780298644.511327:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605025024:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.511350:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b35ce980/0xaf35132e630350c3 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4051 expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.511366:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b35ce980/0xaf35132e630350c3 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4051 expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.511377:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bafa00, rc 0) 00000100:00100000:0.0:1780298644.511407:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255492a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609598592:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 2826us (2924us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.511424:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9893 00000100:00100000:1.0:1780298644.512443:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.512492:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609598720 00000100:00100000:0.0:1780298644.512499:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609598720 00000100:00100000:0.0:1780298644.512538:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9894 00000100:00100000:0.0:1780298644.512545:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2397:x1866777609598720:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.512556:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.512568:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e630350ca lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4058 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.512613:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605025152:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.512629:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.512669:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609598848 00000100:00100000:1.0:1780298644.512675:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609598848 00000100:00100000:0.0:1780298644.512684:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.512723:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9895 00000100:00100000:1.0:1780298644.512737:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7be300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609598848:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.512755:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.512765:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005b92bd2b/0xaf35132e630350d1 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d405f expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.512796:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605025280:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.512833:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837af00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.513413:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605025152:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.513427:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e630350ca lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4058 expref: 21 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.513438:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e630350ca lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4058 expref: 21 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.513445:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae600, rc 0) 00000100:00100000:0.0:1780298644.513476:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2397:x1866777609598720:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 933us (1037us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.513481:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9894 00000100:00100000:1.0:1780298644.513694:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605025280:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.513713:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005b92bd2b/0xaf35132e630350d1 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d405f expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.513729:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005b92bd2b/0xaf35132e630350d1 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d405f expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.513739:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279c00, rc 0) 00000100:00100000:1.0:1780298644.513771:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7be300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609598848:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1039us (1172us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.513782:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9895 00000100:00100000:0.0:1780298644.514692:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.514739:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609598976 00000100:00100000:2.0:1780298644.514751:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609598976 00000100:00100000:2.0:1780298644.514885:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9896 00000100:00100000:2.0:1780298644.514896:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2397:x1866777609598976:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.514913:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.514930:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e630350d8 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4066 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.514987:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b41c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605025408:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.514993:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.515044:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609599104 00000100:00100000:0.0:1780298644.515050:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609599104 00000100:00100000:0.0:1780298644.515093:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9897 00000100:00100000:0.0:1780298644.515101:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609599104:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.515111:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.515122:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e630350df lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d406d expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.515189:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605025536:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.515203:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.515238:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.515996:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b41c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605025408:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.516021:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e630350d8 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4066 expref: 21 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.516039:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e630350d8 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4066 expref: 21 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.516051:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3600, rc 0) 00000100:00100000:2.0:1780298644.516091:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2397:x1866777609598976:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1196us (1399us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.516099:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9896 00000100:00100000:0.0:1780298644.516191:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605025536:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.516211:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e630350df lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d406d expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.516227:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e630350df lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d406d expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.516238:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baea00, rc 0) 00000100:00100000:0.0:1780298644.516273:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609599104:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1173us (1284us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.516294:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9897 00000100:00100000:1.0:1780298644.517491:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.517537:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609599232 00000100:00100000:2.0:1780298644.517543:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609599232 00000100:00100000:2.0:1780298644.517583:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9898 00000100:00100000:2.0:1780298644.517592:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2397:x1866777609599232:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.517604:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.517612:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e630350e6 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4074 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.517626:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.517640:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b41c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605025664:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.517662:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609599360 00000100:00100000:1.0:1780298644.517667:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609599360 00000100:00100000:2.0:1780298644.517690:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.517701:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9899 00000100:00100000:1.0:1780298644.517709:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e4f1850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609599360:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.517715:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.517722:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017dab243/0xaf35132e630350ed lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d407b expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.517745:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605025792:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.517773:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837af00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.518301:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b41c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605025664:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.518320:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e630350e6 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4074 expref: 21 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.518335:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e630350e6 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4074 expref: 21 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.518343:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2600, rc 0) 00000100:00100000:2.0:1780298644.518374:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2397:x1866777609599232:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 785us (886us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.518379:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9898 00000100:00100000:3.0:1780298644.518483:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605025792:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.518498:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000017dab243/0xaf35132e630350ed lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d407b expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.518508:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017dab243/0xaf35132e630350ed lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d407b expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.518514:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279800, rc 0) 00000100:00100000:3.0:1780298644.518537:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e4f1850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609599360:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 831us (914us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.518542:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9899 00000100:00100000:1.0:1780298644.519339:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.519374:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609599488 00000100:00100000:0.0:1780298644.519379:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609599488 00002000:00100000:0.0:1780298644.519409:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (937984->942079) req@ffff9cf24304d880 x1866777609599488/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298644.519421:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24304d880 with x1866777609599488 ext(937984->942079) 00002000:00010000:0.0:1780298644.519425:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24304d880 00000100:00100000:0.0:1780298644.519446:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6322 00000100:00100000:0.0:1780298644.519454:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2397:x1866777609599488:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298644.519588:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2397:x1866777609599488:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 135us (250us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.519593:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6322 00002000:00100000:0.0:1780298644.519597:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (937984->942079) req@ffff9cf24304d880 x1866777609599488/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/440 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298644.519603:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24304d880 with x1866777609599488 ext(937984->942079) 00002000:00010000:0.0:1780298644.519605:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24304d880 00000100:00100000:1.0:1780298644.520597:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.520692:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609599616 00000100:00100000:3.0:1780298644.520697:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609599616 00000100:00100000:1.0:1780298644.520728:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.520748:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609599744 00000100:00100000:2.0:1780298644.520754:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609599744 00000100:00100000:3.0:1780298644.520773:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9900 00000100:00100000:3.0:1780298644.520781:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2397:x1866777609599616:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.520789:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298644.520793:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9901 00010000:00010000:3.0:1780298644.520797:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009c841b52/0xaf35132e630350f4 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4082 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.520804:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609599744:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.520812:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.520823:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e630350fb lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4089 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.520848:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7be300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605025920:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.520858:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605026048:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.520900:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.520901:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.521714:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7be300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605025920:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.521736:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009c841b52/0xaf35132e630350f4 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4082 expref: 18 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.521748:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009c841b52/0xaf35132e630350f4 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4082 expref: 18 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.521757:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157600, rc 0) 00000100:00100000:0.0:1780298644.521778:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605026048:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.521796:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e630350fb lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4089 expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.521808:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e630350fb lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4089 expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.521815:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baea00, rc 0) 00000100:00100000:3.0:1780298644.521821:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2397:x1866777609599616:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1004us (1188us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.521831:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9900 00000100:00100000:0.0:1780298644.521841:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609599744:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1040us (1114us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.521848:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9901 00000100:00100000:1.0:1780298644.522818:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.522876:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609599872 00000100:00100000:3.0:1780298644.522881:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609599872 00000100:00100000:1.0:1780298644.522905:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.522953:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609600000 00000100:00100000:2.0:1780298644.522959:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609600000 00000100:00100000:3.0:1780298644.522960:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9902 00000100:00100000:3.0:1780298644.522974:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2398:x1866777609599872:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.522984:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298644.522991:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9903 00010000:00010000:3.0:1780298644.522994:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e63035102 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4090 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.523003:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609600000:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.523014:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.523025:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e63035109 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4097 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.523036:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7be300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605026176:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.523061:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b41c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605026304:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.523092:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.523094:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.523994:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b41c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605026304:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.524001:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7be300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605026176:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.524026:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e63035109 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4097 expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.524027:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e63035102 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4090 expref: 18 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.524056:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e63035109 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4097 expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.524058:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e63035102 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4090 expref: 18 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.524078:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3600, rc 0) 00010000:00010000:3.0:1780298644.524080:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156a00, rc 0) 00000100:00100000:2.0:1780298644.524113:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609600000:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1113us (1207us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.524117:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2398:x1866777609599872:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1146us (1299us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.524156:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9903 00000100:00100000:3.0:1780298644.524165:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9902 00000100:00100000:1.0:1780298644.525202:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.525235:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609600128 00000100:00100000:2.0:1780298644.525241:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609600128 00000100:00100000:2.0:1780298644.525293:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9904 00000100:00100000:2.0:1780298644.525304:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:29360:x1866777609600128:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.525313:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.525323:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63035110 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d409e expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.525340:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(20 169) 00010000:00010000:2.0:1780298644.525345:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(20 169) 00010000:00010000:2.0:1780298644.525354:0:20252:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63035110 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [151552->720895] (req 151552->159743) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d409e expref: 18 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.525370:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63035110 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [151552->720895] (req 151552->159743) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d409e expref: 18 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.525385:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2600, rc 0) 00000100:00100000:2.0:1780298644.525414:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:29360:x1866777609600128:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 114us (214us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.525420:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9904 00000100:00100000:1.0:1780298644.526106:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.526182:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609600256 00000100:00100000:0.0:1780298644.526187:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609600256 00000100:00100000:0.0:1780298644.526228:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6323 00000100:00100000:0.0:1780298644.526235:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2399:x1866777609600256:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298644.526442:0:8363:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 360462 -> 360462 00002000:00010000:0.0:1780298644.526447:0:8363:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298644 -> 1780298644 00002000:00010000:0.0:1780298644.526449:0:8363:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298645 -> 1780298644 00002000:00010000:0.0:1780298644.526452:0:8363:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298645 -> 1780298644 00002000:00010000:0.0:1780298644.526454:0:8363:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 312 -> 328 00000100:00100000:0.0:1780298644.526491:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2399:x1866777609600256:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 256us (385us total) trans 4294969729 rc 0/0 00000100:00100000:0.0:1780298644.526498:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6323 00000100:00100000:1.0:1780298644.527516:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.527553:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609600384 00000100:00100000:0.0:1780298644.527559:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609600384 00000100:00100000:0.0:1780298644.527596:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9905 00000100:00100000:0.0:1780298644.527605:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2400:x1866777609600384:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.527614:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.527624:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000058e2d0/0xaf35132e63035117 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d40a5 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.527666:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605026432:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.527673:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.527721:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.527725:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609600512 00000100:00100000:2.0:1780298644.527730:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609600512 00000100:00100000:2.0:1780298644.527770:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9906 00000100:00100000:2.0:1780298644.527779:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609600512:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.527788:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.527797:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e6303511e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d40ac expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.527826:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b41c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605026560:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.527862:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00002000:00010000:0.0:1780298644.528433:0:19992:0:(ofd_lvb.c:271:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime: 1780298644 -> 1780298645 00002000:00010000:0.0:1780298644.528437:0:19992:0:(ofd_lvb.c:277:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime: 1780298644 -> 1780298645 00000100:00100000:0.0:1780298644.528448:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605026432:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.528469:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000058e2d0/0xaf35132e63035117 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d40a5 expref: 20 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.528493:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000058e2d0/0xaf35132e63035117 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d40a5 expref: 20 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.528504:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baea00, rc 0) 00000100:00100000:0.0:1780298644.528536:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2400:x1866777609600384:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 933us (1020us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.528543:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9905 00000100:00100000:2.0:1780298644.528725:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b41c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605026560:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.528747:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e6303511e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d40ac expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.528764:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e6303511e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d40ac expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.528777:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2800, rc 0) 00000100:00100000:2.0:1780298644.528811:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609600512:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1034us (1140us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.528819:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9906 00000100:00100000:1.0:1780298644.529892:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.529927:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.529930:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609600640 00000100:00100000:3.0:1780298644.529942:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609600640 00000100:00100000:2.0:1780298644.529977:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609600768 00000100:00100000:2.0:1780298644.529984:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609600768 00000100:00100000:3.0:1780298644.529996:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9907 00000100:00100000:3.0:1780298644.530010:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2400:x1866777609600640:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.530023:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.530041:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009c841b52/0xaf35132e63035125 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d40b3 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.530052:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9908 00000100:00100000:1.0:1780298644.530075:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609600768:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.530091:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298644.530100:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7be300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605026688:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.530105:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e6303512c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d40ba expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.530196:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605026816:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.530221:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.530246:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837af00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.531011:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7be300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605026688:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.531050:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009c841b52/0xaf35132e63035125 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d40b3 expref: 20 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.531070:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009c841b52/0xaf35132e63035125 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d40b3 expref: 20 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.531086:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157600, rc 0) 00000100:00100000:2.0:1780298644.531172:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605026816:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.531200:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e6303512c lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d40ba expref: 14 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.531210:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2400:x1866777609600640:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1201us (1314us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.531227:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9907 00010000:00010000:2.0:1780298644.531233:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e6303512c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d40ba expref: 14 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.531250:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279400, rc 0) 00000100:00100000:2.0:1780298644.531299:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609600768:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1233us (1372us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.531307:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9908 00000100:00100000:1.0:1780298644.532358:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.532401:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609600896 00000100:00100000:0.0:1780298644.532406:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609600896 00000100:00100000:0.0:1780298644.532441:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9909 00000100:00100000:0.0:1780298644.532451:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2397:x1866777609600896:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298644.532463:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:0.0:1780298644.532465:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.532476:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63035133 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d40c1 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.532486:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609601024 00000100:00100000:2.0:1780298644.532492:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609601024 00000100:00100000:0.0:1780298644.532505:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605026944:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.532530:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9910 00000100:00100000:2.0:1780298644.532541:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7be300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609601024:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298644.532551:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00010000:00010000:2.0:1780298644.532562:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.532572:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e6303513a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d40c8 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.532604:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b41c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605027072:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.532630:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.533288:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605026944:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.533313:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63035133 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d40c1 expref: 20 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.533330:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63035133 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d40c1 expref: 20 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.533345:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bafa00, rc 0) 00000100:00100000:0.0:1780298644.533375:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2397:x1866777609600896:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 927us (1016us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.533381:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9909 00000100:00100000:2.0:1780298644.533388:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b41c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605027072:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.533414:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e6303513a lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d40c8 expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.533430:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e6303513a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d40c8 expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.533441:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3600, rc 0) 00000100:00100000:2.0:1780298644.533470:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7be300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609601024:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 933us (1015us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.533477:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9910 00000100:00100000:3.0:1780298644.534323:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.534353:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609601152 00000100:00100000:0.0:1780298644.534357:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609601152 00000100:00100000:0.0:1780298644.534387:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9911 00000100:00100000:0.0:1780298644.534393:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2398:x1866777609601152:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.534401:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.534409:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000058e2d0/0xaf35132e63035141 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d40cf expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.534431:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605027200:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.534464:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.534476:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.534505:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609601280 00000100:00100000:3.0:1780298644.534542:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609601280 00000100:00100000:3.0:1780298644.534601:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9912 00000100:00100000:3.0:1780298644.534608:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609601280:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.534622:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.534636:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a58d48e9/0xaf35132e63035148 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d40d6 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.534680:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605027328:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.534730:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.535046:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605027200:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.535065:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000058e2d0/0xaf35132e63035141 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d40cf expref: 20 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.535081:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000058e2d0/0xaf35132e63035141 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d40cf expref: 20 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.535091:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baea00, rc 0) 00000100:00100000:0.0:1780298644.535147:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2398:x1866777609601152:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 731us (803us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.535157:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9911 00000100:00100000:3.0:1780298644.535379:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605027328:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.535403:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a58d48e9/0xaf35132e63035148 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d40d6 expref: 14 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.535420:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a58d48e9/0xaf35132e63035148 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d40d6 expref: 14 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.535432:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156a00, rc 0) 00000100:00100000:3.0:1780298644.535465:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609601280:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 857us (1001us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.535473:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9912 00000100:00100000:1.0:1780298644.536315:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.536349:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609601408 00000100:00100000:3.0:1780298644.536354:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609601408 00000100:00100000:3.0:1780298644.536384:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9913 00000100:00100000:3.0:1780298644.536390:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:29360:x1866777609601408:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.536400:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.536408:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000eae09c9b/0xaf35132e6303514f lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d40dd expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.536423:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e630350a7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [720896->34275327] (req 720896->729087) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4035 expref: 19 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.536430:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(20 169) 00010000:00010000:3.0:1780298644.536433:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(21 169) 00010000:00010000:3.0:1780298644.536436:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000eae09c9b/0xaf35132e6303514f lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2600960->2609151] (req 2600960->2609151) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d40dd expref: 20 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.536451:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e630350a7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [720896->34275327] (req 720896->729087) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4035 expref: 19 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.536461:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e630350a7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [720896->34275327] (req 720896->729087) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4035 expref: 19 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.536467:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605027456:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298644.536508:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.537187:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bdf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605027456:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298644.537206:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000eae09c9b/0xaf35132e6303514f lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2600960->2609151] (req 2600960->2609151) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d40dd expref: 20 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.537218:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157a00, rc 0) 00000100:00100000:3.0:1780298644.537251:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:29360:x1866777609601408:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 860us (938us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298644.537259:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9913 00000100:00100000:1.0:1780298644.537371:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.537395:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609601536 00000100:00100000:2.0:1780298644.537401:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609601536 00010000:00010000:2.0:1780298644.537420:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e630350a7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [720896->34275327] (req 720896->729087) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4035 expref: 20 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.537444:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15204 00000100:00100000:2.0:1780298644.537449:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2400:x1866777609601536:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298644.537457:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298644.537466:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e630350a7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [720896->34275327] (req 720896->729087) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4035 expref: 20 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.537475:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e630350a7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [720896->34275327] (req 720896->729087) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4035 expref: 20 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.537488:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e630350a7 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [720896->34275327] (req 720896->729087) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4035 expref: 20 pid: 19992 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.537497:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(21 169) 00010000:00010000:2.0:1780298644.537500:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(21 169) 00010000:00010000:2.0:1780298644.537506:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000eae09c9b/0xaf35132e6303514f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2600960->36155391] (req 2600960->2609151) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d40dd expref: 19 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.537513:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000eae09c9b/0xaf35132e6303514f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2600960->36155391] (req 2600960->2609151) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d40dd expref: 19 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.537531:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000eae09c9b/0xaf35132e6303514f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2600960->36155391] (req 2600960->2609151) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d40dd expref: 19 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.537538:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b41c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777605027584:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298644.537593:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.538290:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b41c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777605027584:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298644.538307:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298644.538346:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+19:2400:x1866777609601536:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 896us (976us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298644.538355:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15204 00000100:00100000:3.0:1780298644.538625:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.538672:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609601664 00000100:00100000:2.0:1780298644.538678:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609601664 00002000:00100000:2.0:1780298644.538714:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x2c0000401:0x11:0x0] (2600960->2605055) req@ffff9cf37f7bdf80 x1866777609601664/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.538726:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f7bdf80 with x1866777609601664 ext(2600960->2605055) 00002000:00010000:2.0:1780298644.538731:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f7bdf80 00000100:00100000:2.0:1780298644.538755:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6324 00000100:00100000:2.0:1780298644.538766:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2399:x1866777609601664:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298644.538928:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2399:x1866777609601664:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 164us (302us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.538940:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6324 00002000:00100000:2.0:1780298644.538948:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x2c0000401:0x11:0x0] (2600960->2605055) req@ffff9cf37f7bdf80 x1866777609601664/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/440 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.538957:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f7bdf80 with x1866777609601664 ext(2600960->2605055) 00002000:00010000:2.0:1780298644.538961:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f7bdf80 00000100:00100000:3.0:1780298644.539707:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.539740:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609601792 00000100:00100000:2.0:1780298644.539744:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609601792 00002000:00100000:2.0:1780298644.539760:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (2605056->2609151) req@ffff9cf37f7bf800 x1866777609601792/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.539767:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f7bf800 with x1866777609601792 ext(2605056->2609151) 00002000:00010000:2.0:1780298644.539769:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f7bf800 00000100:00100000:2.0:1780298644.539786:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6325 00000100:00100000:2.0:1780298644.539790:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2397:x1866777609601792:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298644.539860:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2397:x1866777609601792:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 68us (154us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.539868:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6325 00002000:00100000:2.0:1780298644.539875:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (2605056->2609151) req@ffff9cf37f7bf800 x1866777609601792/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/440 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.539884:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f7bf800 with x1866777609601792 ext(2605056->2609151) 00002000:00010000:2.0:1780298644.539887:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f7bf800 00000100:00100000:3.0:1780298644.540942:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.541005:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609601920 00000100:00100000:2.0:1780298644.541011:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609601920 00000100:00100000:3.0:1780298644.541026:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.541060:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609602048 00000100:00100000:2.0:1780298644.541063:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609602048 00000100:00100000:0.0:1780298644.541076:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9914 00000100:00100000:0.0:1780298644.541088:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2397:x1866777609601920:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.541099:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.541113:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63035156 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d40e4 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.541118:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9915 00000100:00100000:1.0:1780298644.541166:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609602048:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.541176:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.541190:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e6303515d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d40eb expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.541201:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605027712:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.541239:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e56f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605027840:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.541260:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.541289:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837ae40 going to sleep for 11 seconds 00002000:00010000:0.0:1780298644.541946:0:9128:0:(ofd_lvb.c:259:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size: 360462 -> 2606243 00002000:00010000:0.0:1780298644.541950:0:9128:0:(ofd_lvb.c:265:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime: 1780298644 -> 1780298645 00000100:00100000:0.0:1780298644.541966:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605027712:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.541998:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63035156 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d40e4 expref: 21 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1780298644.542006:0:19992:0:(ofd_lvb.c:265:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime: 1780298644 -> 1780298645 00010000:00010000:0.0:1780298644.542020:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63035156 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d40e4 expref: 21 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.542027:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e56f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605027840:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.542046:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf200, rc 0) 00010000:00010000:2.0:1780298644.542066:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e6303515d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d40eb expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.542091:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e6303515d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d40eb expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.542092:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2397:x1866777609601920:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1000us (1144us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.542119:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9914 00010000:00010000:2.0:1780298644.542171:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278400, rc 0) 00000100:00100000:2.0:1780298644.542213:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609602048:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1052us (1186us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.542222:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9915 00000100:00100000:1.0:1780298644.543217:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.543250:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609602176 00000100:00100000:3.0:1780298644.543257:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609602176 00000100:00100000:1.0:1780298644.543264:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.543312:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609602304 00000100:00100000:0.0:1780298644.543317:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609602304 00000100:00100000:2.0:1780298644.543367:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9916 00000100:00100000:0.0:1780298644.543381:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9917 00000100:00100000:2.0:1780298644.543385:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2399:x1866777609602176:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.543396:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298644.543402:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609602304:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.543408:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e63035164 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d40f2 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.543415:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.543428:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e6303516b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d40f9 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.543437:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b41880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605027968:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.543478:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605028096:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.543493:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1480 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.543522:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.544193:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b41880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605027968:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.544206:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605028096:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.544216:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e63035164 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d40f2 expref: 21 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.544234:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e63035164 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d40f2 expref: 21 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.544243:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e6303516b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d40f9 expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.544247:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3600, rc 0) 00010000:00010000:0.0:1780298644.544269:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e6303516b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d40f9 expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.544293:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2399:x1866777609602176:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 915us (1077us total) trans 0 rc 301/301 00010000:00010000:0.0:1780298644.544298:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf600, rc 0) 00000100:00100000:2.0:1780298644.544303:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9916 00000100:00100000:0.0:1780298644.544342:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609602304:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 948us (1078us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.544350:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9917 00000100:00100000:3.0:1780298644.545345:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.545384:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609602432 00000100:00100000:0.0:1780298644.545389:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609602432 00000100:00100000:0.0:1780298644.545412:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9918 00000100:00100000:0.0:1780298644.545418:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+19:29360:x1866777609602432:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.545426:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.545437:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e63035172 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4100 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.545454:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000eae09c9b/0xaf35132e6303514f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2600960->36155391] (req 2600960->2609151) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d40dd expref: 19 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.545461:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(21 169) 00010000:00010000:0.0:1780298644.545464:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(22 169) 00010000:00010000:0.0:1780298644.545467:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e63035172 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3641344->3645439] (req 3641344->3645439) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4100 expref: 20 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.545486:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000eae09c9b/0xaf35132e6303514f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2600960->36155391] (req 2600960->2609151) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d40dd expref: 19 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.545498:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000eae09c9b/0xaf35132e6303514f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2600960->36155391] (req 2600960->2609151) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d40dd expref: 19 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.545509:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605028224:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298644.545549:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.546172:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605028224:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298644.546191:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e63035172 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3641344->3645439] (req 3641344->3645439) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4100 expref: 20 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.546205:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf000, rc 0) 00000100:00100000:0.0:1780298644.546234:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:29360:x1866777609602432:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 815us (890us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.546241:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9918 00000100:00100000:3.0:1780298644.546496:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.546540:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609602560 00000100:00100000:2.0:1780298644.546547:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609602560 00002000:00100000:2.0:1780298644.546582:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (2600960->2606242) req@ffff9cf37f7be680 x1866777609602560/t0(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 5776/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.546594:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f7be680 with x1866777609602560 ext(2600960->2606242) 00010000:00010000:2.0:1780298644.546602:0:8364:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000eae09c9b/0xaf35132e6303514f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2600960->36155391] (req 2600960->2609151) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d40dd expref: 20 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.546616:0:8364:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000eae09c9b/0xaf35132e6303514f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2600960->36155391] (req 2600960->2609151) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d40dd expref: 20 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1780298644.546624:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf37f7be680 00000100:00100000:2.0:1780298644.546649:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 207 00000100:00100000:2.0:1780298644.546660:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7be680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2399:x1866777609602560:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:2.0:1780298644.547255:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7be680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:2399:x1866777609602560:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 597us (757us total) trans 4294969730 rc 0/0 00000100:00100000:2.0:1780298644.547264:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 207 00002000:00100000:2.0:1780298644.547272:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (2600960->2606242) req@ffff9cf37f7be680 x1866777609602560/t4294969730(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 5776/448 e 0 to 0 dl 1780298655 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.547301:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f7be680 with x1866777609602560 ext(2600960->2606242) 00010000:00010000:2.0:1780298644.547309:0:8364:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000eae09c9b/0xaf35132e6303514f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2600960->36155391] (req 2600960->2609151) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d40dd expref: 22 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.547325:0:8364:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000eae09c9b/0xaf35132e6303514f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2600960->36155391] (req 2600960->2609151) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d40dd expref: 22 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1780298644.547331:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf37f7be680 00000100:00100000:3.0:1780298644.548057:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.548099:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609602688 00000100:00100000:1.0:1780298644.548107:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609602688 00010000:00010000:1.0:1780298644.548165:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000eae09c9b/0xaf35132e6303514f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2600960->36155391] (req 2600960->2609151) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d40dd expref: 22 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.548199:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15205 00000100:00100000:1.0:1780298644.548218:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bd500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:2397:x1866777609602688:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298644.548232:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:1.0:1780298644.548245:0:6461:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 328 -> 344 00010000:00010000:1.0:1780298644.548250:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000eae09c9b/0xaf35132e6303514f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2600960->36155391] (req 2600960->2609151) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d40dd expref: 22 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.548259:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000eae09c9b/0xaf35132e6303514f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2600960->36155391] (req 2600960->2609151) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d40dd expref: 22 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.548274:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000eae09c9b/0xaf35132e6303514f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2600960->36155391] (req 2600960->2609151) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d40dd expref: 22 pid: 8358 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.548293:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(22 169) 00010000:00010000:1.0:1780298644.548296:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(22 169) 00010000:00010000:1.0:1780298644.548303:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e63035172 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3641344->37195775] (req 3641344->3645439) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4100 expref: 19 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.548309:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e63035172 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3641344->37195775] (req 3641344->3645439) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4100 expref: 19 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.548330:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e63035172 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3641344->37195775] (req 3641344->3645439) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4100 expref: 19 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.548337:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e56d880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777605028352:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298644.548376:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.549054:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e56d880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777605028352:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298644.549076:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298644.549109:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bd500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2397:x1866777609602688:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 900us (1053us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.549117:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15205 00000100:00100000:3.0:1780298644.549560:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.549597:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609602816 00000100:00100000:0.0:1780298644.549602:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609602816 00000100:00100000:0.0:1780298644.549643:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6326 00000100:00100000:0.0:1780298644.549653:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255492300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2400:x1866777609602816:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298644.549761:0:8362:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 2606243 -> 2606243 00002000:00010000:0.0:1780298644.549764:0:8362:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298644.549767:0:8362:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298644.549769:0:8362:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298644.549771:0:8362:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 344 -> 344 00000100:00100000:0.0:1780298644.549800:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255492300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2400:x1866777609602816:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 150us (241us total) trans 4294969731 rc 0/0 00000100:00100000:0.0:1780298644.549807:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6326 00000100:00100000:3.0:1780298644.550736:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.550784:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609602944 00000100:00100000:2.0:1780298644.550792:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609602944 00000100:00100000:3.0:1780298644.550851:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.550851:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9919 00000100:00100000:2.0:1780298644.550869:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255490380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2397:x1866777609602944:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.550880:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.550897:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e63035179 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4107 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.550910:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609603072 00000100:00100000:1.0:1780298644.550917:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609603072 00000100:00100000:2.0:1780298644.550941:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b41880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605028480:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.550961:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9920 00000100:00100000:1.0:1780298644.550970:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255490e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609603072:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.550979:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.550988:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63035180 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d410e expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.551015:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e56dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605028608:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.551016:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1480 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.551044:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a540 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.551762:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b41880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605028480:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.551785:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e63035179 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4107 expref: 22 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.551802:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e63035179 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4107 expref: 22 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.551815:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3600, rc 0) 00000100:00100000:2.0:1780298644.551876:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255490380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2397:x1866777609602944:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1011us (1140us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.551878:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e56dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605028608:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.551891:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9919 00010000:00010000:3.0:1780298644.551930:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63035180 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d410e expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.551951:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63035180 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d410e expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.551969:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279200, rc 0) 00000100:00100000:3.0:1780298644.552045:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255490e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609603072:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1077us (1210us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.552053:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9920 00000100:00100000:1.0:1780298644.552912:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.552952:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.552975:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609603200 00000100:00100000:0.0:1780298644.552982:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609603200 00000100:00100000:3.0:1780298644.553011:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609603328 00000100:00100000:3.0:1780298644.553016:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609603328 00000100:00100000:0.0:1780298644.553026:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9921 00000100:00100000:0.0:1780298644.553039:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2399:x1866777609603200:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.553051:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298644.553062:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9922 00010000:00010000:0.0:1780298644.553066:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63035187 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4115 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.553077:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609603328:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.553087:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.553097:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a58d48e9/0xaf35132e6303518e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d411c expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.553115:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605028736:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.553179:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255492a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605028864:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.553232:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.553237:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.553957:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255492a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605028864:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.553978:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605028736:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.553984:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a58d48e9/0xaf35132e6303518e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d411c expref: 14 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.554003:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a58d48e9/0xaf35132e6303518e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d411c expref: 14 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.554008:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63035187 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4115 expref: 22 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.554023:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156a00, rc 0) 00010000:00010000:1.0:1780298644.554045:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e63035187 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4115 expref: 22 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.554060:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bafa00, rc 0) 00000100:00100000:0.0:1780298644.554067:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609603328:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 993us (1112us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.554076:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9922 00000100:00100000:1.0:1780298644.554091:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2399:x1866777609603200:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1055us (1180us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.554098:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9921 00000100:00100000:1.0:1780298644.555262:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.555299:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609603456 00000100:00100000:3.0:1780298644.555304:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609603456 00000100:00100000:3.0:1780298644.555331:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9923 00000100:00100000:3.0:1780298644.555338:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:29360:x1866777609603456:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.555346:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.555355:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e63035195 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4123 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.555371:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e63035172 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3641344->37195775] (req 3641344->3645439) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4100 expref: 20 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.555381:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(22 169) 00010000:00010000:3.0:1780298644.555384:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(23 169) 00010000:00010000:3.0:1780298644.555387:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e63035195 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3895296->3899391] (req 3895296->3899391) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4123 expref: 22 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.555405:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e63035172 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3641344->37195775] (req 3641344->3645439) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4100 expref: 20 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.555421:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e63035172 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3641344->37195775] (req 3641344->3645439) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4100 expref: 20 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.555430:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255492a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605028992:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298644.555475:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.556272:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255492a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605028992:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298644.556310:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e63035195 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3895296->3899391] (req 3895296->3899391) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4123 expref: 22 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.556324:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157200, rc 0) 00000100:00100000:1.0:1780298644.556347:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.556359:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:29360:x1866777609603456:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1020us (1095us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298644.556367:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9923 00000100:00100000:0.0:1780298644.556392:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609603584 00000100:00100000:0.0:1780298644.556399:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609603584 00010000:00010000:0.0:1780298644.556431:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e63035172 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3641344->37195775] (req 3641344->3645439) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4100 expref: 21 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.556454:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15206 00000100:00100000:0.0:1780298644.556462:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243061500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2400:x1866777609603584:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298644.556473:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298644.556484:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e63035172 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3641344->37195775] (req 3641344->3645439) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4100 expref: 21 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.556493:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e63035172 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3641344->37195775] (req 3641344->3645439) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4100 expref: 21 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.556517:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e63035172 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3641344->37195775] (req 3641344->3645439) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4100 expref: 21 pid: 8356 timeout: 933 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.556530:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(23 169) 00010000:00010000:0.0:1780298644.556534:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(23 169) 00010000:00010000:0.0:1780298644.556544:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e63035195 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3895296->37449727] (req 3895296->3899391) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4123 expref: 21 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.556553:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e63035195 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3895296->37449727] (req 3895296->3899391) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4123 expref: 21 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.556577:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e63035195 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3895296->37449727] (req 3895296->3899391) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4123 expref: 21 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.556586:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd4380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777605029120:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298644.556634:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733936c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.557325:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd4380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777605029120:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298644.557336:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298644.557359:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243061500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2400:x1866777609603584:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 899us (1014us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.557365:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15206 00000100:00100000:2.0:1780298644.557494:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.557533:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609603712 00000100:00100000:0.0:1780298644.557538:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609603712 00000100:00100000:0.0:1780298644.557569:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6327 00000100:00100000:0.0:1780298644.557576:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b41880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:2400:x1866777609603712:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298644.557664:0:8363:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 2606243 -> 2606243 00002000:00010000:0.0:1780298644.557666:0:8363:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298644.557668:0:8363:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298644.557670:0:8363:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298644.557671:0:8363:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 344 -> 344 00000100:00100000:0.0:1780298644.557693:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b41880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2400:x1866777609603712:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 118us (202us total) trans 4294969732 rc 0/0 00000100:00100000:0.0:1780298644.557702:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6327 00000100:00100000:1.0:1780298644.558745:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.558779:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.558780:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609603840 00000100:00100000:3.0:1780298644.558785:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609603840 00000100:00100000:2.0:1780298644.558835:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609603968 00000100:00100000:2.0:1780298644.558842:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609603968 00000100:00100000:0.0:1780298644.558855:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9924 00000100:00100000:3.0:1780298644.558872:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9925 00000100:00100000:0.0:1780298644.558872:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243063100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2400:x1866777609603840:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298644.558886:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243063800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609603968:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.558886:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.558895:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.558902:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e6303519c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d412a expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.558907:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e630351a3 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4131 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.558947:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605029248:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.558950:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255492a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605029376:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.559000:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.559006:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733936c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.559717:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255492a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605029376:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.559738:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e630351a3 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4131 expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.559747:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605029248:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.559752:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e630351a3 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4131 expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.559761:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157c00, rc 0) 00010000:00010000:0.0:1780298644.559767:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e6303519c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d412a expref: 24 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.559778:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e6303519c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d412a expref: 24 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.559790:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243063800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609603968:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 907us (1010us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.559795:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9925 00010000:00010000:0.0:1780298644.559795:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf600, rc 0) 00000100:00100000:0.0:1780298644.559822:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243063100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2400:x1866777609603840:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 957us (1080us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.559829:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9924 00000100:00100000:2.0:1780298644.560886:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.560929:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609604096 00000100:00100000:1.0:1780298644.560937:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609604096 00000100:00100000:2.0:1780298644.560944:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.560976:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609604224 00000100:00100000:1.0:1780298644.560994:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9926 00000100:00100000:1.0:1780298644.561007:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2397:x1866777609604096:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.561014:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.561026:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e630351aa lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4138 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.561031:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609604224 00000100:00100000:1.0:1780298644.561054:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243060a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605029504:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.561068:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9927 00000100:00100000:3.0:1780298644.561076:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609604224:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.561083:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.561090:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c841b52/0xaf35132e630351b1 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d413f expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.561111:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a540 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.561115:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255492a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605029632:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.561171:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.561901:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243060a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605029504:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.561931:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e630351aa lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4138 expref: 24 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.561942:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e630351aa lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4138 expref: 24 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.561951:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279a00, rc 0) 00000100:00100000:2.0:1780298644.561963:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255492a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605029632:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.561983:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2397:x1866777609604096:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 981us (1102us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.561989:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9926 00010000:00010000:2.0:1780298644.562003:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009c841b52/0xaf35132e630351b1 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d413f expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.562027:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c841b52/0xaf35132e630351b1 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d413f expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.562039:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157600, rc 0) 00000100:00100000:2.0:1780298644.562090:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609604224:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1013us (1147us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.562099:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9927 00000100:00100000:1.0:1780298644.563083:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.563102:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.563155:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609604480 00000100:00100000:0.0:1780298644.563157:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609604352 00000100:00100000:3.0:1780298644.563163:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609604480 00000100:00100000:0.0:1780298644.563164:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609604352 00000100:00100000:0.0:1780298644.563213:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9928 00000100:00100000:1.0:1780298644.563225:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9929 00000100:00100000:0.0:1780298644.563228:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243061f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2400:x1866777609604352:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298644.563239:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243060000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609604480:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.563239:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.563250:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e630351b8 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4146 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.563251:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.563262:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e630351bf lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d414d expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.563289:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605029760:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.563309:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243063480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605029888:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.563334:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733936c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.563344:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.564025:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243063480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605029888:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.564042:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605029760:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.564044:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e630351bf lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d414d expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.564063:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e630351bf lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d414d expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.564066:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e630351b8 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4146 expref: 21 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.564071:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279400, rc 0) 00010000:00010000:2.0:1780298644.564092:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e630351b8 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4146 expref: 21 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.564100:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243060000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609604480:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 867us (995us total) trans 0 rc 301/301 00010000:00010000:2.0:1780298644.564104:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae400, rc 0) 00000100:00100000:1.0:1780298644.564108:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9929 00000100:00100000:2.0:1780298644.564160:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243061f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2400:x1866777609604352:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 939us (1078us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.564165:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9928 00000100:00100000:2.0:1780298644.565185:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.565213:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609604608 00000100:00100000:0.0:1780298644.565217:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609604608 00000100:00100000:0.0:1780298644.565246:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9930 00000100:00100000:0.0:1780298644.565253:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2397:x1866777609604608:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.565260:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.565267:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630351c6 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4154 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.565303:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605030016:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.565311:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.565342:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609604736 00000100:00100000:2.0:1780298644.565347:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609604736 00000100:00100000:0.0:1780298644.565348:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733936c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.565384:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9931 00000100:00100000:2.0:1780298644.565394:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255492a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609604736:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.565401:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.565409:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e630351cd lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d415b expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.565437:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605030144:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.565467:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1480 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.566008:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605030016:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.566029:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630351c6 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4154 expref: 21 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.566044:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630351c6 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4154 expref: 21 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.566054:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf000, rc 0) 00000100:00100000:0.0:1780298644.566086:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2397:x1866777609604608:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 833us (902us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.566091:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605030144:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.566097:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9930 00010000:00010000:2.0:1780298644.566111:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e630351cd lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d415b expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.566121:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e630351cd lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d415b expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.566153:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3000, rc 0) 00000100:00100000:2.0:1780298644.566175:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255492a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609604736:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 785us (867us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.566181:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9931 00000100:00100000:3.0:1780298644.568425:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.568451:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.568470:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609604864 00000100:00100000:2.0:1780298644.568476:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609604864 00000100:00100000:1.0:1780298644.568484:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609604992 00000100:00100000:1.0:1780298644.568490:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609604992 00000100:00100000:2.0:1780298644.568510:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9932 00000100:00100000:2.0:1780298644.568520:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243808700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2400:x1866777609604864:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.568531:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298644.568538:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9933 00010000:00010000:2.0:1780298644.568540:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e630351d4 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4162 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.568551:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243809c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609604992:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.568564:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298644.568578:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605030272:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.568580:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e630351db lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4169 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.568614:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605030400:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.568627:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1480 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.568647:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733936c0 going to sleep for 11 seconds 00002000:00010000:2.0:1780298644.569376:0:8358:0:(ofd_lvb.c:259:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size: 2606243 -> 5683587 00000100:00100000:2.0:1780298644.569387:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605030272:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.569403:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e630351d4 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4162 expref: 21 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.569403:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605030400:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.569420:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e630351d4 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4162 expref: 21 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.569428:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3c00, rc 0) 00010000:00010000:0.0:1780298644.569431:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e630351db lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4169 expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.569447:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e630351db lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4169 expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.569456:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243808700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2400:x1866777609604864:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 936us (1031us total) trans 0 rc 301/301 00010000:00010000:0.0:1780298644.569456:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf600, rc 0) 00000100:00100000:2.0:1780298644.569466:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9932 00000100:00100000:0.0:1780298644.569483:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243809c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609604992:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 937us (1031us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.569491:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9933 00000100:00100000:1.0:1780298644.570493:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.570525:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609605120 00000100:00100000:2.0:1780298644.570530:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609605120 00000100:00100000:2.0:1780298644.570560:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9934 00000100:00100000:2.0:1780298644.570569:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243063480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2397:x1866777609605120:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.570585:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.570594:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e630351e2 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4170 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.570624:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605030528:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.570641:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.570659:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1480 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.570675:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609605248 00000100:00100000:0.0:1780298644.570680:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609605248 00000100:00100000:0.0:1780298644.570708:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9935 00000100:00100000:0.0:1780298644.570715:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243062300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609605248:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.570723:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.570730:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e630351e9 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4177 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.570760:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605030656:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.570794:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733936c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.571339:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605030528:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.571351:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e630351e2 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4170 expref: 21 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.571361:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e630351e2 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4170 expref: 21 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.571368:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3600, rc 0) 00000100:00100000:1.0:1780298644.571392:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243063480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2397:x1866777609605120:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 826us (900us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.571397:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9934 00000100:00100000:2.0:1780298644.571521:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605030656:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.571538:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e630351e9 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4177 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.571551:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e630351e9 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4177 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.571561:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baec00, rc 0) 00000100:00100000:2.0:1780298644.571586:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243062300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609605248:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 871us (945us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.571594:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9935 00000100:00100000:1.0:1780298644.572619:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.572638:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.572652:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609605376 00000100:00100000:0.0:1780298644.572657:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609605376 00000100:00100000:2.0:1780298644.572674:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609605504 00000100:00100000:2.0:1780298644.572679:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609605504 00000100:00100000:0.0:1780298644.572693:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9936 00000100:00100000:0.0:1780298644.572703:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243062d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2398:x1866777609605376:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.572713:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298644.572718:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9937 00010000:00010000:0.0:1780298644.572721:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630351f0 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d417e expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.572730:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243063b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609605504:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.572741:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.572749:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e630351f7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4185 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.572760:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605030784:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.572781:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605030912:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.572807:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.572818:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb16c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.573616:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605030912:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.573636:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e630351f7 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4185 expref: 14 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.573639:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605030784:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.573659:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e630351f7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4185 expref: 14 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.573666:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630351f0 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d417e expref: 24 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.573672:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2e00, rc 0) 00010000:00010000:0.0:1780298644.573687:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630351f0 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d417e expref: 24 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.573699:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf000, rc 0) 00000100:00100000:0.0:1780298644.573728:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243062d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2398:x1866777609605376:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1028us (1109us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.573736:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243063b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609605504:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1007us (1093us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.573740:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9936 00000100:00100000:2.0:1780298644.573750:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9937 00000100:00100000:1.0:1780298644.574857:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.574890:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.574893:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609605632 00000100:00100000:3.0:1780298644.574898:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609605632 00000100:00100000:3.0:1780298644.574934:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609605760 00000100:00100000:3.0:1780298644.574937:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609605760 00000100:00100000:3.0:1780298644.574962:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9938 00000100:00100000:0.0:1780298644.574969:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9939 00000100:00100000:3.0:1780298644.574973:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243060a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2397:x1866777609605632:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298644.574982:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244810a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609605760:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.574983:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.574991:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.574996:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e630351fe lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d418c expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.574999:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c37b15a2/0xaf35132e63035205 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4193 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.575043:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605031040:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.575077:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243808380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605031168:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.575093:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.575111:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.575904:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243808380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605031168:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.575908:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605031040:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.575930:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c37b15a2/0xaf35132e63035205 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4193 expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.575941:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e630351fe lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d418c expref: 24 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.575950:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c37b15a2/0xaf35132e63035205 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4193 expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.575959:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e630351fe lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d418c expref: 24 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.575965:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baee00, rc 0) 00010000:00010000:3.0:1780298644.575973:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157c00, rc 0) 00000100:00100000:3.0:1780298644.576002:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243060a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2397:x1866777609605632:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1031us (1145us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.576003:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244810a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609605760:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1023us (1112us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.576016:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9938 00000100:00100000:0.0:1780298644.576026:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9939 00000100:00100000:1.0:1780298644.577072:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.577105:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.577108:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609605888 00000100:00100000:0.0:1780298644.577114:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609605888 00000100:00100000:2.0:1780298644.577179:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609606016 00000100:00100000:2.0:1780298644.577186:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609606016 00000100:00100000:0.0:1780298644.577189:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9940 00000100:00100000:0.0:1780298644.577201:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244813100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2398:x1866777609605888:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.577211:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298644.577219:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9941 00010000:00010000:0.0:1780298644.577220:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e6303520c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d419a expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.577231:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244810000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609606016:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.577241:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.577250:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63035213 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d41a1 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.577259:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605031296:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.577295:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b41c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605031424:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.577314:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.577323:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb10c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.578175:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b41c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605031424:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.578176:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605031296:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.578197:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63035213 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41a1 expref: 14 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.578202:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e6303520c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d419a expref: 24 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.578234:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e6303520c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d419a expref: 24 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.578236:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63035213 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41a1 expref: 14 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.578256:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3c00, rc 0) 00010000:00010000:0.0:1780298644.578259:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf600, rc 0) 00000100:00100000:2.0:1780298644.578291:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244810000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609606016:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1065us (1186us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.578296:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9941 00000100:00100000:0.0:1780298644.578303:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244813100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2398:x1866777609605888:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1104us (1229us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.578311:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9940 00000100:00100000:1.0:1780298644.579229:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.579253:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609606144 00000100:00100000:3.0:1780298644.579257:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609606144 00000100:00100000:1.0:1780298644.579271:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.579284:0:3625:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1780298644.579299:0:3625:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3625:1866777605031552:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1780298644.579302:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609606272 00000100:00100000:3.0:1780298644.579306:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609606272 00000100:00100000:0.0:1780298644.579311:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9942 00000100:00100000:0.0:1780298644.579320:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244811880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2397:x1866777609606144:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.579327:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.579335:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005d10d3aa/0xaf35132e6303521a lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d41a8 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.579343:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9943 00000100:00100000:2.0:1780298644.579345:0:3625:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1780298644.579360:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244813b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609606272:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298644.579369:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605031680:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.579370:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298644.579380:0:6482:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777605031552 00010000:00010000:3.0:1780298644.579382:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e0b8e7b/0xaf35132e63035221 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d41af expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.579384:0:6482:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777605031552 00000100:00100000:0.0:1780298644.579412:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.579418:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243808380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605031808:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.579427:0:6482:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 648 00000100:00100000:2.0:1780298644.579438:0:6482:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b42680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3625:x1866777605031552:12345-0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1780298644.579447:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000004:00000024:2.0:1780298644.579468:0:6482:0:(mdt_handler.c:543:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1446252544 avail 1300013056 00000100:00100000:2.0:1780298644.579509:0:6482:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b42680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3625:x1866777605031552:12345-0@lo:41:osp-pre-1-0.0 Request processed in 75us (169us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.579514:0:6482:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 648 00000100:00100000:0.0:1780298644.579538:0:3625:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b41c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3625:1866777605031552:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1780298644.580018:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605031680:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.580036:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005d10d3aa/0xaf35132e6303521a lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41a8 expref: 24 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.580051:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005d10d3aa/0xaf35132e6303521a lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41a8 expref: 24 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.580062:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baec00, rc 0) 00000100:00100000:0.0:1780298644.580099:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244811880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2397:x1866777609606144:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 781us (872us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.580107:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9942 00000100:00100000:3.0:1780298644.580210:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243808380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605031808:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.580243:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009e0b8e7b/0xaf35132e63035221 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41af expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.580256:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e0b8e7b/0xaf35132e63035221 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41af expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.580307:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156c00, rc 0) 00000100:00100000:2.0:1780298644.580356:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244813b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609606272:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1000us (1084us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.580361:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9943 00000100:00100000:1.0:1780298644.581251:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.581288:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609606400 00000100:00100000:1.0:1780298644.581294:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.581294:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609606400 00000100:00100000:0.0:1780298644.581324:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609606528 00000100:00100000:0.0:1780298644.581326:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609606528 00000100:00100000:2.0:1780298644.581335:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9944 00000100:00100000:3.0:1780298644.581347:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9945 00000100:00100000:2.0:1780298644.581349:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244813480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2398:x1866777609606400:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298644.581358:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244811c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609606528:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.581358:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.581366:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.581370:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63035228 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d41b6 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.581375:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e6303522f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d41bd expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.581399:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605031936:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.581404:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243808380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605032064:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.581441:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.581443:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb10c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.582157:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605031936:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.582186:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63035228 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41b6 expref: 24 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.582201:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63035228 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41b6 expref: 24 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.582208:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3000, rc 0) 00000100:00100000:2.0:1780298644.582232:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244813480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2398:x1866777609606400:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 890us (982us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.582237:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9944 00000100:00100000:3.0:1780298644.582265:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243808380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605032064:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.582295:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e6303522f lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41bd expref: 14 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.582304:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e6303522f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41bd expref: 14 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.582342:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157c00, rc 0) 00000100:00100000:2.0:1780298644.582388:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244811c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609606528:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1034us (1097us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.582392:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9945 00000100:00100000:1.0:1780298644.583385:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.583403:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.583417:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609606656 00000100:00100000:3.0:1780298644.583423:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609606656 00000100:00100000:0.0:1780298644.583431:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609606784 00000100:00100000:0.0:1780298644.583435:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609606784 00000100:00100000:3.0:1780298644.583462:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9946 00000100:00100000:3.0:1780298644.583474:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e4f3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2397:x1866777609606656:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298644.583479:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9947 00010000:00010000:3.0:1780298644.583485:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298644.583491:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244811f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609606784:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.583495:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004ba675d2/0xaf35132e63035236 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d41c4 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.583500:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.583511:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c37b15a2/0xaf35132e6303523d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d41cb expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.583529:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243808380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605032192:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.583547:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605032320:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.583577:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.583591:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.584321:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243808380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605032192:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.584343:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004ba675d2/0xaf35132e63035236 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41c4 expref: 24 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.584355:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004ba675d2/0xaf35132e63035236 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41c4 expref: 24 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.584362:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156e00, rc 0) 00000100:00100000:2.0:1780298644.584368:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605032320:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.584386:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e4f3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2397:x1866777609606656:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 917us (1001us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.584393:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9946 00010000:00010000:2.0:1780298644.584397:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c37b15a2/0xaf35132e6303523d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41cb expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.584415:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c37b15a2/0xaf35132e6303523d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41cb expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.584430:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baee00, rc 0) 00000100:00100000:2.0:1780298644.584459:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244811f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609606784:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 972us (1053us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.584466:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9947 00000100:00100000:1.0:1780298644.585396:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.585422:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609606912 00000100:00100000:3.0:1780298644.585427:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609606912 00000100:00100000:3.0:1780298644.585457:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9948 00000100:00100000:3.0:1780298644.585465:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244812680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:29360:x1866777609606912:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.585499:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.585508:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63035244 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d41d2 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.585524:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e6303507d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1470464->35024895] (req 1470464->1478655) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d400b expref: 12 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.585531:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(20 122) 00010000:00010000:2.0:1780298644.585534:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(21 122) 00010000:00010000:2.0:1780298644.585538:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63035244 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2269184->2277375] (req 2269184->2277375) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41d2 expref: 10 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.585553:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e6303507d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1470464->35024895] (req 1470464->1478655) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d400b expref: 12 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.585562:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e6303507d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1470464->35024895] (req 1470464->1478655) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d400b expref: 12 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.585570:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605032448:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298644.585599:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb10c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.586261:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605032448:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298644.586289:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63035244 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2269184->2277375] (req 2269184->2277375) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41d2 expref: 10 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.586301:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3c00, rc 0) 00000100:00100000:2.0:1780298644.586331:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244812680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:29360:x1866777609606912:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 867us (935us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.586338:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9948 00000100:00100000:1.0:1780298644.586535:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.586562:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609607040 00000100:00100000:3.0:1780298644.586567:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609607040 00010000:00010000:3.0:1780298644.586588:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e6303507d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1470464->35024895] (req 1470464->1478655) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d400b expref: 13 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.586618:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15207 00000100:00100000:3.0:1780298644.586625:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244812300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609607040:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298644.586635:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298644.586648:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e6303507d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1470464->35024895] (req 1470464->1478655) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d400b expref: 13 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.586661:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e6303507d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1470464->35024895] (req 1470464->1478655) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d400b expref: 13 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.586678:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e6303507d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1470464->35024895] (req 1470464->1478655) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d400b expref: 13 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.586688:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(21 122) 00010000:00010000:3.0:1780298644.586692:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(21 122) 00010000:00010000:3.0:1780298644.586698:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63035244 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2269184->2277375) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41d2 expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.586707:0:6463:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63035244 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2269184->2277375) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41d2 expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.586755:0:6463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63035244 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2269184->2277375) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41d2 expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.586765:0:6463:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777605032576:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298644.586799:0:6463:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb10c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.587528:0:6463:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777605032576:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298644.587538:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298644.587567:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244812300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609607040:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 943us (1032us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.587575:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15207 00000100:00100000:1.0:1780298644.587774:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.587808:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609607168 00000100:00100000:2.0:1780298644.587812:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609607168 00000100:00100000:2.0:1780298644.587850:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9949 00000100:00100000:2.0:1780298644.587858:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244812d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2400:x1866777609607168:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.587867:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.587877:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e6303524b lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d41d9 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.587912:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605032704:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.587946:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb10c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.588660:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605032704:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.588675:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e6303524b lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41d9 expref: 21 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.588685:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e6303524b lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41d9 expref: 21 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.588691:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3e00, rc 0) 00000100:00100000:2.0:1780298644.588713:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244812d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2400:x1866777609607168:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 857us (942us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.588719:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9949 00000100:00100000:1.0:1780298644.589508:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.589550:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609607296 00000100:00100000:2.0:1780298644.589554:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609607296 00002000:00100000:2.0:1780298644.589585:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (2269184->2273279) req@ffff9cf244811180 x1866777609607296/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.589595:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244811180 with x1866777609607296 ext(2269184->2273279) 00002000:00010000:2.0:1780298644.589599:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf244811180 00000100:00100000:2.0:1780298644.589624:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6328 00000100:00100000:2.0:1780298644.589633:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244811180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609607296:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298644.589771:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244811180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609607296:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 141us (265us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.589778:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6328 00002000:00100000:2.0:1780298644.589784:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (2269184->2273279) req@ffff9cf244811180 x1866777609607296/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/440 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.589793:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244811180 with x1866777609607296 ext(2269184->2273279) 00002000:00010000:2.0:1780298644.589796:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf244811180 00000100:00100000:1.0:1780298644.590514:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.590546:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609607424 00000100:00100000:2.0:1780298644.590551:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609607424 00002000:00100000:2.0:1780298644.590570:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x12:0x0] (2273280->2277375) req@ffff9cf244812a00 x1866777609607424/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.590580:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244812a00 with x1866777609607424 ext(2273280->2277375) 00002000:00010000:2.0:1780298644.590583:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf244812a00 00000100:00100000:2.0:1780298644.590602:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6329 00000100:00100000:2.0:1780298644.590609:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244812a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609607424:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298644.590679:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244812a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609607424:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 70us (165us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.590685:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6329 00002000:00100000:2.0:1780298644.590691:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x12:0x0] (2273280->2277375) req@ffff9cf244812a00 x1866777609607424/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/440 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.590700:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244812a00 with x1866777609607424 ext(2273280->2277375) 00002000:00010000:2.0:1780298644.590703:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf244812a00 00000100:00100000:1.0:1780298644.591559:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.591601:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609607552 00000100:00100000:0.0:1780298644.591607:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609607552 00000100:00100000:0.0:1780298644.591642:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9950 00000100:00100000:0.0:1780298644.591651:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244810e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2398:x1866777609607552:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.591660:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.591672:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005d10d3aa/0xaf35132e63035252 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d41e0 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.591690:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.591711:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605032832:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.591739:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609607680 00000100:00100000:3.0:1780298644.591745:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609607680 00000100:00100000:0.0:1780298644.591759:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.591783:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9951 00000100:00100000:3.0:1780298644.591792:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244813800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609607680:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.591802:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.591812:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63035259 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d41e7 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.591827:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(21 122) 00010000:00010000:3.0:1780298644.591832:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(21 122) 00010000:00010000:3.0:1780298644.591838:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63035259 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41e7 expref: 13 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.591895:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63035259 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41e7 expref: 13 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.591906:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157c00, rc 0) 00000100:00100000:3.0:1780298644.591938:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244813800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609607680:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 148us (248us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298644.591945:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9951 00000100:00100000:0.0:1780298644.592383:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605032832:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.592397:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005d10d3aa/0xaf35132e63035252 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41e0 expref: 24 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.592407:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005d10d3aa/0xaf35132e63035252 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41e0 expref: 24 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.592414:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baec00, rc 0) 00000100:00100000:0.0:1780298644.592436:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244810e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2398:x1866777609607552:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 788us (878us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.592440:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9950 00000100:00100000:1.0:1780298644.593297:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.593330:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609607808 00000100:00100000:2.0:1780298644.593336:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609607808 00000100:00100000:2.0:1780298644.593366:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9952 00000100:00100000:2.0:1780298644.593375:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244810700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2400:x1866777609607808:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.593384:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.593392:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e63035260 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d41ee expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.593418:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605032960:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.593455:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb10c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.594069:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605032960:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.594084:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e63035260 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41ee expref: 24 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.594097:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e63035260 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41ee expref: 24 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.594107:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2400, rc 0) 00000100:00100000:2.0:1780298644.594206:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244810700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2400:x1866777609607808:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 832us (908us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.594214:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9952 00000100:00100000:1.0:1780298644.595009:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.595048:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609607936 00000100:00100000:2.0:1780298644.595054:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609607936 00000100:00100000:2.0:1780298644.595085:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9953 00000100:00100000:2.0:1780298644.595092:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244810a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:29360:x1866777609607936:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.595101:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.595110:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e63035267 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d41f5 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.595160:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63035244 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2269184->2277375) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41d2 expref: 11 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.595171:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63035259 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41e7 expref: 12 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.595178:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(21 122) 00010000:00010000:2.0:1780298644.595182:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(21 122) 00010000:00010000:2.0:1780298644.595187:0:20252:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e63035267 lrc: 5/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [565248->573439] (req 565248->573439) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41f5 expref: 11 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.595209:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63035259 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41e7 expref: 12 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.595223:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63035259 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41e7 expref: 12 pid: 12101 timeout: 934 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.595236:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605033088:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298644.595299:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63035244 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2269184->2277375) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41d2 expref: 11 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.595310:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63035244 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2269184->2277375) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41d2 expref: 11 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.595328:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b40000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605033216:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298644.595361:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb10c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.595921:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605033088:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298644.596038:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.596056:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b40000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605033216:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298644.596078:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e63035267 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [565248->573439] (req 565248->573439) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41f5 expref: 11 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.596085:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609608064 00010000:00010000:2.0:1780298644.596089:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3000, rc 0) 00000100:00100000:0.0:1780298644.596091:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609608064 00010000:00010000:0.0:1780298644.596115:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63035259 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41e7 expref: 13 pid: 12101 timeout: 934 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.596120:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244810a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:29360:x1866777609607936:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1025us (1109us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.596150:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9953 00000100:00100000:0.0:1780298644.596186:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15208 00000100:00100000:0.0:1780298644.596197:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244810000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609608064:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298644.596207:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298644.596221:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63035259 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41e7 expref: 13 pid: 12101 timeout: 934 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.596234:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63035259 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41e7 expref: 13 pid: 12101 timeout: 934 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.596250:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63035259 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41e7 expref: 13 pid: 12101 timeout: 934 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.596263:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298644.596319:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244810000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2399:x1866777609608064:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 124us (280us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.596327:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.596338:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15208 00000100:00100000:0.0:1780298644.596357:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609608192 00000100:00100000:0.0:1780298644.596361:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609608192 00010000:00010000:0.0:1780298644.596378:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63035244 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2269184->2277375) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41d2 expref: 11 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.596410:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15209 00000100:00100000:0.0:1780298644.596419:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348af6450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609608192:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298644.596427:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298644.596436:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63035244 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2269184->2277375) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41d2 expref: 11 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.596447:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63035244 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2269184->2277375) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41d2 expref: 11 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.596463:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63035244 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2269184->2277375) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41d2 expref: 11 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.596474:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(21 122) 00010000:00010000:0.0:1780298644.596478:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(21 122) 00010000:00010000:0.0:1780298644.596486:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e63035267 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [565248->34119679] (req 565248->573439) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41f5 expref: 10 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.596496:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e63035267 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [565248->34119679] (req 565248->573439) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41f5 expref: 10 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.596536:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e63035267 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [565248->34119679] (req 565248->573439) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41f5 expref: 10 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.596556:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd5c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:20253:1866777605033344:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298644.596616:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393b40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.597298:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.597340:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609608320 00000100:00100000:2.0:1780298644.597344:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609608320 00000100:00100000:2.0:1780298644.597381:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6330 00000100:00100000:2.0:1780298644.597390:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244813100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609608320:12345-192.168.203.25@tcp:22:fsx.0 00000100:00100000:0.0:1780298644.597508:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd5c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:20253:1866777605033344:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298644.597517:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298644.597538:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348af6450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609608192:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1122us (1214us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.597544:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15209 00002000:00010000:2.0:1780298644.597628:0:8364:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 2212497 -> 2212497 00002000:00010000:2.0:1780298644.597632:0:8364:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298645 -> 1780298644 00002000:00010000:2.0:1780298644.597635:0:8364:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298645 -> 1780298644 00002000:00010000:2.0:1780298644.597637:0:8364:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298645 -> 1780298644 00002000:00010000:2.0:1780298644.597639:0:8364:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 40 -> 56 00000100:00100000:2.0:1780298644.597669:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244813100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609608320:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 280us (370us total) trans 4294969278 rc 0/0 00000100:00100000:2.0:1780298644.597677:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6330 00000100:00100000:1.0:1780298644.598480:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.598614:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609608448 00000100:00100000:2.0:1780298644.598620:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609608448 00000100:00100000:2.0:1780298644.598655:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9954 00000100:00100000:2.0:1780298644.598664:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244811880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2399:x1866777609608448:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.598675:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.598685:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e6303526e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d41fc expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.598722:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b40000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605033472:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.598775:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb10c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.598845:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.599119:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609608576 00000100:00100000:2.0:1780298644.599156:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609608576 00000100:00100000:2.0:1780298644.599188:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9955 00000100:00100000:2.0:1780298644.599195:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244813b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609608576:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.599203:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.599212:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e63035275 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4203 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.599241:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605033600:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.599295:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb16c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.599727:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b40000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605033472:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.599750:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e6303526e lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41fc expref: 24 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.599767:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000578270eb/0xaf35132e6303526e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41fc expref: 24 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.599784:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2800, rc 0) 00000100:00100000:2.0:1780298644.599820:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244811880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2399:x1866777609608448:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1152us (1334us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.599858:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9954 00002000:00010000:2.0:1780298644.600079:0:9128:0:(ofd_lvb.c:265:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime: 1780298644 -> 1780298645 00002000:00010000:2.0:1780298644.600082:0:9128:0:(ofd_lvb.c:271:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime: 1780298644 -> 1780298645 00002000:00010000:2.0:1780298644.600085:0:9128:0:(ofd_lvb.c:277:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime: 1780298644 -> 1780298645 00000100:00100000:2.0:1780298644.600093:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605033600:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.600107:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e63035275 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4203 expref: 13 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.600119:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e63035275 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4203 expref: 13 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.600143:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3e00, rc 0) 00000100:00100000:2.0:1780298644.600171:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244813b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609608576:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 976us (1326us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.600178:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9955 00000100:00100000:3.0:1780298644.601007:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.601035:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.601051:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609608704 00000100:00100000:2.0:1780298644.601056:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609608704 00000100:00100000:0.0:1780298644.601075:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609608832 00000100:00100000:0.0:1780298644.601081:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609608832 00000100:00100000:2.0:1780298644.601090:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9956 00000100:00100000:2.0:1780298644.601099:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24380a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2400:x1866777609608704:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298644.601105:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9957 00010000:00010000:2.0:1780298644.601111:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298644.601118:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243808a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2399:x1866777609608832:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.601121:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e6303527c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d420a expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.601147:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.601160:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e0b8e7b/0xaf35132e63035283 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4211 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.601173:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605033728:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.601195:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243808000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605033856:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.601218:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb16c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.601230:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.602028:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605033728:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.602049:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e6303527c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d420a expref: 24 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.602064:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e6303527c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d420a expref: 24 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.602074:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2000, rc 0) 00000100:00100000:3.0:1780298644.602084:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243808000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605033856:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.602107:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24380a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2400:x1866777609608704:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1009us (1100us total) trans 0 rc 301/301 00010000:00010000:3.0:1780298644.602113:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009e0b8e7b/0xaf35132e63035283 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4211 expref: 13 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.602118:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9956 00010000:00010000:3.0:1780298644.602154:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e0b8e7b/0xaf35132e63035283 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4211 expref: 13 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.602170:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156c00, rc 0) 00000100:00100000:3.0:1780298644.602194:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243808a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2399:x1866777609608832:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1081us (1159us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.602202:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9957 00000100:00100000:1.0:1780298644.603303:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.603325:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.603347:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609609088 00000100:00100000:0.0:1780298644.603352:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609609088 00000100:00100000:2.0:1780298644.603357:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609608960 00000100:00100000:2.0:1780298644.603362:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609608960 00000100:00100000:0.0:1780298644.603389:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9958 00000100:00100000:0.0:1780298644.603403:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244813480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609609088:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298644.603409:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9959 00010000:00010000:0.0:1780298644.603414:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.603425:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e6303528a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d421f expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.603430:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244811c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2398:x1866777609608960:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.603444:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.603454:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e63035291 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4218 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.603469:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605033984:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.603488:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605034112:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.603528:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.603533:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb16c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.604340:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605034112:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.604353:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605033984:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.604362:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e63035291 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4218 expref: 21 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.604372:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e6303528a lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d421f expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.604382:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e63035291 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4218 expref: 21 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.604387:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e6303528a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d421f expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.604402:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2400, rc 0) 00010000:00010000:0.0:1780298644.604404:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baec00, rc 0) 00000100:00100000:0.0:1780298644.604443:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244813480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609609088:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1044us (1137us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.604444:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244811c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2398:x1866777609608960:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1016us (1113us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.604458:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9958 00000100:00100000:2.0:1780298644.604465:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9959 00000100:00100000:1.0:1780298644.605447:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.605489:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609609216 00000100:00100000:3.0:1780298644.605494:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609609216 00000100:00100000:3.0:1780298644.605531:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9960 00000100:00100000:3.0:1780298644.605540:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244812300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:29360:x1866777609609216:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.605549:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.605558:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e63035298 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4226 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.605575:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(23 169) 00010000:00010000:3.0:1780298644.605578:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(23 169) 00010000:00010000:3.0:1780298644.605585:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e63035298 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3465216->3891199] (req 3465216->3473407) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4226 expref: 21 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.605597:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e63035298 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3465216->3891199] (req 3465216->3473407) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4226 expref: 21 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.605606:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157000, rc 0) 00000100:00100000:3.0:1780298644.605635:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244812300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:29360:x1866777609609216:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 97us (191us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298644.605642:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9960 00000100:00100000:1.0:1780298644.606313:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.606344:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609609344 00000100:00100000:0.0:1780298644.606348:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609609344 00000100:00100000:0.0:1780298644.606378:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6331 00000100:00100000:0.0:1780298644.606384:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244810000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2399:x1866777609609344:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298644.606552:0:8362:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 5683587 -> 3472008 00002000:00010000:0.0:1780298644.606556:0:8362:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298644.606559:0:8362:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298644.606561:0:8362:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298644.606563:0:8362:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 344 -> 360 00000100:00100000:0.0:1780298644.606593:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244810000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2399:x1866777609609344:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 208us (280us total) trans 4294969733 rc 0/0 00000100:00100000:0.0:1780298644.606600:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6331 00000100:00100000:1.0:1780298644.607582:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.607613:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609609472 00000100:00100000:3.0:1780298644.607617:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609609472 00000100:00100000:3.0:1780298644.607705:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9961 00000100:00100000:3.0:1780298644.607713:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2400:x1866777609609472:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.607722:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298644.607727:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:3.0:1780298644.607731:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000042c16ab0/0xaf35132e6303529f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d422d expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.607762:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609609600 00000100:00100000:2.0:1780298644.607766:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609609600 00000100:00100000:3.0:1780298644.607771:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243808000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605034240:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.607795:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9962 00000100:00100000:2.0:1780298644.607800:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609609600:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.607808:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.607815:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e630352a6 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4234 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.607839:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605034368:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.607867:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb16c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.607877:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856cc0 going to sleep for 11 seconds 00002000:00010000:3.0:1780298644.608551:0:20252:0:(ofd_lvb.c:259:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size: 3472008 -> 5683587 00000100:00100000:3.0:1780298644.608562:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243808000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605034240:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.608601:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000042c16ab0/0xaf35132e6303529f lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d422d expref: 24 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.608614:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000042c16ab0/0xaf35132e6303529f lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d422d expref: 24 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.608628:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605034368:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.608646:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e630352a6 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4234 expref: 13 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.608667:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156800, rc 0) 00010000:00010000:2.0:1780298644.608672:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e630352a6 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4234 expref: 13 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.608681:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3800, rc 0) 00000100:00100000:3.0:1780298644.608692:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2400:x1866777609609472:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 980us (1110us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.608696:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9961 00000100:00100000:2.0:1780298644.608704:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609609600:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 901us (976us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.608710:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9962 00000100:00100000:1.0:1780298644.609759:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.609794:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.609796:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609609728 00000100:00100000:2.0:1780298644.609801:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609609728 00000100:00100000:2.0:1780298644.609827:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609609856 00000100:00100000:2.0:1780298644.609829:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609609856 00000100:00100000:1.0:1780298644.609841:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9963 00000100:00100000:2.0:1780298644.609848:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9964 00000100:00100000:1.0:1780298644.609856:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2398:x1866777609609728:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298644.609857:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609609856:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.609870:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.609872:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.609879:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e630352ad lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4242 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.609884:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e630352b4 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d423b expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.609905:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605034496:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.609925:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605034624:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.609944:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb16c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.609964:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.610674:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605034496:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.610697:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e630352ad lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4242 expref: 13 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.610714:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e630352ad lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4242 expref: 13 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.610726:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2e00, rc 0) 00000100:00100000:3.0:1780298644.610757:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605034624:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.610763:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609609856:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 906us (968us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.610772:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9964 00010000:00010000:3.0:1780298644.610817:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e630352b4 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d423b expref: 24 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.610832:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e630352b4 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d423b expref: 24 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.610839:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279c00, rc 0) 00000100:00100000:3.0:1780298644.610860:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2398:x1866777609609728:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1012us (1103us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.610930:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9963 00000100:00100000:1.0:1780298644.611933:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.611983:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609609984 00000100:00100000:0.0:1780298644.611989:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609609984 00002000:00100000:0.0:1780298644.612018:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (4005888->4009983) req@ffff9cf3481f9f80 x1866777609609984/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298644.612029:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481f9f80 with x1866777609609984 ext(4005888->4009983) 00002000:00010000:0.0:1780298644.612034:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481f9f80 00000100:00100000:0.0:1780298644.612056:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6332 00000100:00100000:0.0:1780298644.612065:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2397:x1866777609609984:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298644.612236:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2397:x1866777609609984:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 173us (304us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.612241:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6332 00002000:00100000:0.0:1780298644.612245:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (4005888->4009983) req@ffff9cf3481f9f80 x1866777609609984/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/440 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298644.612252:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481f9f80 with x1866777609609984 ext(4005888->4009983) 00002000:00010000:0.0:1780298644.612255:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481f9f80 00000100:00100000:1.0:1780298644.613099:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.613172:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609610112 00000100:00100000:2.0:1780298644.613178:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609610112 00002000:00100000:2.0:1780298644.613200:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x2c0000401:0x11:0x0] (4014080->4018175) req@ffff9cf3481f8e00 x1866777609610112/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.613210:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481f8e00 with x1866777609610112 ext(4014080->4018175) 00002000:00010000:2.0:1780298644.613213:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481f8e00 00000100:00100000:2.0:1780298644.613237:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6333 00000100:00100000:2.0:1780298644.613245:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2397:x1866777609610112:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298644.613333:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2397:x1866777609610112:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 89us (234us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.613341:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6333 00002000:00100000:2.0:1780298644.613348:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x2c0000401:0x11:0x0] (4014080->4018175) req@ffff9cf3481f8e00 x1866777609610112/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/440 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.613357:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481f8e00 with x1866777609610112 ext(4014080->4018175) 00002000:00010000:2.0:1780298644.613361:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481f8e00 00000100:00100000:1.0:1780298644.614228:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.614266:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609610368 00000100:00100000:0.0:1780298644.614270:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609610368 00000100:00100000:0.0:1780298644.614310:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9965 00000100:00100000:0.0:1780298644.614320:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609610368:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.614331:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.614341:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e630352bb lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4250 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.614343:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.614375:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605034752:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.614381:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609610240 00000100:00100000:2.0:1780298644.614387:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609610240 00000100:00100000:2.0:1780298644.614427:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9966 00000100:00100000:0.0:1780298644.614438:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.614441:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2397:x1866777609610240:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.614451:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.614460:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e630352c2 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4249 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.614491:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605034880:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.614523:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb16c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.615168:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605034752:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.615183:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e630352bb lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4250 expref: 13 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.615199:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e630352bb lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4250 expref: 13 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.615206:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baec00, rc 0) 00000100:00100000:0.0:1780298644.615242:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609610368:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 924us (1014us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.615250:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9965 00000100:00100000:2.0:1780298644.615427:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605034880:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.615438:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e630352c2 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4249 expref: 24 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.615447:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e630352c2 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4249 expref: 24 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.615454:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2000, rc 0) 00000100:00100000:2.0:1780298644.615476:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2397:x1866777609610240:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1043us (1133us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.615482:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9966 00000100:00100000:1.0:1780298644.616362:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.616393:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609610496 00000100:00100000:0.0:1780298644.616397:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609610496 00000100:00100000:0.0:1780298644.616423:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9967 00000100:00100000:0.0:1780298644.616430:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2398:x1866777609610496:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.616438:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.616446:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e630352c9 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4257 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.616468:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605035008:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.616510:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.616521:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.616561:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609610624 00000100:00100000:0.0:1780298644.616566:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609610624 00000100:00100000:0.0:1780298644.616596:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9968 00000100:00100000:0.0:1780298644.616604:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243808000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609610624:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.616612:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.616619:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d4d8e3d7/0xaf35132e630352d0 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d425e expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.616649:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605035136:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.616681:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733936c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.617226:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605035008:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.617264:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e630352c9 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4257 expref: 24 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.617302:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e630352c9 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4257 expref: 24 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.617317:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf600, rc 0) 00000100:00100000:3.0:1780298644.617360:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605035136:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.617371:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2398:x1866777609610496:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 939us (1005us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.617380:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9967 00010000:00010000:3.0:1780298644.617386:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d4d8e3d7/0xaf35132e630352d0 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d425e expref: 13 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.617407:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d4d8e3d7/0xaf35132e630352d0 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d425e expref: 13 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.617423:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf000, rc 0) 00000100:00100000:3.0:1780298644.617454:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243808000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609610624:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 851us (941us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.617510:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9968 00000100:00100000:1.0:1780298644.618661:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.618702:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609610752 00000100:00100000:2.0:1780298644.618706:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609610752 00002000:00100000:2.0:1780298644.618726:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (4874240->4878335) req@ffff9cf3481fb800 x1866777609610752/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.618734:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fb800 with x1866777609610752 ext(4874240->4878335) 00002000:00010000:2.0:1780298644.618737:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481fb800 00000100:00100000:2.0:1780298644.618755:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6334 00000100:00100000:2.0:1780298644.618761:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2397:x1866777609610752:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298644.618854:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2397:x1866777609610752:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 92us (196us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.618859:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6334 00002000:00100000:2.0:1780298644.618864:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (4874240->4878335) req@ffff9cf3481fb800 x1866777609610752/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/440 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.618870:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fb800 with x1866777609610752 ext(4874240->4878335) 00002000:00010000:2.0:1780298644.618872:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481fb800 00000100:00100000:1.0:1780298644.619977:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.619997:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.620026:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609610880 00000100:00100000:0.0:1780298644.620027:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609611008 00000100:00100000:3.0:1780298644.620037:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609610880 00000100:00100000:0.0:1780298644.620037:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609611008 00000100:00100000:0.0:1780298644.620091:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9969 00000100:00100000:0.0:1780298644.620104:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609611008:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298644.620110:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9970 00010000:00010000:0.0:1780298644.620121:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.620164:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e630352d7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d426c expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.620181:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2398:x1866777609610880:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298644.620205:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605035264:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.620225:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.620237:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000488d1c2a/0xaf35132e630352de lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4265 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.620274:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605035392:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.620304:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.620332:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837ae40 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.620999:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605035264:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.621023:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e630352d7 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d426c expref: 13 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.621039:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e630352d7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d426c expref: 13 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.621048:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae400, rc 0) 00000100:00100000:0.0:1780298644.621085:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609611008:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 984us (1084us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.621094:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9969 00000100:00100000:1.0:1780298644.621159:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605035392:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.621185:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000488d1c2a/0xaf35132e630352de lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4265 expref: 24 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.621199:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000488d1c2a/0xaf35132e630352de lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4265 expref: 24 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.621212:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279400, rc 0) 00000100:00100000:1.0:1780298644.621233:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2398:x1866777609610880:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1115us (1258us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.621239:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9970 00000100:00100000:1.0:1780298644.622295:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.622411:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.622441:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609611136 00000100:00100000:0.0:1780298644.622444:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609611264 00000100:00100000:3.0:1780298644.622448:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609611136 00000100:00100000:0.0:1780298644.622449:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609611264 00000100:00100000:0.0:1780298644.622496:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9971 00000100:00100000:0.0:1780298644.622507:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609611264:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.622519:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.622533:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e630352e5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d427a expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.622537:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9972 00000100:00100000:3.0:1780298644.622557:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2397:x1866777609611136:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.622567:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298644.622579:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605035520:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.622590:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000042c16ab0/0xaf35132e630352ec lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4273 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.622626:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24380b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605035648:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.622628:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.622661:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.623349:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605035520:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.623370:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e630352e5 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d427a expref: 13 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.623395:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e630352e5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d427a expref: 13 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.623406:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf600, rc 0) 00000100:00100000:0.0:1780298644.623442:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609611264:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 939us (1032us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.623448:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9971 00000100:00100000:3.0:1780298644.623485:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24380b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605035648:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.623546:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000042c16ab0/0xaf35132e630352ec lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4273 expref: 24 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.623556:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000042c16ab0/0xaf35132e630352ec lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4273 expref: 24 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.623563:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156800, rc 0) 00000100:00100000:3.0:1780298644.623585:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2397:x1866777609611136:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1040us (1293us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.623590:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9972 00000100:00100000:1.0:1780298644.624524:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.624573:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609611520 00000100:00100000:2.0:1780298644.624579:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609611520 00000100:00100000:2.0:1780298644.624622:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62037 00000100:00100000:2.0:1780298644.624633:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+139:29360:x1866777609611520:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:2.0:1780298644.624681:0:6477:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298644.624703:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278600/0xaf35132e6303501b lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 95 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fb0 expref: 139 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.624719:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298644.624724:0:6477:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298644.624880:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2800/0xaf35132e630352f3 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 94 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.624895:0:6477:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278c00/0xaf35132e6303504c lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 94 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fda expref: 172 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.624917:0:6477:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2800/0xaf35132e630352f3 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 94 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.624946:0:6477:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278c00/0xaf35132e6303504c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 94 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fda expref: 172 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.624959:0:6477:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278c00/0xaf35132e6303504c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 94 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fda expref: 172 pid: 17578 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.624973:0:6477:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b40000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6477:1866777605035776:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298644.625019:0:6477:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb10c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.626707:0:6477:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b40000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6477:1866777605035776:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298644.626723:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:2.0:1780298644.626728:0:6477:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2800/0xaf35132e630352f3 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 94 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.626777:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609611648 00000100:00100000:3.0:1780298644.626822:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609611648 00010000:00010000:3.0:1780298644.626850:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278c00/0xaf35132e6303504c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 94 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fda expref: 173 pid: 17578 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.626887:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15210 00000100:00100000:3.0:1780298644.626895:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481faa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+173:2399:x1866777609611648:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298644.626906:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298644.626911:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278c00/0xaf35132e6303504c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 95 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fda expref: 173 pid: 17578 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.626919:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278c00/0xaf35132e6303504c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 95 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fda expref: 173 pid: 17578 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.626932:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278c00/0xaf35132e6303504c lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 95 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d3fda expref: 173 pid: 17578 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.626942:0:6462:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298644.626945:0:6462:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2800/0xaf35132e630352f3 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 94 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.626964:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2800/0xaf35132e630352f3 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 94 type: IBT flags: 0x40290000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.626972:0:6462:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2800/0xaf35132e630352f3 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 94 type: IBT flags: 0x40290000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.626996:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298644.627018:0:6477:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2800/0xaf35132e630352f3 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 93 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.627028:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2800/0xaf35132e630352f3 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 93 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.627036:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481faa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+172:2399:x1866777609611648:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 142us (315us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298644.627043:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15210 00000001:00080000:2.0:1780298644.627080:0:6477:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991090 00000004:00010000:2.0:1780298644.627111:0:6477:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 0000000020d9a02a reply state 00000000b9758a1f transno 4294991091 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2800/0xaf35132e630352f3 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 93 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.627149:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2800/0xaf35132e630352f3 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 93 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.627158:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298644.627162:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1780298644.627165:0:6477:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1780298644.627237:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+141:29360:x1866777609611520:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 2605us (2711us total) trans 4294991091 rc 0/0 00000100:00100000:2.0:1780298644.627245:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62037 00000020:00080000:2.0:1780298644.627250:0:6477:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1780298644.627976:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.628012:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609611776 00000100:00100000:2.0:1780298644.628018:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609611776 00000100:00100000:2.0:1780298644.628057:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9973 00000100:00100000:2.0:1780298644.628065:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:29360:x1866777609611776:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.628075:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.628086:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e630352fa lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4281 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.628107:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63035110 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [151552->720895] (req 151552->159743) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d409e expref: 22 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.628116:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e63035298 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3465216->3891199] (req 3465216->3473407) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4226 expref: 23 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.628153:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e63035195 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3895296->37449727] (req 3895296->3899391) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4123 expref: 22 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.628163:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 3 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(23 169) 00010000:00010000:2.0:1780298644.628167:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(26 169) 00010000:00010000:2.0:1780298644.628173:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e630352fa lrc: 6/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4281 expref: 23 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.628194:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e63035195 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3895296->37449727] (req 3895296->3899391) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4123 expref: 22 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.628206:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e63035195 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3895296->37449727] (req 3895296->3899391) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4123 expref: 22 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.628217:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b40000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605035904:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298644.628269:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e63035298 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3465216->3891199] (req 3465216->3473407) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4226 expref: 23 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.628292:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e63035298 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3465216->3891199] (req 3465216->3473407) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4226 expref: 23 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.628302:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605036032:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298644.628337:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63035110 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [151552->720895] (req 151552->159743) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d409e expref: 22 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.628348:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63035110 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [151552->720895] (req 151552->159743) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d409e expref: 22 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.628358:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605036160:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298644.628385:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb16c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.629035:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b40000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605035904:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298644.629042:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605036032:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298644.629184:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605036160:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298644.629201:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e630352fa lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4281 expref: 23 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.629212:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3600, rc 0) 00000100:00100000:2.0:1780298644.629234:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+23:29360:x1866777609611776:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1171us (1258us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.629240:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9973 00000100:00100000:1.0:1780298644.629345:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.629374:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.629402:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609611904 00000100:00100000:1.0:1780298644.629412:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.629413:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609611904 00000100:00100000:0.0:1780298644.629425:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609612160 00000100:00100000:0.0:1780298644.629431:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609612160 00010000:00010000:3.0:1780298644.629445:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e63035298 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3465216->3891199] (req 3465216->3473407) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4226 expref: 23 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.629451:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63035110 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [151552->720895] (req 151552->159743) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d409e expref: 23 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.629468:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609612032 00000100:00100000:2.0:1780298644.629479:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609612032 00002000:00100000:2.0:1780298644.629511:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (4005888->5683586) req@ffff9cf37f736d80 x1866777609612032/t0(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 504/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00000100:00100000:3.0:1780298644.629517:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15211 00002000:00100000:2.0:1780298644.629532:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f736d80 with x1866777609612032 ext(4005888->5683586) 00000100:00100000:0.0:1780298644.629533:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15212 00000100:00100000:3.0:1780298644.629542:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f734000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+23:2399:x1866777609611904:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298644.629550:0:8364:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e63035195 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3895296->37449727] (req 3895296->3899391) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4123 expref: 24 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.629559:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:0.0:1780298644.629561:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f736a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+24:2398:x1866777609612160:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298644.629576:0:8364:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e63035195 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [3895296->37449727] (req 3895296->3899391) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4123 expref: 24 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.629582:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:2.0:1780298644.629584:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf37f736d80 00010000:00010000:3.0:1780298644.629598:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e63035298 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [3465216->3891199] (req 3465216->3473407) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4226 expref: 23 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.629604:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63035110 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 8 type: EXT [151552->720895] (req 151552->159743) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d409e expref: 24 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.629622:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 208 00000100:00100000:2.0:1780298644.629647:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f736d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+24:2400:x1866777609612032:12345-192.168.203.25@tcp:4:fsx.0 00010000:00010000:3.0:1780298644.629648:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e63035298 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [3465216->3891199] (req 3465216->3473407) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4226 expref: 23 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.629673:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63035110 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [151552->720895] (req 151552->159743) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d409e expref: 24 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.629700:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e63035298 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [3465216->3891199] (req 3465216->3473407) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4226 expref: 23 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.629710:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63035110 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 8 type: EXT [151552->720895] (req 151552->159743) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d409e expref: 24 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.629720:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1780298644.629736:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298644.629767:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f734000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2399:x1866777609611904:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 234us (423us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.629775:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f736a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+24:2398:x1866777609612160:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 223us (397us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298644.629787:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15211 00000100:00100000:0.0:1780298644.629797:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15212 00000100:00100000:2.0:1780298644.631064:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f736d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+24:2400:x1866777609612032:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 1426us (1654us total) trans 4294969734 rc 0/0 00000100:00100000:2.0:1780298644.631073:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 208 00002000:00100000:2.0:1780298644.631080:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (4005888->5683586) req@ffff9cf37f736d80 x1866777609612032/t4294969734(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 504/448 e 0 to 0 dl 1780298655 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.631090:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f736d80 with x1866777609612032 ext(4005888->5683586) 00010000:00010000:2.0:1780298644.631097:0:8364:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e63035195 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3895296->37449727] (req 3895296->3899391) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4123 expref: 24 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.631108:0:8364:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e63035195 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3895296->37449727] (req 3895296->3899391) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4123 expref: 24 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1780298644.631113:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf37f736d80 00000100:00100000:3.0:1780298644.631905:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.631961:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609612288 00000100:00100000:2.0:1780298644.631967:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609612288 00010000:00010000:2.0:1780298644.631987:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e63035195 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3895296->37449727] (req 3895296->3899391) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4123 expref: 24 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.632009:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15213 00000100:00100000:2.0:1780298644.632015:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24380b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+24:2397:x1866777609612288:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298644.632024:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:2.0:1780298644.632032:0:6463:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 360 -> 400 00010000:00010000:2.0:1780298644.632037:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e63035195 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3895296->37449727] (req 3895296->3899391) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4123 expref: 24 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.632045:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e63035195 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3895296->37449727] (req 3895296->3899391) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4123 expref: 24 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.632057:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e63035195 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3895296->37449727] (req 3895296->3899391) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4123 expref: 24 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.632078:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(26 169) 00010000:00010000:2.0:1780298644.632081:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(26 169) 00010000:00010000:2.0:1780298644.632086:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e630352fa lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4281 expref: 21 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.632092:0:6463:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e630352fa lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4281 expref: 21 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.632115:0:6463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e630352fa lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4281 expref: 21 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.632140:0:6463:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6463:1866777605036288:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298644.632192:0:6463:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb16c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.632878:0:6463:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6463:1866777605036288:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298644.632891:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298644.632914:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24380b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2397:x1866777609612288:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 900us (1015us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.632919:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15213 00000100:00100000:3.0:1780298644.633023:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.633077:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609612416 00000100:00100000:2.0:1780298644.633083:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609612416 00000100:00100000:2.0:1780298644.633121:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9974 00000100:00100000:2.0:1780298644.633158:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243809180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:29360:x1866777609612416:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.633172:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.633185:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e63035301 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4288 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.633204:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e63035267 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [565248->34119679] (req 565248->573439) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41f5 expref: 12 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.633212:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(21 122) 00010000:00010000:2.0:1780298644.633215:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(22 122) 00010000:00010000:2.0:1780298644.633219:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e63035301 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4288 expref: 12 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.633239:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e63035267 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [565248->34119679] (req 565248->573439) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41f5 expref: 12 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.633265:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e63035267 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [565248->34119679] (req 565248->573439) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41f5 expref: 12 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.633288:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605036416:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298644.633331:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb16c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.633878:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605036416:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298644.633895:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e63035301 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4288 expref: 12 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.633906:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3e00, rc 0) 00000100:00100000:2.0:1780298644.633938:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243809180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:29360:x1866777609612416:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 784us (917us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.633949:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9974 00000100:00100000:1.0:1780298644.634165:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.634197:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609612544 00000100:00100000:2.0:1780298644.634202:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609612544 00010000:00010000:2.0:1780298644.634222:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e63035267 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [565248->34119679] (req 565248->573439) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41f5 expref: 12 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.634251:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15214 00000100:00100000:2.0:1780298644.634259:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f734a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609612544:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298644.634268:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298644.634293:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e63035267 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [565248->34119679] (req 565248->573439) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41f5 expref: 12 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.634304:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e63035267 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [565248->34119679] (req 565248->573439) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41f5 expref: 12 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.634318:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e63035267 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [565248->34119679] (req 565248->573439) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d41f5 expref: 12 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.634328:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(22 122) 00010000:00010000:2.0:1780298644.634331:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(22 122) 00010000:00010000:2.0:1780298644.634336:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e63035301 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4288 expref: 11 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.634344:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e63035301 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4288 expref: 11 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.634370:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e63035301 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4288 expref: 11 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.634379:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:8374:1866777605036544:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298644.634419:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb16c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.635157:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:8374:1866777605036544:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298644.635175:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298644.635212:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f734a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609612544:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 954us (1050us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.635220:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15214 00000100:00100000:3.0:1780298644.635388:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.635427:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.635431:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609612672 00000100:00100000:0.0:1780298644.635438:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609612672 00002000:00010000:0.0:1780298644.635460:0:8362:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (5011835->18446744073709551615) 00002000:00100000:0.0:1780298644.635463:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243809880 with x1866777609612672 ext(5011835->18446744073709551615) 00002000:00010000:0.0:1780298644.635471:0:8362:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf243809880. 00000100:00100000:2.0:1780298644.635473:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609612800 00000100:00100000:2.0:1780298644.635480:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609612800 00002000:00010000:2.0:1780298644.635501:0:16972:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4194304->18446744073709551615) 00000100:00100000:0.0:1780298644.635505:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6335 00002000:00100000:2.0:1780298644.635507:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243808700 with x1866777609612800 ext(4194304->18446744073709551615) 00002000:00010000:2.0:1780298644.635517:0:16972:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf243808700. 00000100:00100000:0.0:1780298644.635525:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243809880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2400:x1866777609612672:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:2.0:1780298644.635549:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6336 00000100:00100000:2.0:1780298644.635559:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243808700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609612800:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:2.0:1780298644.635718:0:16972:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 2212497 -> 4194304 00002000:00010000:2.0:1780298644.635723:0:16972:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298644.635725:0:16972:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298645 -> 1780298644 00002000:00010000:2.0:1780298644.635728:0:16972:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298644.635730:0:16972:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 56 -> 56 00000100:00100000:2.0:1780298644.635767:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243808700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609612800:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 210us (345us total) trans 4294969279 rc 0/0 00002000:00010000:0.0:1780298644.635772:0:8362:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 5683587 -> 5011835 00000100:00100000:2.0:1780298644.635777:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6336 00002000:00010000:0.0:1780298644.635782:0:8362:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298644.635784:0:16972:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4194304->18446744073709551615) 00002000:00010000:0.0:1780298644.635788:0:8362:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00100000:2.0:1780298644.635791:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243808700 with x1866777609612800 ext(4194304->18446744073709551615) 00002000:00010000:0.0:1780298644.635795:0:8362:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298644.635798:0:16972:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf243808700. 00002000:00010000:0.0:1780298644.635800:0:8362:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 400 -> 384 00000100:00100000:0.0:1780298644.635840:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243809880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+23:2400:x1866777609612672:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 323us (482us total) trans 4294969735 rc 0/0 00000100:00100000:0.0:1780298644.635848:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6335 00002000:00010000:0.0:1780298644.635852:0:8362:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (5011835->18446744073709551615) 00002000:00100000:0.0:1780298644.635856:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243809880 with x1866777609612672 ext(5011835->18446744073709551615) 00002000:00010000:0.0:1780298644.635861:0:8362:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf243809880. 00000100:00100000:1.0:1780298644.636610:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.636649:0:7411:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609612928 00000100:00100000:2.0:1780298644.636654:0:7411:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609612928 00000100:00100000:2.0:1780298644.636689:0:7411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62038 00000100:00100000:2.0:1780298644.636701:0:7411:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f737480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+141:29360:x1866777609612928:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.636716:0:7411:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.636733:0:7411:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2400/0xaf35132e63035308 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 94 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d428f expref: -99 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.636817:0:7411:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2000/0xaf35132e6303530f lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 95 type: IBT flags: 0x40000000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.636828:0:7411:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2000/0xaf35132e6303530f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 95 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.636835:0:7411:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2000/0xaf35132e6303530f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 95 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298644.636851:0:7411:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 9206139 00000004:00200000:2.0:1780298644.636854:0:7411:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298644.636867:0:7411:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2000/0xaf35132e6303530f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 95 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.636882:0:7411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2400/0xaf35132e63035308 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 95 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d428f expref: 143 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.636897:0:7411:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2000/0xaf35132e6303530f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 94 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d428f expref: 142 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.636903:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298644.636905:0:7411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298644.636907:0:7411:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2000, rc 0) 00000100:00100000:2.0:1780298644.636934:0:7411:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f737480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+142:29360:x1866777609612928:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 238us (323us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.636940:0:7411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62038 00000100:00100000:1.0:1780298644.638088:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.638146:0:6476:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609613184 00000100:00100000:2.0:1780298644.638152:0:6476:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609613184 00000100:00100000:2.0:1780298644.638181:0:6476:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62039 00000100:00100000:2.0:1780298644.638188:0:6476:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f735f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+172:29360:x1866777609613184:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:2.0:1780298644.638338:0:6476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3800/0xaf35132e63035316 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 95 type: IBT flags: 0x40000000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.638350:0:6476:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2000/0xaf35132e6303530f lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 95 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d428f expref: 141 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.638369:0:6476:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3800/0xaf35132e63035316 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 95 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.638391:0:6476:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2000/0xaf35132e6303530f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 95 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d428f expref: 141 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.638400:0:6476:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2000/0xaf35132e6303530f lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 95 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d428f expref: 141 pid: 7411 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.638409:0:6476:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6476:1866777605036672:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298644.638455:0:6476:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb16c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.639104:0:6476:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43100 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6476:1866777605036672:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298644.639122:0:6476:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3800/0xaf35132e63035316 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 95 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.639262:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.639312:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609613312 00000100:00100000:0.0:1780298644.639318:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609613312 00010000:00010000:0.0:1780298644.639340:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2000/0xaf35132e6303530f lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 95 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d428f expref: 142 pid: 7411 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.639369:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15215 00000100:00100000:0.0:1780298644.639378:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f737800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+142:2399:x1866777609613312:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298644.639389:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298644.639395:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2000/0xaf35132e6303530f lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 96 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d428f expref: 142 pid: 7411 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.639407:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2000/0xaf35132e6303530f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 96 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d428f expref: 142 pid: 7411 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.639425:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2000/0xaf35132e6303530f lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 96 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d428f expref: 142 pid: 7411 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.639440:0:20253:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298644.639445:0:20253:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3800/0xaf35132e63035316 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 95 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.639458:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3800/0xaf35132e63035316 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 95 type: IBT flags: 0x40290000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.639465:0:20253:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3800/0xaf35132e63035316 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 95 type: IBT flags: 0x40290000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.639489:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298644.639505:0:6476:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3800/0xaf35132e63035316 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 94 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.639511:0:6476:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3800/0xaf35132e63035316 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 94 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.639535:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f737800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+141:2399:x1866777609613312:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 157us (272us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.639544:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15215 00000001:00080000:2.0:1780298644.639574:0:6476:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991092 00000004:00010000:2.0:1780298644.639621:0:6476:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000211ca07c reply state 000000008f09d418 transno 4294991093 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3800/0xaf35132e63035316 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 94 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.639632:0:6476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3800/0xaf35132e63035316 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 94 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.639638:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298644.639639:0:6476:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1780298644.639641:0:6476:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1780298644.639697:0:6476:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f735f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+175:29360:x1866777609613184:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1509us (1608us total) trans 4294991093 rc 0/0 00000100:00100000:2.0:1780298644.639706:0:6476:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62039 00000020:00080000:2.0:1780298644.639712:0:6476:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1780298644.640373:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.640408:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609613440 00000100:00100000:2.0:1780298644.640413:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609613440 00000100:00100000:2.0:1780298644.640445:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9975 00000100:00100000:2.0:1780298644.640450:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f734e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:29360:x1866777609613440:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.640459:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.640469:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e6303531d lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4296 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.640486:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e630352fa lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4281 expref: 22 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.640492:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(26 169) 00010000:00010000:2.0:1780298644.640495:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(27 169) 00010000:00010000:2.0:1780298644.640499:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e6303531d lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4296 expref: 24 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.640516:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e630352fa lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4281 expref: 22 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.640524:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e630352fa lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4281 expref: 22 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.640535:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605036800:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298644.640590:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.641169:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605036800:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298644.641185:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e6303531d lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4296 expref: 24 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.641199:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2e00, rc 0) 00000100:00100000:2.0:1780298644.641226:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f734e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+24:29360:x1866777609613440:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 774us (853us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.641233:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9975 00000100:00100000:1.0:1780298644.641437:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.641463:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609613568 00000100:00100000:3.0:1780298644.641467:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609613568 00010000:00010000:3.0:1780298644.641483:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e630352fa lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4281 expref: 23 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.641505:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15216 00000100:00100000:3.0:1780298644.641512:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348af4c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+23:2400:x1866777609613568:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298644.641521:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298644.641536:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e630352fa lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4281 expref: 23 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.641545:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e630352fa lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4281 expref: 23 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.641557:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e630352fa lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4281 expref: 23 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.641565:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(27 169) 00010000:00010000:3.0:1780298644.641567:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(27 169) 00010000:00010000:3.0:1780298644.641571:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e6303531d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4296 expref: 23 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.641580:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e6303531d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4296 expref: 23 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.641597:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e6303531d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4296 expref: 23 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.641608:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243809c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777605036928:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298644.641642:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.642467:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243809c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777605036928:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298644.642483:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:3.0:1780298644.642500:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298644.642528:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609613696 00000100:00100000:2.0:1780298644.642534:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609613696 00000100:00100000:2.0:1780298644.642580:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9976 00000100:00100000:2.0:1780298644.642587:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f736300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:29360:x1866777609613696:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298644.642596:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348af4c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2400:x1866777609613568:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1024us (1100us total) trans 0 rc 0/0 00010000:00010000:2.0:1780298644.642597:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.642611:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63035324 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d429d expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.642620:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15216 00010000:00010000:2.0:1780298644.642639:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e63035301 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4288 expref: 11 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.642646:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(22 122) 00010000:00010000:2.0:1780298644.642649:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(23 122) 00010000:00010000:2.0:1780298644.642654:0:20252:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63035324 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d429d expref: 13 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.642670:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e63035301 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4288 expref: 11 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.642678:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e63035301 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4288 expref: 11 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.642686:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605037056:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298644.642730:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.643299:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605037056:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298644.643315:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63035324 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d429d expref: 13 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.643324:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3c00, rc 0) 00000100:00100000:2.0:1780298644.643348:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f736300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:29360:x1866777609613696:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 762us (867us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.643352:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9976 00000100:00100000:1.0:1780298644.643513:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.643555:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609613824 00000100:00100000:0.0:1780298644.643562:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609613824 00010000:00010000:0.0:1780298644.643587:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e63035301 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4288 expref: 12 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.643622:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15217 00000100:00100000:0.0:1780298644.643630:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f737b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609613824:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298644.643641:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298644.643655:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e63035301 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4288 expref: 12 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.643667:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e63035301 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4288 expref: 12 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.643682:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e63035301 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4288 expref: 12 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.643694:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(23 122) 00010000:00010000:0.0:1780298644.643698:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(23 122) 00010000:00010000:0.0:1780298644.643704:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63035324 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d429d expref: 12 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.643711:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63035324 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d429d expref: 12 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.643737:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63035324 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d429d expref: 12 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.643749:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd5c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:7400:1866777605037184:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298644.643798:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.644506:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd5c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:7400:1866777605037184:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298644.644580:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298644.644697:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f737b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609613824:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1067us (1185us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298644.644703:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15217 00000100:00100000:1.0:1780298644.644712:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.644770:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609614080 00000100:00100000:0.0:1780298644.644776:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609614080 00000100:00100000:1.0:1780298644.644796:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00002000:00010000:0.0:1780298644.644803:0:19951:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4157841->18446744073709551615) 00002000:00100000:0.0:1780298644.644809:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f736680 with x1866777609614080 ext(4157841->18446744073709551615) 00002000:00010000:0.0:1780298644.644818:0:19951:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f736680. 00000100:00100000:0.0:1780298644.644844:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609613952 00000100:00100000:0.0:1780298644.644848:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609613952 00000100:00100000:3.0:1780298644.644851:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6337 00000100:00100000:3.0:1780298644.644859:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f736680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609614080:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:0.0:1780298644.644864:0:19951:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:0.0:1780298644.644868:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f734000 with x1866777609613952 ext(4194304->18446744073709551615) 00002000:00010000:0.0:1780298644.644873:0:19951:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f734000. 00000100:00100000:0.0:1780298644.644896:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6338 00000100:00100000:0.0:1780298644.644907:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f734000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+24:2399:x1866777609613952:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:3.0:1780298644.644987:0:19928:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4194304 -> 4157841 00002000:00010000:3.0:1780298644.644990:0:19928:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:3.0:1780298644.644992:0:19928:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298644 -> 1780298644 00002000:00010000:3.0:1780298644.644994:0:19928:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:3.0:1780298644.644995:0:19928:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 56 -> 56 00000100:00100000:3.0:1780298644.645026:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f736680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609614080:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 169us (319us total) trans 4294969280 rc 0/0 00000100:00100000:3.0:1780298644.645035:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6337 00002000:00010000:3.0:1780298644.645041:0:19928:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4157841->18446744073709551615) 00002000:00100000:3.0:1780298644.645045:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f736680 with x1866777609614080 ext(4157841->18446744073709551615) 00002000:00010000:3.0:1780298644.645052:0:19928:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f736680. 00002000:00010000:0.0:1780298644.645053:0:19951:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 5011835 -> 4194304 00002000:00010000:0.0:1780298644.645058:0:19951:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298644.645060:0:19951:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298644.645063:0:19951:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298644.645066:0:19951:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 384 -> 384 00000100:00100000:0.0:1780298644.645095:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f734000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+25:2399:x1866777609613952:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 189us (299us total) trans 4294969736 rc 0/0 00000100:00100000:0.0:1780298644.645106:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6338 00002000:00010000:0.0:1780298644.645110:0:19951:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:0.0:1780298644.645114:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f734000 with x1866777609613952 ext(4194304->18446744073709551615) 00002000:00010000:0.0:1780298644.645119:0:19951:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f734000. 00000100:00100000:2.0:1780298644.645934:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.645991:0:17578:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609614208 00000100:00100000:1.0:1780298644.645998:0:17578:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609614208 00000100:00100000:1.0:1780298644.646040:0:17578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62040 00000100:00100000:1.0:1780298644.646049:0:17578:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b43100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+141:29360:x1866777609614208:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.646061:0:17578:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.646074:0:17578:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279200/0xaf35132e6303532b lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 95 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d42a4 expref: -99 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.646175:0:17578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278000/0xaf35132e63035332 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 96 type: IBT flags: 0x40000000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.646189:0:17578:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278000/0xaf35132e63035332 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 96 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.646198:0:17578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278000/0xaf35132e63035332 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 96 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1780298644.646219:0:17578:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 8352145 00000004:00200000:1.0:1780298644.646222:0:17578:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1780298644.646238:0:17578:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278000/0xaf35132e63035332 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 96 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.646257:0:17578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279200/0xaf35132e6303532b lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 96 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42a4 expref: 143 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.646267:0:17578:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278000/0xaf35132e63035332 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 95 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42a4 expref: 142 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.646288:0:17578:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.646291:0:17578:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1780298644.646293:0:17578:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278000, rc 0) 00000100:00100000:1.0:1780298644.646323:0:17578:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+142:29360:x1866777609614208:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 277us (390us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.646328:0:17578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62040 00000100:00100000:1.0:1780298644.647095:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.647154:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609614336 00000100:00100000:1.0:1780298644.647154:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.647164:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609614336 00000100:00100000:3.0:1780298644.647210:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609614464 00000100:00100000:3.0:1780298644.647215:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609614464 00000100:00100000:2.0:1780298644.647216:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9977 00000100:00100000:2.0:1780298644.647229:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f736a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2399:x1866777609614336:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.647241:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298644.647246:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9978 00000100:00100000:3.0:1780298644.647256:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f734a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609614464:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.647266:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.647267:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63035339 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d42ab expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.647285:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bed20494/0xaf35132e63035340 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d42b2 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.647326:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24380b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605037312:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.647330:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605037440:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.647383:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.647383:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1a80 going to sleep for 11 seconds 00002000:00010000:3.0:1780298644.648093:0:12101:0:(ofd_lvb.c:271:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime: 1780298644 -> 1780298645 00000100:00100000:3.0:1780298644.648107:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24380b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605037312:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.648158:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bed20494/0xaf35132e63035340 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42b2 expref: 12 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.648174:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bed20494/0xaf35132e63035340 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42b2 expref: 12 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.648186:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157000, rc 0) 00000100:00100000:2.0:1780298644.648229:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605037440:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.648253:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63035339 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42ab expref: 23 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.648261:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f734a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609614464:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1003us (1132us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.648274:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9978 00010000:00010000:2.0:1780298644.648290:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63035339 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42ab expref: 23 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.648299:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3000, rc 0) 00000100:00100000:2.0:1780298644.648320:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f736a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2399:x1866777609614336:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1097us (1227us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.648326:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9977 00000100:00100000:2.0:1780298644.649317:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.649353:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.649376:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609614592 00000100:00100000:1.0:1780298644.649385:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609614592 00000100:00100000:0.0:1780298644.649412:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609614720 00000100:00100000:0.0:1780298644.649419:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609614720 00000100:00100000:1.0:1780298644.649434:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9979 00000100:00100000:1.0:1780298644.649448:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2399:x1866777609614592:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.649461:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298644.649468:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9980 00010000:00010000:1.0:1780298644.649476:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f4922155/0xaf35132e63035347 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d42b9 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.649483:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b40000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609614720:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.649504:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.649515:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e6303534e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d42c0 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.649546:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f737800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605037568:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.649556:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b41180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605037696:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.649620:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.649622:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837ae40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.650558:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f737800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605037568:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.650585:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b41180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605037696:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.650591:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f4922155/0xaf35132e63035347 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42b9 expref: 23 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.650604:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f4922155/0xaf35132e63035347 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42b9 expref: 23 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.650612:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e6303534e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42c0 expref: 12 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.650620:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278600, rc 0) 00010000:00010000:3.0:1780298644.650639:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e6303534e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42c0 expref: 12 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.650651:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3600, rc 0) 00000100:00100000:2.0:1780298644.650652:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2399:x1866777609614592:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1209us (1336us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.650659:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9979 00000100:00100000:3.0:1780298644.650683:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b40000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609614720:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1205us (1329us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.650691:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9980 00000100:00100000:0.0:1780298644.651938:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.651977:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609614848 00000100:00100000:3.0:1780298644.651983:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609614848 00000100:00100000:3.0:1780298644.652018:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9981 00000100:00100000:3.0:1780298644.652026:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:29360:x1866777609614848:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.652036:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.652051:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042c16ab0/0xaf35132e63035355 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d42c7 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.652072:0:19992:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63035324 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d429d expref: 13 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.652080:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(23 122) 00010000:00010000:3.0:1780298644.652084:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(24 122) 00010000:00010000:3.0:1780298644.652089:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042c16ab0/0xaf35132e63035355 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2674688->2682879] (req 2674688->2682879) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42c7 expref: 12 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.652110:0:19992:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63035324 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d429d expref: 13 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.652148:0:19992:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63035324 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d429d expref: 13 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.652162:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24380b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605037824:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298644.652243:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.653028:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24380b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605037824:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298644.653048:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000042c16ab0/0xaf35132e63035355 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2674688->2682879] (req 2674688->2682879) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42c7 expref: 12 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.653061:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156800, rc 0) 00000100:00100000:0.0:1780298644.653100:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:29360:x1866777609614848:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1075us (1163us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.653108:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9981 00000100:00100000:1.0:1780298644.653267:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.653316:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609614976 00000100:00100000:2.0:1780298644.653321:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609614976 00010000:00010000:2.0:1780298644.653343:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63035324 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d429d expref: 14 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.653373:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15218 00000100:00100000:2.0:1780298644.653381:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f737800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609614976:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298644.653391:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298644.653405:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63035324 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d429d expref: 14 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.653418:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63035324 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d429d expref: 14 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.653436:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63035324 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d429d expref: 14 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.653446:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(24 122) 00010000:00010000:2.0:1780298644.653450:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(24 122) 00010000:00010000:2.0:1780298644.653458:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000042c16ab0/0xaf35132e63035355 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2674688->2682879) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42c7 expref: 11 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.653468:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042c16ab0/0xaf35132e63035355 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2674688->2682879) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42c7 expref: 11 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.653490:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000042c16ab0/0xaf35132e63035355 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2674688->2682879) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42c7 expref: 11 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.653498:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b41180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6461:1866777605037952:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298644.653530:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1480 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.654245:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b41180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6461:1866777605037952:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298644.654258:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298644.654297:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f737800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609614976:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 917us (1034us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.654305:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15218 00000100:00100000:1.0:1780298644.654630:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.654673:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609615104 00000100:00100000:2.0:1780298644.654678:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609615104 00002000:00100000:2.0:1780298644.654702:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (2674688->2682879) req@ffff9cf241a80a80 x1866777609615104/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.654713:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf241a80a80 with x1866777609615104 ext(2674688->2682879) 00002000:00010000:2.0:1780298644.654717:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf241a80a80 00000100:00100000:2.0:1780298644.654742:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6339 00000100:00100000:2.0:1780298644.654751:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a80a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609615104:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298644.654907:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a80a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609615104:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 158us (280us total) trans 0 rc 8192/8192 00000100:00100000:2.0:1780298644.654912:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6339 00002000:00100000:2.0:1780298644.654916:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (2674688->2682879) req@ffff9cf241a80a80 x1866777609615104/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/8632 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 8192/8192 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.654922:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf241a80a80 with x1866777609615104 ext(2674688->2682879) 00002000:00010000:2.0:1780298644.654924:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf241a80a80 00000100:00100000:1.0:1780298644.655873:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.655911:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609615232 00000100:00100000:2.0:1780298644.655915:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609615232 00000100:00100000:2.0:1780298644.655949:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62041 00000100:00100000:2.0:1780298644.655955:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a81500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+175:29360:x1866777609615232:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.655970:0:6477:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.655983:0:6477:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3000/0xaf35132e6303535c lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 96 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d42ce expref: -99 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.656053:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2400/0xaf35132e63035363 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 97 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.656067:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2400/0xaf35132e63035363 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 97 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.656078:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2400/0xaf35132e63035363 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 97 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298644.656102:0:6477:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 8352145 00000004:00200000:2.0:1780298644.656105:0:6477:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298644.656118:0:6477:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2400/0xaf35132e63035363 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 97 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.656158:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3000/0xaf35132e6303535c lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 97 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42ce expref: 177 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.656171:0:6477:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2400/0xaf35132e63035363 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 96 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42ce expref: 176 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.656177:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298644.656180:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298644.656182:0:6477:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2400, rc 0) 00000100:00100000:2.0:1780298644.656212:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a81500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+176:29360:x1866777609615232:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 257us (343us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.656216:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62041 00000100:00100000:1.0:1780298644.657292:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.657339:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609615360 00000100:00100000:2.0:1780298644.657345:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609615360 00000100:00100000:2.0:1780298644.657382:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9982 00000100:00100000:2.0:1780298644.657388:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a82a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609615360:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.657400:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.657411:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e6303536a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d42d5 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.657428:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(24 122) 00010000:00010000:2.0:1780298644.657431:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(24 122) 00010000:00010000:2.0:1780298644.657437:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e6303536a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42d5 expref: 14 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.657448:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e6303536a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42d5 expref: 14 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.657457:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2000, rc 0) 00000100:00100000:2.0:1780298644.657493:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a82a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609615360:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 104us (203us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.657500:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9982 00000100:00100000:1.0:1780298644.658763:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.658817:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609615488 00000100:00100000:3.0:1780298644.658826:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609615488 00000100:00100000:3.0:1780298644.658872:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6340 00000100:00100000:3.0:1780298644.658883:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a81f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+25:2400:x1866777609615488:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:3.0:1780298644.659096:0:19950:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:3.0:1780298644.659101:0:19950:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:3.0:1780298644.659103:0:19950:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:3.0:1780298644.659106:0:19950:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:3.0:1780298644.659108:0:19950:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 384 -> 400 00000100:00100000:3.0:1780298644.659171:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a81f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+26:2400:x1866777609615488:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 291us (409us total) trans 4294969737 rc 0/0 00000100:00100000:3.0:1780298644.659180:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6340 00000100:00100000:1.0:1780298644.660191:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.660234:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609615616 00000100:00100000:3.0:1780298644.660239:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609615616 00000100:00100000:3.0:1780298644.660273:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9983 00000100:00100000:3.0:1780298644.660431:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a82d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2398:x1866777609615616:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.660441:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.660453:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e63035371 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d42dc expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.660508:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243809c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605038080:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.660570:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.661366:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243809c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605038080:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.661413:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e63035371 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42dc expref: 23 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.661432:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e63035371 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42dc expref: 23 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.661444:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157000, rc 0) 00000100:00100000:3.0:1780298644.661476:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a82d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2398:x1866777609615616:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1052us (1286us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.661484:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9983 00000100:00100000:3.0:1780298644.662350:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.662396:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609615744 00000100:00100000:2.0:1780298644.662406:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609615744 00000100:00100000:2.0:1780298644.662453:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9984 00000100:00100000:2.0:1780298644.662464:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243809c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2398:x1866777609615744:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.662476:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.662489:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63035378 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d42e3 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.662540:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b40380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605038208:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.662619:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb13c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.663333:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b40380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605038208:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.663362:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63035378 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42e3 expref: 23 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.663380:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63035378 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42e3 expref: 23 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.663393:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2600, rc 0) 00000100:00100000:2.0:1780298644.663431:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243809c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2398:x1866777609615744:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 971us (1081us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.663439:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9984 00000100:00100000:1.0:1780298644.664911:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.664974:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609615872 00000100:00100000:0.0:1780298644.664982:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609615872 00000100:00100000:0.0:1780298644.665036:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9985 00000100:00100000:0.0:1780298644.665046:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a82300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2397:x1866777609615872:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.665057:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.665068:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e6303537f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d42ea expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.665104:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605038336:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.665209:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.665851:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605038336:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.665870:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e6303537f lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42ea expref: 23 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.665885:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e6303537f lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42ea expref: 23 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.665896:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae000, rc 0) 00000100:00100000:0.0:1780298644.665927:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a82300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2397:x1866777609615872:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 883us (1018us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.665934:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9985 00000100:00100000:1.0:1780298644.666768:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.666804:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609616000 00000100:00100000:2.0:1780298644.666809:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609616000 00000100:00100000:2.0:1780298644.666839:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9986 00000100:00100000:2.0:1780298644.666846:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a82680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2397:x1866777609616000:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.666853:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.666861:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e63035386 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d42f1 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.666892:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b40380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605038464:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.666941:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb13c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.667654:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b40380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605038464:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.667674:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e63035386 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42f1 expref: 23 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.667690:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e63035386 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42f1 expref: 23 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.667700:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3c00, rc 0) 00000100:00100000:2.0:1780298644.667733:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a82680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2397:x1866777609616000:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 887us (965us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.667741:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9986 00000100:00100000:3.0:1780298644.669041:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.669090:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609616128 00000100:00100000:0.0:1780298644.669096:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609616128 00002000:00100000:0.0:1780298644.669147:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (1404928->1409023) req@ffff9cf243808380 x1866777609616128/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298644.669160:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243808380 with x1866777609616128 ext(1404928->1409023) 00002000:00010000:0.0:1780298644.669165:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf243808380 00000100:00100000:0.0:1780298644.669186:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6341 00000100:00100000:0.0:1780298644.669195:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243808380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2397:x1866777609616128:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298644.669386:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243808380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2397:x1866777609616128:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 192us (346us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298644.669394:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6341 00002000:00100000:0.0:1780298644.669402:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (1404928->1409023) req@ffff9cf243808380 x1866777609616128/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/4536 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298644.669412:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243808380 with x1866777609616128 ext(1404928->1409023) 00002000:00010000:0.0:1780298644.669416:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf243808380 00000100:00100000:1.0:1780298644.670335:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.670416:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609616256 00000100:00100000:3.0:1780298644.670423:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609616256 00002000:00100000:3.0:1780298644.670457:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x12:0x0] (1409024->1413119) req@ffff9cf241a81880 x1866777609616256/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298644.670467:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf241a81880 with x1866777609616256 ext(1409024->1413119) 00002000:00010000:3.0:1780298644.670471:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf241a81880 00000100:00100000:3.0:1780298644.670496:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6342 00000100:00100000:3.0:1780298644.670505:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a81880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2397:x1866777609616256:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298644.670623:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a81880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2397:x1866777609616256:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 119us (289us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1780298644.670630:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6342 00002000:00100000:3.0:1780298644.670636:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x12:0x0] (1409024->1413119) req@ffff9cf241a81880 x1866777609616256/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/4536 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298644.670645:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf241a81880 with x1866777609616256 ext(1409024->1413119) 00002000:00010000:3.0:1780298644.670649:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf241a81880 00000100:00100000:1.0:1780298644.671552:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.671607:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609616384 00000100:00100000:3.0:1780298644.671613:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609616384 00000100:00100000:3.0:1780298644.671652:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9987 00000100:00100000:3.0:1780298644.671662:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a83b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2397:x1866777609616384:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.671672:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.671684:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009e0b8e7b/0xaf35132e6303538d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d42f8 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.671727:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24380b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605038592:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.671789:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.672520:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24380b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605038592:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.672575:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009e0b8e7b/0xaf35132e6303538d lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42f8 expref: 23 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.672598:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009e0b8e7b/0xaf35132e6303538d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42f8 expref: 23 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.672617:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156c00, rc 0) 00000100:00100000:3.0:1780298644.672666:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a83b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2397:x1866777609616384:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1005us (1115us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.672676:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9987 00000100:00100000:1.0:1780298644.673455:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.673499:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609616512 00000100:00100000:2.0:1780298644.673506:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609616512 00000100:00100000:2.0:1780298644.673543:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9988 00000100:00100000:2.0:1780298644.673553:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2397:x1866777609616512:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.673563:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.673572:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63035394 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d42ff expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.673612:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b40380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605038720:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.673665:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb13c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.674696:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b40380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605038720:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.674716:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63035394 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42ff expref: 23 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.674727:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63035394 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42ff expref: 23 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.674734:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3000, rc 0) 00000100:00100000:2.0:1780298644.674760:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2397:x1866777609616512:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1211us (1308us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.674766:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9988 00000100:00100000:3.0:1780298644.675756:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.675810:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609616640 00000100:00100000:0.0:1780298644.675821:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609616640 00000100:00100000:0.0:1780298644.675870:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9989 00000100:00100000:0.0:1780298644.675881:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24380b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:29360:x1866777609616640:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.675892:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.675906:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a3f03a/0xaf35132e6303539b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4306 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.675929:0:19992:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000042c16ab0/0xaf35132e63035355 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2674688->2682879) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42c7 expref: 11 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.675940:0:19992:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e6303536a lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42d5 expref: 15 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.675948:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(24 122) 00010000:00010000:0.0:1780298644.675953:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(24 122) 00010000:00010000:0.0:1780298644.675958:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a3f03a/0xaf35132e6303539b lrc: 5/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1388544->1392639] (req 1388544->1392639) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4306 expref: 15 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.675984:0:19992:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e6303536a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42d5 expref: 15 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.675999:0:19992:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e6303536a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42d5 expref: 15 pid: 9128 timeout: 934 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.676010:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605038848:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298644.676075:0:19992:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000042c16ab0/0xaf35132e63035355 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2674688->2682879) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42c7 expref: 11 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.676087:0:19992:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000042c16ab0/0xaf35132e63035355 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2674688->2682879) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42c7 expref: 11 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.676099:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605038976:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298644.676191:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733936c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.676837:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605038848:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298644.676850:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605038976:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298644.676874:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000001a3f03a/0xaf35132e6303539b lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1388544->1392639] (req 1388544->1392639) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4306 expref: 15 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.676889:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae600, rc 0) 00000100:00100000:0.0:1780298644.676925:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24380b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:29360:x1866777609616640:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1046us (1170us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.676933:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9989 00000100:00100000:2.0:1780298644.677199:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.677248:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609616768 00000100:00100000:1.0:1780298644.677256:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609616768 00000100:00100000:2.0:1780298644.677260:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.677293:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609616896 00010000:00010000:1.0:1780298644.677302:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e6303536a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42d5 expref: 15 pid: 9128 timeout: 934 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.677330:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609616896 00000100:00100000:1.0:1780298644.677339:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15219 00010000:00010000:3.0:1780298644.677347:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000042c16ab0/0xaf35132e63035355 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2674688->2682879) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42c7 expref: 12 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.677355:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b40380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2400:x1866777609616768:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298644.677369:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1780298644.677375:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15220 00000100:00100000:3.0:1780298644.677382:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b42300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609616896:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298644.677388:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e6303536a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42d5 expref: 15 pid: 9128 timeout: 934 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.677393:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298644.677406:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e6303536a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42d5 expref: 15 pid: 9128 timeout: 934 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.677420:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000042c16ab0/0xaf35132e63035355 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2674688->2682879) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42c7 expref: 12 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.677429:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e6303536a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42d5 expref: 15 pid: 9128 timeout: 934 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.677438:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000042c16ab0/0xaf35132e63035355 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 2674688->2682879) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42c7 expref: 12 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.677460:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042c16ab0/0xaf35132e63035355 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2674688->2682879) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42c7 expref: 12 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.677462:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(24 122) 00010000:00010000:1.0:1780298644.677472:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(24 122) 00010000:00010000:1.0:1780298644.677482:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000001a3f03a/0xaf35132e6303539b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1388544->34942975] (req 1388544->1392639) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4306 expref: 14 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.677498:0:6463:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000001a3f03a/0xaf35132e6303539b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1388544->34942975] (req 1388544->1392639) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4306 expref: 14 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.677509:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298644.677534:0:6463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000001a3f03a/0xaf35132e6303539b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1388544->34942975] (req 1388544->1392639) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4306 expref: 14 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.677545:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b42300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609616896:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 164us (288us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.677550:0:6463:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fd500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777605039104:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298644.677555:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15220 00000100:00100000:1.0:1780298644.677627:0:6463:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837af00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.678833:0:6463:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fd500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777605039104:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298644.678851:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298644.678883:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b40380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609616768:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1535us (1687us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.678892:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15219 00000100:00100000:2.0:1780298644.679092:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.679167:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609617024 00000100:00100000:0.0:1780298644.679175:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609617024 00002000:00100000:0.0:1780298644.679217:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (1388544->1392639) req@ffff9cf242b42a00 x1866777609617024/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298644.679227:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b42a00 with x1866777609617024 ext(1388544->1392639) 00002000:00010000:0.0:1780298644.679231:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf242b42a00 00000100:00100000:0.0:1780298644.679264:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6343 00000100:00100000:0.0:1780298644.679274:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2397:x1866777609617024:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298644.679466:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2397:x1866777609617024:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 193us (373us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298644.679475:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6343 00002000:00100000:0.0:1780298644.679484:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (1388544->1392639) req@ffff9cf242b42a00 x1866777609617024/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/4536 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298644.679493:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b42a00 with x1866777609617024 ext(1388544->1392639) 00002000:00010000:0.0:1780298644.679496:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf242b42a00 00000100:00100000:1.0:1780298644.680679:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.680720:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609617152 00000100:00100000:2.0:1780298644.680726:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609617152 00002000:00100000:2.0:1780298644.680759:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (1388544->1392639) req@ffff9cf3481ff480 x1866777609617152/t0(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 4584/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.680769:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481ff480 with x1866777609617152 ext(1388544->1392639) 00002000:00010000:2.0:1780298644.680775:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf3481ff480 00000100:00100000:2.0:1780298644.680803:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6344 00000100:00100000:2.0:1780298644.680812:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481ff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2397:x1866777609617152:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:2.0:1780298644.681336:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481ff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609617152:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 526us (659us total) trans 4294969281 rc 0/0 00000100:00100000:2.0:1780298644.681341:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6344 00002000:00100000:2.0:1780298644.681346:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (1388544->1392639) req@ffff9cf3481ff480 x1866777609617152/t4294969281(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 4584/448 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.681352:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481ff480 with x1866777609617152 ext(1388544->1392639) 00002000:00010000:2.0:1780298644.681359:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf3481ff480 00000100:00100000:1.0:1780298644.682181:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.682232:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609617280 00000100:00100000:2.0:1780298644.682237:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609617280 00000100:00100000:2.0:1780298644.682269:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9990 00000100:00100000:2.0:1780298644.682287:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2398:x1866777609617280:12345-192.168.203.25@tcp:16:fsx.0 00000001:00080000:1.0:1780298644.683088:0:8351:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0000: transno 4294969281 is committed 00000001:00080000:1.0:1780298644.683101:0:8351:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0000: transno 4294969279 is committed 00000100:00100000:2.0:1780298644.683213:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609617280:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 936us (1030us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.683221:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9990 00000100:00100000:3.0:1780298644.684202:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.684267:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609617408 00000100:00100000:0.0:1780298644.684287:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609617408 00000100:00100000:0.0:1780298644.684323:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9991 00000100:00100000:0.0:1780298644.684342:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481faa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609617408:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.684353:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.684369:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e630353a2 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d430d expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.684426:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605039232:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.684488:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733936c0 going to sleep for 11 seconds 00002000:00010000:0.0:1780298644.685267:0:8358:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks: 56 -> 64 00000100:00100000:0.0:1780298644.685292:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605039232:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.685316:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e630353a2 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d430d expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.685336:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e630353a2 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d430d expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.685347:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae000, rc 0) 00000100:00100000:0.0:1780298644.685377:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481faa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609617408:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1047us (1177us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.685384:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9991 00000100:00100000:1.0:1780298644.686339:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.686388:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609617536 00000100:00100000:2.0:1780298644.686393:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609617536 00000100:00100000:2.0:1780298644.686426:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9992 00000100:00100000:2.0:1780298644.686435:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609617536:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.686445:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.686455:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e630353a9 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4314 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.686497:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b41c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605039360:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.686553:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb13c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.687268:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b41c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605039360:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.687297:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e630353a9 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4314 expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.687309:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e630353a9 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4314 expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.687316:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3000, rc 0) 00000100:00100000:2.0:1780298644.687348:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609617536:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 915us (1011us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.687355:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9992 00000100:00100000:1.0:1780298644.688460:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.688506:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609617664 00000100:00100000:0.0:1780298644.688512:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609617664 00000100:00100000:0.0:1780298644.688548:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9993 00000100:00100000:0.0:1780298644.688557:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e4f2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:29360:x1866777609617664:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.688568:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.688578:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e630353b0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d431b expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.688597:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(24 122) 00010000:00010000:0.0:1780298644.688602:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(24 122) 00010000:00010000:0.0:1780298644.688611:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e630353b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [430080->1388543] (req 430080->434175) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d431b expref: 7 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.688626:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e630353b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [430080->1388543] (req 430080->434175) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d431b expref: 7 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.688648:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baea00, rc 0) 00000100:00100000:0.0:1780298644.688680:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e4f2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609617664:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 124us (223us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.688686:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9993 00000100:00100000:1.0:1780298644.689538:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.689575:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609617792 00000100:00100000:3.0:1780298644.689589:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609617792 00002000:00100000:3.0:1780298644.689624:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (430080->434175) req@ffff9cf3481fc700 x1866777609617792/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298644.689635:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fc700 with x1866777609617792 ext(430080->434175) 00002000:00010000:3.0:1780298644.689640:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf3481fc700 00000100:00100000:3.0:1780298644.689660:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6345 00000100:00100000:3.0:1780298644.689668:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609617792:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298644.689814:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609617792:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 146us (275us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1780298644.689822:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6345 00002000:00100000:3.0:1780298644.689830:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (430080->434175) req@ffff9cf3481fc700 x1866777609617792/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/4536 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298644.689839:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fc700 with x1866777609617792 ext(430080->434175) 00002000:00010000:3.0:1780298644.689843:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf3481fc700 00000100:00100000:3.0:1780298644.691227:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.691300:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609617920 00000100:00100000:2.0:1780298644.691307:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609617920 00000100:00100000:2.0:1780298644.691346:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9994 00000100:00100000:2.0:1780298644.691354:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609617920:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.691366:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.691379:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e630353b7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4322 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.691428:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b41c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605039488:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.691490:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb13c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.692175:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b41c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605039488:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.692196:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e630353b7 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4322 expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.692211:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e630353b7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4322 expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.692221:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3c00, rc 0) 00000100:00100000:2.0:1780298644.692251:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609617920:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 899us (1027us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.692258:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9994 00000100:00100000:1.0:1780298644.693093:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.693151:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609618048 00000100:00100000:0.0:1780298644.693156:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609618048 00000100:00100000:0.0:1780298644.693188:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9995 00000100:00100000:0.0:1780298644.693197:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481ff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609618048:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.693206:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.693215:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e630353be lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4329 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.693245:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605039616:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.693303:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733936c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.693889:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605039616:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.693906:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e630353be lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4329 expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.693920:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e630353be lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4329 expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.693929:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae000, rc 0) 00000100:00100000:0.0:1780298644.693966:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481ff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609618048:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 764us (866us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.693978:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9995 00000100:00100000:1.0:1780298644.694831:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.694865:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609618176 00000100:00100000:2.0:1780298644.694870:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609618176 00000100:00100000:2.0:1780298644.694898:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9996 00000100:00100000:2.0:1780298644.694906:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609618176:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.694915:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.694924:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e630353c5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4330 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.694952:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b41c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605039744:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.694989:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb13c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.695703:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b41c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605039744:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.695722:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e630353c5 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4330 expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.695737:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e630353c5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4330 expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.695745:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2000, rc 0) 00000100:00100000:2.0:1780298644.695768:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609618176:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 864us (937us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.695774:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9996 00000100:00100000:3.0:1780298644.696534:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.696564:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609618304 00000100:00100000:0.0:1780298644.696568:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609618304 00000100:00100000:0.0:1780298644.696600:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9997 00000100:00100000:0.0:1780298644.696607:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609618304:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.696614:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.696624:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b35ce980/0xaf35132e630353cc lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4337 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.696658:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605039872:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.696710:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733936c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.697400:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605039872:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.697414:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b35ce980/0xaf35132e630353cc lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4337 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.697423:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b35ce980/0xaf35132e630353cc lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4337 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.697430:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bafa00, rc 0) 00000100:00100000:0.0:1780298644.697453:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609618304:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 848us (920us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.697458:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9997 00000100:00100000:1.0:1780298644.698270:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.698307:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609618432 00000100:00100000:0.0:1780298644.698312:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609618432 00000100:00100000:0.0:1780298644.698356:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6346 00000100:00100000:0.0:1780298644.698366:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609618432:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298644.698504:0:8362:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4157841 -> 4157841 00002000:00010000:0.0:1780298644.698507:0:8362:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298644.698508:0:8362:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298644.698510:0:8362:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298644.698511:0:8362:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 64 -> 64 00000100:00100000:0.0:1780298644.698536:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609618432:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 173us (266us total) trans 4294969282 rc 0/0 00000100:00100000:0.0:1780298644.698543:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6346 00000100:00100000:3.0:1780298644.699417:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.699449:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609618560 00000100:00100000:2.0:1780298644.699453:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609618560 00000100:00100000:2.0:1780298644.699480:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9998 00000100:00100000:2.0:1780298644.699488:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2397:x1866777609618560:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.699497:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.699511:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e630353d3 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d433e expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.699523:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.699558:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b41c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605040000:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.699559:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609618688 00000100:00100000:0.0:1780298644.699564:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609618688 00000100:00100000:0.0:1780298644.699594:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 9999 00000100:00100000:0.0:1780298644.699601:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609618688:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.699608:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.699616:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e630353da lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4345 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.699622:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb13c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.699640:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605040128:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.699661:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733936c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.700302:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b41c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605040000:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.700327:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e630353d3 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d433e expref: 23 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.700344:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e630353d3 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d433e expref: 23 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.700360:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3000, rc 0) 00000100:00100000:2.0:1780298644.700390:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2397:x1866777609618560:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 902us (973us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.700399:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9998 00000100:00100000:0.0:1780298644.700421:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605040128:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.700446:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e630353da lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4345 expref: 8 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.700457:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e630353da lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4345 expref: 8 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.700466:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf200, rc 0) 00000100:00100000:0.0:1780298644.700488:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609618688:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 889us (969us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.700492:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 9999 00000100:00100000:1.0:1780298644.701334:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.701367:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609618816 00000100:00100000:2.0:1780298644.701373:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609618816 00000100:00100000:2.0:1780298644.701404:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10000 00000100:00100000:2.0:1780298644.701412:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481ffb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2398:x1866777609618816:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.701421:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.701431:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e630353e1 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d434c expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.701452:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.701460:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b41c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605040256:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.701501:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609618944 00000100:00100000:0.0:1780298644.701505:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609618944 00000100:00100000:2.0:1780298644.701507:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb13c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.701538:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10001 00000100:00100000:0.0:1780298644.701553:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609618944:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.701561:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.701569:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e630353e8 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4353 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.701607:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605040384:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.701639:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733936c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.702152:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b41c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605040256:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.702172:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e630353e1 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d434c expref: 23 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.702187:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e630353e1 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d434c expref: 23 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.702198:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3600, rc 0) 00000100:00100000:2.0:1780298644.702225:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481ffb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2398:x1866777609618816:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 813us (889us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.702232:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10000 00000100:00100000:0.0:1780298644.702389:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605040384:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.702403:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e630353e8 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4353 expref: 8 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.702413:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e630353e8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4353 expref: 8 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.702421:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae000, rc 0) 00000100:00100000:0.0:1780298644.702452:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609618944:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 896us (992us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.702460:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10001 00000100:00100000:3.0:1780298644.703698:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.703743:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609619072 00000100:00100000:1.0:1780298644.703747:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609619072 00000100:00100000:1.0:1780298644.703776:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10002 00000100:00100000:1.0:1780298644.703781:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609619072:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.703789:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.703797:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f4922155/0xaf35132e630353ef lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d435a expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.703824:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605040512:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.703875:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837af00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.704580:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605040512:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.704604:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f4922155/0xaf35132e630353ef lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d435a expref: 12 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.704621:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f4922155/0xaf35132e630353ef lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d435a expref: 12 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.704632:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278600, rc 0) 00000100:00100000:1.0:1780298644.704663:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609619072:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 881us (966us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.704672:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10002 00000100:00100000:2.0:1780298644.705978:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.706022:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609619200 00000100:00100000:3.0:1780298644.706028:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609619200 00002000:00100000:3.0:1780298644.706062:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x12:0x0] (1507328->1572863) req@ffff9cf242b41c00 x1866777609619200/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298644.706072:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b41c00 with x1866777609619200 ext(1507328->1572863) 00002000:00010000:3.0:1780298644.706077:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf242b41c00 00000100:00100000:3.0:1780298644.706097:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6347 00000100:00100000:3.0:1780298644.706106:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b41c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609619200:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298644.707306:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b41c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609619200:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 1200us (1328us total) trans 0 rc 65536/65536 00000100:00100000:1.0:1780298644.707316:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6347 00002000:00100000:1.0:1780298644.707323:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x12:0x0] (1507328->1572863) req@ffff9cf242b41c00 x1866777609619200/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/440 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 65536/65536 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.707333:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b41c00 with x1866777609619200 ext(1507328->1572863) 00002000:00010000:1.0:1780298644.707337:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf242b41c00 00000100:00100000:2.0:1780298644.708715:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.708770:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609619328 00000100:00100000:0.0:1780298644.708778:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609619328 00002000:00100000:0.0:1780298644.708805:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (1536000->1544191) req@ffff9cf242b43800 x1866777609619328/t0(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 8680/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298644.708815:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b43800 with x1866777609619328 ext(1536000->1544191) 00002000:00010000:0.0:1780298644.708827:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf242b43800 00000100:00100000:0.0:1780298644.708877:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6348 00000100:00100000:0.0:1780298644.708885:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b43800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609619328:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:0.0:1780298644.709411:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609619328:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 526us (697us total) trans 4294969283 rc 0/0 00000100:00100000:0.0:1780298644.709418:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6348 00002000:00100000:0.0:1780298644.709425:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (1536000->1544191) req@ffff9cf242b43800 x1866777609619328/t4294969283(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 8680/448 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298644.709433:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b43800 with x1866777609619328 ext(1536000->1544191) 00002000:00010000:0.0:1780298644.709437:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf242b43800 00000100:00100000:2.0:1780298644.710293:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.710336:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609619456 00000100:00100000:0.0:1780298644.710341:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609619456 00000100:00100000:0.0:1780298644.710388:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10003 00000100:00100000:0.0:1780298644.710399:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609619456:12345-192.168.203.25@tcp:16:fsx.0 00000001:00080000:1.0:1780298644.711088:0:8351:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0000: transno 4294969283 is committed 00000100:00100000:0.0:1780298644.711190:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609619456:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 793us (896us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.711199:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10003 00000100:00100000:2.0:1780298644.712238:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.712398:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609619584 00000100:00100000:1.0:1780298644.712403:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609619584 00000100:00100000:1.0:1780298644.712458:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10004 00000100:00100000:1.0:1780298644.712468:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2397:x1866777609619584:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.712480:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298644.712494:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:1.0:1780298644.712500:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f4922155/0xaf35132e630353f6 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4361 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.712541:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609619712 00000100:00100000:0.0:1780298644.712546:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609619712 00000100:00100000:1.0:1780298644.712554:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605040640:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.712584:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10005 00000100:00100000:0.0:1780298644.712596:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609619712:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.712606:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298644.712607:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837af00 going to sleep for 11 seconds 00010000:00010000:0.0:1780298644.712618:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e630353fd lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4368 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.712661:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605040768:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.712695:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733936c0 going to sleep for 11 seconds 00002000:00010000:0.0:1780298644.713566:0:19992:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks: 64 -> 88 00000100:00100000:0.0:1780298644.713592:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605040768:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.713618:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e630353fd lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4368 expref: 8 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.713641:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e630353fd lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4368 expref: 8 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.713646:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605040640:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.713665:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf200, rc 0) 00010000:00010000:1.0:1780298644.713666:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f4922155/0xaf35132e630353f6 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4361 expref: 23 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.713678:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f4922155/0xaf35132e630353f6 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4361 expref: 23 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.713686:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278600, rc 0) 00000100:00100000:0.0:1780298644.713705:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609619712:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1111us (1211us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.713706:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2397:x1866777609619584:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1240us (1469us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.713721:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10004 00000100:00100000:0.0:1780298644.713730:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10005 00000100:00100000:2.0:1780298644.714679:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.714714:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609619840 00000100:00100000:1.0:1780298644.714721:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609619840 00000100:00100000:1.0:1780298644.714755:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10006 00000100:00100000:1.0:1780298644.714763:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b40000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2400:x1866777609619840:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.714773:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.714783:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63035404 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d436f expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.714817:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605040896:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.714825:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.714866:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609619968 00000100:00100000:0.0:1780298644.714871:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609619968 00000100:00100000:1.0:1780298644.714873:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837af00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.714909:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10007 00000100:00100000:0.0:1780298644.714919:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b41180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609619968:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.714927:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.714937:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e6303540b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4376 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.714964:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605041024:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.715004:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733936c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.715711:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605040896:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.715716:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605041024:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.715735:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e6303540b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4376 expref: 8 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.715737:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63035404 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d436f expref: 23 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.715758:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e6303540b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4376 expref: 8 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.715761:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63035404 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d436f expref: 23 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.715773:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae000, rc 0) 00010000:00010000:1.0:1780298644.715775:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279200, rc 0) 00000100:00100000:0.0:1780298644.715799:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b41180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609619968:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 882us (973us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.715800:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b40000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2400:x1866777609619840:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1037us (1122us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.715811:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10007 00000100:00100000:1.0:1780298644.715819:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10006 00000100:00100000:2.0:1780298644.716846:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.716879:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609620096 00000100:00100000:3.0:1780298644.716885:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609620096 00000100:00100000:3.0:1780298644.716916:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10008 00000100:00100000:3.0:1780298644.716924:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243845180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609620096:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.716933:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.716942:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042c16ab0/0xaf35132e63035412 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d437d expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.716976:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605041152:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.717020:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.717629:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605041152:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.717667:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000042c16ab0/0xaf35132e63035412 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d437d expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.717682:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042c16ab0/0xaf35132e63035412 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d437d expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.717692:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156800, rc 0) 00000100:00100000:2.0:1780298644.717722:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243845180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609620096:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 800us (876us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.717730:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10008 00000100:00100000:2.0:1780298644.718756:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.718789:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609620224 00000100:00100000:0.0:1780298644.718793:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609620224 00002000:00100000:0.0:1780298644.718817:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (98304->102399) req@ffff9cf243845f80 x1866777609620224/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298644.718825:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243845f80 with x1866777609620224 ext(98304->102399) 00002000:00010000:0.0:1780298644.718829:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf243845f80 00000100:00100000:0.0:1780298644.718846:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6349 00000100:00100000:0.0:1780298644.718850:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243845f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+26:2399:x1866777609620224:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298644.718955:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243845f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+26:2399:x1866777609620224:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 104us (200us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298644.718962:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6349 00002000:00100000:0.0:1780298644.718968:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (98304->102399) req@ffff9cf243845f80 x1866777609620224/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/4536 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298644.718977:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243845f80 with x1866777609620224 ext(98304->102399) 00002000:00010000:0.0:1780298644.718981:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf243845f80 00000100:00100000:0.0:1780298644.720205:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.720239:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609620352 00000100:00100000:3.0:1780298644.720244:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609620352 00002000:00100000:3.0:1780298644.720265:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (98304->102399) req@ffff9cf347cd5880 x1866777609620352/t0(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 4584/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298644.720286:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf347cd5880 with x1866777609620352 ext(98304->102399) 00002000:00010000:3.0:1780298644.720291:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf347cd5880 00000100:00100000:3.0:1780298644.720309:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6350 00000100:00100000:3.0:1780298644.720316:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+26:2400:x1866777609620352:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:3.0:1780298644.720713:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+28:2400:x1866777609620352:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 396us (508us total) trans 4294969738 rc 0/0 00000100:00100000:3.0:1780298644.720721:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6350 00002000:00100000:3.0:1780298644.720727:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (98304->102399) req@ffff9cf347cd5880 x1866777609620352/t4294969738(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 4584/448 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298644.720737:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf347cd5880 with x1866777609620352 ext(98304->102399) 00002000:00010000:3.0:1780298644.720740:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf347cd5880 00000100:00100000:3.0:1780298644.721469:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.721509:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609620480 00000100:00100000:0.0:1780298644.721514:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609620480 00000100:00100000:0.0:1780298644.721546:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10009 00000100:00100000:0.0:1780298644.721552:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+28:2398:x1866777609620480:12345-192.168.203.25@tcp:16:fsx.0 00000001:00080000:1.0:1780298644.722436:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969738 is committed 00000001:00080000:1.0:1780298644.722463:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969735 is committed 00000100:00100000:0.0:1780298644.722599:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609620480:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 1045us (1129us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.722614:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10009 00000100:00100000:2.0:1780298644.723708:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.723751:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609620608 00000100:00100000:1.0:1780298644.723757:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609620608 00000100:00100000:1.0:1780298644.723797:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10010 00000100:00100000:1.0:1780298644.723807:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243846d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609620608:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.723820:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.723835:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63035419 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4384 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.723858:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.723887:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605041280:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.723897:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609620736 00000100:00100000:0.0:1780298644.723903:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609620736 00000100:00100000:1.0:1780298644.723938:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837af00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.723943:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10011 00000100:00100000:0.0:1780298644.723953:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609620736:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.723962:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.723971:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63035420 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d438b expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.723996:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605041408:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.724024:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.724616:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605041280:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.724642:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63035419 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4384 expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.724661:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63035419 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4384 expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.724673:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279200, rc 0) 00000100:00100000:1.0:1780298644.724704:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243846d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609620608:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 900us (996us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.724711:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10010 00000100:00100000:0.0:1780298644.724782:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605041408:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.724797:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63035420 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d438b expref: 8 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.724809:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63035420 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d438b expref: 8 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.724819:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae000, rc 0) 00000100:00100000:0.0:1780298644.724849:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609620736:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 898us (991us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.724857:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10011 00000100:00100000:3.0:1780298644.725823:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.725839:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.725860:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609620864 00000100:00100000:2.0:1780298644.725864:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609620864 00000100:00100000:1.0:1780298644.725874:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609620992 00000100:00100000:1.0:1780298644.725880:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609620992 00000100:00100000:2.0:1780298644.725892:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10012 00000100:00100000:2.0:1780298644.725899:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609620864:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.725906:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298644.725914:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10013 00010000:00010000:2.0:1780298644.725915:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e63035427 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4392 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.725926:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481faa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609620992:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.725933:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.725941:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e6303542e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4399 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.725943:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243846a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605041536:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.725973:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605041664:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.726004:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb13c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.726005:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.726653:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243846a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605041536:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.726663:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605041664:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.726671:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e63035427 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4392 expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.726680:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e6303542e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4399 expref: 8 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.726684:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e63035427 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4392 expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.726698:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3e00, rc 0) 00010000:00010000:0.0:1780298644.726714:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e6303542e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4399 expref: 8 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.726722:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf200, rc 0) 00000100:00100000:2.0:1780298644.726726:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609620864:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 827us (901us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.726731:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10012 00000100:00100000:0.0:1780298644.726747:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481faa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609620992:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 823us (904us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.726752:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10013 00000100:00100000:3.0:1780298644.727642:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.727680:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609621120 00000100:00100000:1.0:1780298644.727686:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609621120 00000100:00100000:1.0:1780298644.727722:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10014 00000100:00100000:1.0:1780298644.727731:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609621120:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.727742:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.727751:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f4922155/0xaf35132e63035435 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d43a0 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.727772:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000001a3f03a/0xaf35132e6303539b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1388544->34942975] (req 1388544->1392639) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4306 expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.727781:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(24 122) 00010000:00010000:1.0:1780298644.727786:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(25 122) 00010000:00010000:1.0:1780298644.727791:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f4922155/0xaf35132e63035435 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1896448->1904639] (req 1896448->1904639) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d43a0 expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.727812:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000001a3f03a/0xaf35132e6303539b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1388544->34942975] (req 1388544->1392639) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4306 expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.727836:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000001a3f03a/0xaf35132e6303539b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1388544->34942975] (req 1388544->1392639) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4306 expref: 9 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.727848:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605041792:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298644.727888:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837af00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.728491:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605041792:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298644.728508:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f4922155/0xaf35132e63035435 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1896448->1904639] (req 1896448->1904639) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d43a0 expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.728520:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278600, rc 0) 00000100:00100000:1.0:1780298644.728549:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:29360:x1866777609621120:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 820us (906us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.728557:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10014 00000100:00100000:3.0:1780298644.728780:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.728820:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609621248 00000100:00100000:0.0:1780298644.728825:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609621248 00010000:00010000:0.0:1780298644.728849:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000001a3f03a/0xaf35132e6303539b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1388544->34942975] (req 1388544->1392639) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4306 expref: 10 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.728877:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15221 00000100:00100000:0.0:1780298644.728885:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343966d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609621248:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298644.728895:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298644.728907:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000001a3f03a/0xaf35132e6303539b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [1388544->34942975] (req 1388544->1392639) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4306 expref: 10 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.728920:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000001a3f03a/0xaf35132e6303539b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [1388544->34942975] (req 1388544->1392639) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4306 expref: 10 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.728937:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a3f03a/0xaf35132e6303539b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [1388544->34942975] (req 1388544->1392639) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4306 expref: 10 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.728951:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(25 122) 00010000:00010000:0.0:1780298644.728955:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(25 122) 00010000:00010000:0.0:1780298644.728965:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000f4922155/0xaf35132e63035435 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1896448->35450879] (req 1896448->1904639) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d43a0 expref: 7 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.728975:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f4922155/0xaf35132e63035435 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1896448->35450879] (req 1896448->1904639) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d43a0 expref: 7 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.729000:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000f4922155/0xaf35132e63035435 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1896448->35450879] (req 1896448->1904639) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d43a0 expref: 7 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.729011:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd4e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:20253:1866777605041920:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298644.729046:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.729717:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd4e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:20253:1866777605041920:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298644.729728:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298644.729755:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343966d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609621248:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 871us (974us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.729762:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15221 00000100:00100000:3.0:1780298644.729863:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.729898:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609621376 00000100:00100000:0.0:1780298644.729902:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609621376 00000100:00100000:0.0:1780298644.729940:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6351 00000100:00100000:0.0:1780298644.729948:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343966a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609621376:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298644.730109:0:8362:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4157841 -> 4157841 00002000:00010000:0.0:1780298644.730113:0:8362:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298644.730117:0:8362:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298644.730119:0:8362:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298644.730121:0:8362:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 88 -> 88 00000100:00100000:0.0:1780298644.730171:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343966a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609621376:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 225us (338us total) trans 4294969284 rc 0/0 00000100:00100000:0.0:1780298644.730180:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6351 00000100:00100000:2.0:1780298644.731023:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.731048:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.731061:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609621504 00000100:00100000:0.0:1780298644.731066:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609621504 00000100:00100000:0.0:1780298644.731092:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609621632 00000100:00100000:0.0:1780298644.731093:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609621632 00000100:00100000:1.0:1780298644.731109:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10015 00000100:00100000:0.0:1780298644.731117:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10016 00000100:00100000:1.0:1780298644.731145:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243846a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609621504:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298644.731146:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243847480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609621632:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.731158:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.731161:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.731167:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b35ce980/0xaf35132e6303543c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d43ae expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.731173:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e63035443 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d43a7 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.731198:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605042048:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.731215:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605042176:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.731246:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.731250:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837af00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.731907:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605042048:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.731923:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b35ce980/0xaf35132e6303543c lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d43ae expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.731933:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605042176:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.731935:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b35ce980/0xaf35132e6303543c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d43ae expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.731947:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bafa00, rc 0) 00010000:00010000:1.0:1780298644.731959:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e63035443 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d43a7 expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.731972:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243847480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609621632:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 847us (924us total) trans 0 rc 301/301 00010000:00010000:1.0:1780298644.731976:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e63035443 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d43a7 expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.731982:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10016 00010000:00010000:1.0:1780298644.731994:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279800, rc 0) 00000100:00100000:1.0:1780298644.732020:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243846a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609621504:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 903us (997us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.732028:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10015 00000100:00100000:3.0:1780298644.732856:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.732892:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609621760 00000100:00100000:0.0:1780298644.732896:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609621760 00000100:00100000:0.0:1780298644.732926:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10017 00000100:00100000:0.0:1780298644.732934:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343965880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609621760:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.732943:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.732951:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000441cddf1/0xaf35132e6303544a lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d43b5 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.732982:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605042304:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.733001:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.733032:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.733046:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609621888 00000100:00100000:1.0:1780298644.733052:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609621888 00000100:00100000:1.0:1780298644.733082:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10018 00000100:00100000:1.0:1780298644.733092:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243845c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609621888:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.733100:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.733108:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e63035451 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d43bc expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.733156:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605042432:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.733196:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837af00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.733600:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605042304:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.733616:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000441cddf1/0xaf35132e6303544a lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d43b5 expref: 7 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.733630:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000441cddf1/0xaf35132e6303544a lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d43b5 expref: 7 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.733637:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf400, rc 0) 00000100:00100000:0.0:1780298644.733665:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343965880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609621760:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 733us (810us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.733673:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10017 00000100:00100000:1.0:1780298644.733796:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605042432:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.733814:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e63035451 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d43bc expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.733828:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e63035451 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d43bc expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.733836:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279a00, rc 0) 00000100:00100000:1.0:1780298644.733864:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243845c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609621888:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 775us (864us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.733871:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10018 00000100:00100000:1.0:1780298644.734709:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.734735:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609622016 00000100:00100000:3.0:1780298644.734740:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609622016 00000100:00100000:3.0:1780298644.734766:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10019 00000100:00100000:3.0:1780298644.734774:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609622016:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.734782:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.734790:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bed20494/0xaf35132e63035458 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d43c3 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.734816:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343964000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605042560:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.734854:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.735398:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343964000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605042560:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.735426:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bed20494/0xaf35132e63035458 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d43c3 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.735440:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bed20494/0xaf35132e63035458 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d43c3 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.735449:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157000, rc 0) 00000100:00100000:3.0:1780298644.735478:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609622016:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 705us (768us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.735485:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10019 00000100:00100000:2.0:1780298644.736212:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.736242:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609622144 00000100:00100000:1.0:1780298644.736246:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609622144 00000100:00100000:1.0:1780298644.736284:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10020 00000100:00100000:1.0:1780298644.736293:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243844700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609622144:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.736301:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.736309:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e6303545f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d43ca expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.736335:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605042688:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.736365:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837af00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.736966:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605042688:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.736983:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e6303545f lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d43ca expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.736998:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e6303545f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d43ca expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.737008:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279200, rc 0) 00000100:00100000:0.0:1780298644.737034:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243844700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609622144:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 744us (823us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.737042:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10020 00000100:00100000:2.0:1780298644.737872:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.737946:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609622272 00000100:00100000:3.0:1780298644.737952:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609622272 00000100:00100000:3.0:1780298644.737985:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10021 00000100:00100000:3.0:1780298644.737994:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243844a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609622272:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298644.738000:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:3.0:1780298644.738006:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.738018:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63035466 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d43d1 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.738032:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609622400 00000100:00100000:2.0:1780298644.738035:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609622400 00000100:00100000:2.0:1780298644.738058:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10022 00000100:00100000:2.0:1780298644.738063:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609622400:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.738069:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.738075:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e6303546d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d43d8 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.738087:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343964000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605042816:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.738097:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255488000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605042944:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.738162:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.738163:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb13c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.738780:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343964000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605042816:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.738802:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63035466 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d43d1 expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.738817:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e63035466 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d43d1 expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.738831:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255488000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605042944:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.738849:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e6303546d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d43d8 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.738862:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e6303546d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d43d8 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.738867:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157c00, rc 0) 00010000:00010000:1.0:1780298644.738871:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3600, rc 0) 00000100:00100000:3.0:1780298644.738895:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243844a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609622272:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 902us (1021us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.738895:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609622400:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 831us (899us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.738909:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10022 00000100:00100000:3.0:1780298644.738914:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10021 00000100:00100000:0.0:1780298644.739670:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.739696:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609622528 00000100:00100000:3.0:1780298644.739701:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609622528 00000100:00100000:0.0:1780298644.739701:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.739731:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609622656 00000100:00100000:3.0:1780298644.739734:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609622656 00000100:00100000:1.0:1780298644.739745:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10023 00000100:00100000:3.0:1780298644.739755:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10024 00000100:00100000:1.0:1780298644.739756:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609622528:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.739765:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298644.739768:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609622656:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.739774:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63035474 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d43df expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.739775:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.739783:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004ba675d2/0xaf35132e6303547b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d43e6 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.739797:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605043072:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.739810:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343964000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605043200:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.739832:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a840 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.739836:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.740563:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343964000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605043200:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.740583:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004ba675d2/0xaf35132e6303547b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d43e6 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.740594:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004ba675d2/0xaf35132e6303547b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d43e6 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.740602:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156e00, rc 0) 00000100:00100000:1.0:1780298644.740661:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605043072:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.740662:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609622656:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 896us (960us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.740673:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10024 00010000:00010000:1.0:1780298644.740689:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63035474 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d43df expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.740702:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63035474 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d43df expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.740713:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278c00, rc 0) 00000100:00100000:1.0:1780298644.740742:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609622528:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 991us (1073us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.740749:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10023 00000100:00100000:0.0:1780298644.741616:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.741656:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609622784 00000100:00100000:1.0:1780298644.741660:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609622784 00000100:00100000:0.0:1780298644.741670:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.741691:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609622912 00000100:00100000:1.0:1780298644.741695:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609622912 00000100:00100000:1.0:1780298644.741727:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10025 00000100:00100000:2.0:1780298644.741736:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10026 00000100:00100000:1.0:1780298644.741741:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609622784:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298644.741752:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609622912:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.741752:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.741761:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.741762:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63035482 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d43ed expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.741769:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e63035489 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d43f4 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.741801:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255489c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605043456:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.741801:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605043328:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.741847:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb13c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.741848:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a840 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.742516:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255489c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605043456:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.742531:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e63035489 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d43f4 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.742537:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605043328:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.742543:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e63035489 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d43f4 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.742551:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3000, rc 0) 00010000:00010000:1.0:1780298644.742561:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63035482 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d43ed expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.742576:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609622912:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 829us (906us total) trans 0 rc 301/301 00010000:00010000:1.0:1780298644.742579:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63035482 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d43ed expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.742594:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10026 00010000:00010000:1.0:1780298644.742597:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279a00, rc 0) 00000100:00100000:1.0:1780298644.742618:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609622784:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 884us (1009us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.742623:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10025 00000100:00100000:3.0:1780298644.743591:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.743618:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609623040 00000100:00100000:1.0:1780298644.743622:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609623040 00002000:00100000:1.0:1780298644.743642:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x12:0x0] (2465792->2469887) req@ffff9cf343964000 x1866777609623040/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.743649:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf343964000 with x1866777609623040 ext(2465792->2469887) 00002000:00010000:1.0:1780298644.743653:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf343964000 00000100:00100000:1.0:1780298644.743668:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6352 00000100:00100000:1.0:1780298644.743673:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343964000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609623040:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298644.743788:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343964000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609623040:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 115us (198us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298644.743793:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6352 00002000:00100000:1.0:1780298644.743798:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x12:0x0] (2465792->2469887) req@ffff9cf343964000 x1866777609623040/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/4536 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.743804:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf343964000 with x1866777609623040 ext(2465792->2469887) 00002000:00010000:1.0:1780298644.743806:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf343964000 00000100:00100000:0.0:1780298644.744695:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.744714:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.744730:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609623168 00000100:00100000:2.0:1780298644.744734:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609623168 00000100:00100000:3.0:1780298644.744739:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609623296 00000100:00100000:3.0:1780298644.744743:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609623296 00000100:00100000:2.0:1780298644.744761:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10027 00000100:00100000:2.0:1780298644.744770:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609623168:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298644.744776:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10028 00010000:00010000:2.0:1780298644.744776:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.744786:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63035490 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d43fb expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.744787:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244811f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609623296:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.744799:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.744807:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e0b8e7b/0xaf35132e63035497 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4402 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.744814:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255489c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605043584:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.744839:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343964a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605043712:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.744856:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb13c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.744870:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.745512:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255489c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605043584:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.745515:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343964a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605043712:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.745530:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63035490 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d43fb expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.745537:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009e0b8e7b/0xaf35132e63035497 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4402 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.745546:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63035490 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d43fb expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.745551:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e0b8e7b/0xaf35132e63035497 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4402 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.745557:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2600, rc 0) 00010000:00010000:3.0:1780298644.745561:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156c00, rc 0) 00000100:00100000:2.0:1780298644.745588:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609623168:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 820us (892us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.745592:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244811f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609623296:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 807us (874us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.745598:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10027 00000100:00100000:3.0:1780298644.745605:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10028 00000100:00100000:0.0:1780298644.746512:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.746542:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609623424 00000100:00100000:2.0:1780298644.746546:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609623424 00000100:00100000:2.0:1780298644.746569:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10029 00000100:00100000:2.0:1780298644.746574:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244812680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609623424:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.746581:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.746588:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e6303549e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4409 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.746607:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255489c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605043840:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.746648:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb13c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.746669:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.746693:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609623552 00000100:00100000:1.0:1780298644.746697:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609623552 00000100:00100000:1.0:1780298644.746723:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10030 00000100:00100000:1.0:1780298644.746727:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244812d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609623552:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.746733:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.746739:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017dab243/0xaf35132e630354a5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4410 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.746757:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605043968:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.746786:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837ae40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.747244:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255489c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605043840:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.747257:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e6303549e lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4409 expref: 7 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.747266:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e6303549e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4409 expref: 7 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.747272:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3e00, rc 0) 00000100:00100000:2.0:1780298644.747306:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244812680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609623424:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 731us (794us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.747311:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10029 00000100:00100000:1.0:1780298644.747423:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605043968:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.747439:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000017dab243/0xaf35132e630354a5 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4410 expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.747461:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017dab243/0xaf35132e630354a5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4410 expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.747471:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279800, rc 0) 00000100:00100000:1.0:1780298644.747498:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244812d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609623552:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 769us (829us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.747506:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10030 00000100:00100000:0.0:1780298644.748214:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.748240:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609623680 00000100:00100000:2.0:1780298644.748244:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609623680 00000100:00100000:2.0:1780298644.748289:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6353 00000100:00100000:2.0:1780298644.748298:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244813800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609623680:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298644.748455:0:19951:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4157841 -> 4157841 00002000:00010000:2.0:1780298644.748457:0:19951:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298644.748459:0:19951:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298644.748461:0:19951:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298644.748463:0:19951:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 88 -> 104 00000100:00100000:2.0:1780298644.748485:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244813800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609623680:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 190us (277us total) trans 4294969285 rc 0/0 00000100:00100000:2.0:1780298644.748490:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6353 00000100:00100000:3.0:1780298644.749234:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.749283:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609623808 00000100:00100000:0.0:1780298644.749288:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609623808 00000100:00100000:0.0:1780298644.749313:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10031 00000100:00100000:0.0:1780298644.749320:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343964a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609623808:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.749327:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.749335:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c37b15a2/0xaf35132e630354ac lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4417 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.749371:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244810e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605044096:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.749403:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.749926:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244810e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605044096:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.749938:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c37b15a2/0xaf35132e630354ac lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4417 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.749947:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c37b15a2/0xaf35132e630354ac lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4417 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.749953:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baee00, rc 0) 00000100:00100000:0.0:1780298644.749975:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343964a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609623808:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 657us (742us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.749980:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10031 00000100:00100000:1.0:1780298644.750675:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.750700:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609623936 00000100:00100000:3.0:1780298644.750705:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609623936 00000100:00100000:3.0:1780298644.750724:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10032 00000100:00100000:3.0:1780298644.750730:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609623936:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.750736:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.750742:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e630354b3 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d441e expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.750761:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343966300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605044224:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.750791:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.751354:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343966300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605044224:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.751376:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e630354b3 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d441e expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.751385:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e630354b3 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d441e expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.751391:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157c00, rc 0) 00000100:00100000:0.0:1780298644.751413:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609623936:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 684us (739us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.751418:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10032 00000100:00100000:0.0:1780298644.752061:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.752084:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609624064 00000100:00100000:3.0:1780298644.752088:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609624064 00000100:00100000:3.0:1780298644.752108:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10033 00000100:00100000:3.0:1780298644.752112:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244810e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609624064:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.752118:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.752144:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e0b8e7b/0xaf35132e630354ba lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4425 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.752159:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000f4922155/0xaf35132e63035435 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1896448->35450879] (req 1896448->1904639) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d43a0 expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.752165:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(25 122) 00010000:00010000:3.0:1780298644.752169:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(26 122) 00010000:00010000:3.0:1780298644.752173:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e0b8e7b/0xaf35132e630354ba lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2306048->2318335] (req 2306048->2318335) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4425 expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.752188:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000f4922155/0xaf35132e63035435 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1896448->35450879] (req 1896448->1904639) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d43a0 expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.752198:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000f4922155/0xaf35132e63035435 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1896448->35450879] (req 1896448->1904639) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d43a0 expref: 9 pid: 8356 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.752206:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343965f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605044352:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298644.752232:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.752748:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343965f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605044352:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298644.752762:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009e0b8e7b/0xaf35132e630354ba lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2306048->2318335] (req 2306048->2318335) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4425 expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.752769:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156c00, rc 0) 00000100:00100000:3.0:1780298644.752789:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244810e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609624064:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 677us (729us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298644.752794:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10033 00000100:00100000:0.0:1780298644.752910:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.752940:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609624192 00000100:00100000:2.0:1780298644.752944:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609624192 00010000:00010000:2.0:1780298644.752959:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000f4922155/0xaf35132e63035435 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1896448->35450879] (req 1896448->1904639) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d43a0 expref: 10 pid: 8356 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.752979:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15222 00000100:00100000:2.0:1780298644.752984:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244810700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609624192:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298644.752993:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298644.753003:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000f4922155/0xaf35132e63035435 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [1896448->35450879] (req 1896448->1904639) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d43a0 expref: 10 pid: 8356 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.753015:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000f4922155/0xaf35132e63035435 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [1896448->35450879] (req 1896448->1904639) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d43a0 expref: 10 pid: 8356 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.753030:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f4922155/0xaf35132e63035435 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [1896448->35450879] (req 1896448->1904639) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d43a0 expref: 10 pid: 8356 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.753040:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(26 122) 00010000:00010000:2.0:1780298644.753044:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(26 122) 00010000:00010000:2.0:1780298644.753053:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009e0b8e7b/0xaf35132e630354ba lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2306048->35860479] (req 2306048->2318335) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4425 expref: 9 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.753068:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e0b8e7b/0xaf35132e630354ba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2306048->35860479] (req 2306048->2318335) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4425 expref: 9 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.753089:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000009e0b8e7b/0xaf35132e630354ba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2306048->35860479] (req 2306048->2318335) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4425 expref: 9 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.753096:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255489c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:8374:1866777605044480:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298644.753121:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb13c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.753711:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255489c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:8374:1866777605044480:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298644.753723:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298644.753753:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244810700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609624192:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 768us (843us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.753761:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15222 00000100:00100000:1.0:1780298644.753928:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.753954:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609624320 00000100:00100000:0.0:1780298644.753958:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609624320 00002000:00100000:0.0:1780298644.753974:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (2306048->2310143) req@ffff9cf3481fe300 x1866777609624320/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298644.753981:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fe300 with x1866777609624320 ext(2306048->2310143) 00002000:00010000:0.0:1780298644.753984:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf3481fe300 00000100:00100000:0.0:1780298644.753999:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6354 00000100:00100000:0.0:1780298644.754003:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609624320:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298644.754089:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609624320:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 85us (161us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298644.754094:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6354 00002000:00100000:0.0:1780298644.754099:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (2306048->2310143) req@ffff9cf3481fe300 x1866777609624320/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/4536 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298644.754105:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fe300 with x1866777609624320 ext(2306048->2310143) 00002000:00010000:0.0:1780298644.754108:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf3481fe300 00000100:00100000:0.0:1780298644.754753:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.754795:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609624448 00000100:00100000:3.0:1780298644.754799:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609624448 00002000:00100000:3.0:1780298644.754813:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (2314240->2318335) req@ffff9cf244810a80 x1866777609624448/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298644.754822:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244810a80 with x1866777609624448 ext(2314240->2318335) 00002000:00010000:3.0:1780298644.754826:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf244810a80 00000100:00100000:3.0:1780298644.754844:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6355 00000100:00100000:3.0:1780298644.754853:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244810a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609624448:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298644.754924:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244810a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609624448:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 73us (172us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1780298644.754930:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6355 00002000:00100000:3.0:1780298644.754935:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (2314240->2318335) req@ffff9cf244810a80 x1866777609624448/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/4536 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298644.754941:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244810a80 with x1866777609624448 ext(2314240->2318335) 00002000:00010000:3.0:1780298644.754944:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf244810a80 00000100:00100000:3.0:1780298644.755697:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.755751:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609624576 00000100:00100000:1.0:1780298644.755756:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609624576 00000100:00100000:3.0:1780298644.755766:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.755783:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609624704 00000100:00100000:1.0:1780298644.755786:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609624704 00000100:00100000:2.0:1780298644.755797:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10034 00000100:00100000:2.0:1780298644.755811:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343965f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609624576:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298644.755811:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10035 00010000:00010000:2.0:1780298644.755820:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298644.755824:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343964380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609624704:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.755827:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e630354c1 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d442c expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.755831:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.755840:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005b92bd2b/0xaf35132e630354c8 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4433 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.755851:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255489c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605044608:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.755872:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605044736:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.755890:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb13c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.755906:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837ae40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.756530:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255489c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605044608:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.756547:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e630354c1 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d442c expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.756558:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605044736:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.756560:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e630354c1 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d442c expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.756570:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3e00, rc 0) 00010000:00010000:0.0:1780298644.756578:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005b92bd2b/0xaf35132e630354c8 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4433 expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.756595:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343965f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609624576:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 790us (896us total) trans 0 rc 301/301 00010000:00010000:0.0:1780298644.756596:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005b92bd2b/0xaf35132e630354c8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4433 expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.756606:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10034 00010000:00010000:0.0:1780298644.756611:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279c00, rc 0) 00000100:00100000:0.0:1780298644.756632:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343964380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609624704:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 813us (866us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.756637:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10035 00000100:00100000:0.0:1780298644.757404:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.757471:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609624832 00000100:00100000:3.0:1780298644.757475:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609624832 00000100:00100000:1.0:1780298644.757505:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.757523:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609624960 00000100:00100000:3.0:1780298644.757527:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10036 00000100:00100000:2.0:1780298644.757528:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609624960 00000100:00100000:3.0:1780298644.757536:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244811880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609624832:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.757543:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.757550:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009c841b52/0xaf35132e630354cf lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d443a expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.757550:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10037 00000100:00100000:2.0:1780298644.757560:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609624960:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.757566:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.757574:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e630354d6 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4441 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.757578:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343966300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605044864:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.757605:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255489c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605044992:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.757622:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.757632:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb13c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.758285:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343966300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605044864:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.758305:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009c841b52/0xaf35132e630354cf lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d443a expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.758317:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255489c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605044992:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.758319:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009c841b52/0xaf35132e630354cf lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d443a expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.758339:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e630354d6 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4441 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.758353:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e630354d6 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4441 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.758368:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2600, rc 0) 00010000:00010000:3.0:1780298644.758387:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157600, rc 0) 00000100:00100000:2.0:1780298644.758400:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609624960:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 842us (894us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.758412:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244811880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609624832:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 875us (1003us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.758412:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10037 00000100:00100000:3.0:1780298644.758422:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10036 00000100:00100000:0.0:1780298644.759313:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.759349:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609625088 00000100:00100000:1.0:1780298644.759354:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609625088 00000100:00100000:1.0:1780298644.759379:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10038 00000100:00100000:1.0:1780298644.759386:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244813b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609625088:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.759393:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.759400:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e630354dd lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4448 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.759424:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605045120:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.759457:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.759978:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605045120:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.759991:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e630354dd lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4448 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.760002:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e630354dd lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4448 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.760008:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278400, rc 0) 00000100:00100000:2.0:1780298644.760033:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244813b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609625088:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 649us (721us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.760039:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10038 00000100:00100000:0.0:1780298644.760752:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.760783:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609625216 00000100:00100000:1.0:1780298644.760787:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609625216 00000100:00100000:1.0:1780298644.760805:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10039 00000100:00100000:1.0:1780298644.760810:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244813480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609625216:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.760815:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.760821:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e630354e4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d444f expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.760832:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e630353b0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [430080->1388543] (req 430080->434175) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d431b expref: 8 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.760837:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(26 122) 00010000:00010000:1.0:1780298644.760840:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(27 122) 00010000:00010000:1.0:1780298644.760842:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e630354e4 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [651264->659455] (req 651264->659455) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d444f expref: 11 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.760853:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e630353b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [430080->1388543] (req 430080->434175) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d431b expref: 8 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.760862:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e630353b0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [430080->1388543] (req 430080->434175) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d431b expref: 8 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.760868:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605045248:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298644.760894:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a540 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.761405:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605045248:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298644.761415:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e630354e4 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [651264->659455] (req 651264->659455) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d444f expref: 11 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.761422:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279400, rc 0) 00000100:00100000:1.0:1780298644.761441:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244813480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:29360:x1866777609625216:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 631us (689us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.761446:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10039 00000100:00100000:0.0:1780298644.761635:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.761677:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609625344 00000100:00100000:1.0:1780298644.761681:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609625344 00002000:00100000:1.0:1780298644.761699:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (430080->434175) req@ffff9cf244811c00 x1866777609625344/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 4584/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.761706:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244811c00 with x1866777609625344 ext(430080->434175) 00010000:00010000:1.0:1780298644.761711:0:8362:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e630353b0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [430080->1388543] (req 430080->434175) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d431b expref: 9 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.761718:0:8362:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e630353b0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [430080->1388543] (req 430080->434175) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d431b expref: 9 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1780298644.761727:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9cf244811c00 00000100:00100000:1.0:1780298644.761742:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 209 00000100:00100000:1.0:1780298644.761747:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244811c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609625344:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:1.0:1780298644.762174:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244811c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609625344:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 425us (538us total) trans 4294969286 rc 0/0 00000100:00100000:1.0:1780298644.762179:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 209 00002000:00100000:1.0:1780298644.762184:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (430080->434175) req@ffff9cf244811c00 x1866777609625344/t4294969286(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 4584/448 e 0 to 0 dl 1780298655 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.762191:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244811c00 with x1866777609625344 ext(430080->434175) 00010000:00010000:1.0:1780298644.762195:0:8362:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e630353b0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [430080->1388543] (req 430080->434175) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d431b expref: 11 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.762207:0:8362:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e630353b0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [430080->1388543] (req 430080->434175) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d431b expref: 11 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1780298644.762212:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9cf244811c00 00000100:00100000:0.0:1780298644.762797:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.762818:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609625472 00000100:00100000:3.0:1780298644.762822:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609625472 00010000:00010000:3.0:1780298644.762837:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e630353b0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [430080->1388543] (req 430080->434175) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d431b expref: 11 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.762856:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15223 00000100:00100000:3.0:1780298644.762860:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348af5c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609625472:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298644.762866:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:3.0:1780298644.762872:0:6462:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 104 -> 112 00010000:00010000:3.0:1780298644.762875:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e630353b0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [430080->1388543] (req 430080->434175) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d431b expref: 11 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.762882:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e630353b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [430080->1388543] (req 430080->434175) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d431b expref: 11 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.762892:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e630353b0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [430080->1388543] (req 430080->434175) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d431b expref: 11 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.762902:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(27 122) 00010000:00010000:3.0:1780298644.762905:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(27 122) 00010000:00010000:3.0:1780298644.762910:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e630354e4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [651264->2306047] (req 651264->659455) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d444f expref: 10 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.762915:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e630354e4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [651264->2306047] (req 651264->659455) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d444f expref: 10 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.762929:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e630354e4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [651264->2306047] (req 651264->659455) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d444f expref: 10 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.762935:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343966300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777605045376:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298644.762964:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.763575:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343966300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777605045376:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298644.763589:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298644.763616:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348af5c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609625472:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 755us (819us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.763622:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15223 00000100:00100000:0.0:1780298644.763695:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.763717:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609625600 00000100:00100000:1.0:1780298644.763722:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609625600 00000100:00100000:1.0:1780298644.763747:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6356 00000100:00100000:1.0:1780298644.763754:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244812300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609625600:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298644.763818:0:19949:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4157841 -> 4157841 00002000:00010000:1.0:1780298644.763821:0:19949:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298644.763822:0:19949:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298644.763824:0:19949:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298644.763826:0:19949:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 112 -> 128 00000100:00100000:1.0:1780298644.763849:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244812300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2399:x1866777609625600:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 97us (156us total) trans 4294969287 rc 0/0 00000100:00100000:1.0:1780298644.763854:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6356 00000100:00100000:1.0:1780298644.764569:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.764601:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609625728 00000100:00100000:0.0:1780298644.764605:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609625728 00000100:00100000:0.0:1780298644.764635:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10040 00000100:00100000:0.0:1780298644.764644:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e4f3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609625728:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.764652:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298644.764656:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:0.0:1780298644.764663:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e630354eb lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4456 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.764681:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609625856 00000100:00100000:2.0:1780298644.764686:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609625856 00000100:00100000:0.0:1780298644.764695:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244810700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605045504:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.764714:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10041 00000100:00100000:2.0:1780298644.764722:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343966d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609625856:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.764729:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.764737:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e630354f2 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d445d expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.764737:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.764763:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255489c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605045632:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.764786:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.765404:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244810700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605045504:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.765420:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e630354eb lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4456 expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.765430:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e630354eb lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4456 expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.765437:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf200, rc 0) 00000100:00100000:2.0:1780298644.765439:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255489c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605045632:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.765460:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e630354f2 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d445d expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.765462:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e4f3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609625728:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 821us (893us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.765477:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10040 00010000:00010000:2.0:1780298644.765484:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e630354f2 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d445d expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.765493:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3000, rc 0) 00000100:00100000:2.0:1780298644.765515:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343966d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609625856:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 795us (860us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.765522:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10041 00000100:00100000:1.0:1780298644.766353:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.766381:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609625984 00000100:00100000:2.0:1780298644.766386:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609625984 00000100:00100000:1.0:1780298644.766386:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.766414:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609626112 00000100:00100000:2.0:1780298644.766416:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609626112 00000100:00100000:0.0:1780298644.766420:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10042 00000100:00100000:0.0:1780298644.766427:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609625984:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.766433:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298644.766436:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10043 00010000:00010000:0.0:1780298644.766440:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000441cddf1/0xaf35132e630354f9 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4464 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.766446:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609626112:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.766456:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.766464:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63035500 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d446b expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.766469:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244811f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605045760:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.766495:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255489c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605045888:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.766508:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.766521:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.767223:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244811f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605045760:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.767241:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000441cddf1/0xaf35132e630354f9 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4464 expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.767253:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000441cddf1/0xaf35132e630354f9 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4464 expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.767261:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf400, rc 0) 00000100:00100000:2.0:1780298644.767272:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255489c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605045888:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.767298:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63035500 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d446b expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.767302:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609625984:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 874us (949us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.767315:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10042 00010000:00010000:2.0:1780298644.767318:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a36d96d0/0xaf35132e63035500 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d446b expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.767325:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3c00, rc 0) 00000100:00100000:2.0:1780298644.767341:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609626112:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 900us (959us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.767346:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10043 00000100:00100000:1.0:1780298644.768088:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.768106:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609626240 00000100:00100000:3.0:1780298644.768180:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609626240 00000100:00100000:3.0:1780298644.768205:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10044 00000100:00100000:3.0:1780298644.768213:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481ff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609626240:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.768219:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.768225:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004ba675d2/0xaf35132e63035507 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4472 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.768235:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.768246:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343966300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605046016:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.768259:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609626368 00000100:00100000:0.0:1780298644.768264:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609626368 00000100:00100000:0.0:1780298644.768305:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10045 00000100:00100000:2.0:1780298644.768311:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.768313:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255489c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609626368:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.768323:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.768332:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e6303550e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4479 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.768369:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244812680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605046144:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.768404:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733936c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.768865:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343966300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605046016:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.768882:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004ba675d2/0xaf35132e63035507 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4472 expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.768893:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004ba675d2/0xaf35132e63035507 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4472 expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.768899:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156e00, rc 0) 00000100:00100000:0.0:1780298644.768921:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481ff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609626240:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 711us (834us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.768926:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10044 00000100:00100000:0.0:1780298644.769035:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244812680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605046144:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.769047:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e6303550e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4479 expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.769055:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e6303550e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4479 expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.769061:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae000, rc 0) 00000100:00100000:0.0:1780298644.769081:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255489c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609626368:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 771us (849us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.769087:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10045 00000100:00100000:3.0:1780298644.769757:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.769784:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609626496 00000100:00100000:3.0:1780298644.769785:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.769789:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609626496 00000100:00100000:1.0:1780298644.769811:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609626624 00000100:00100000:1.0:1780298644.769813:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609626624 00000100:00100000:0.0:1780298644.769820:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10046 00000100:00100000:0.0:1780298644.769827:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343966300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609626496:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.769833:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298644.769839:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10047 00010000:00010000:0.0:1780298644.769839:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005d10d3aa/0xaf35132e63035515 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4480 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.769851:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343965880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609626624:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.769858:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298644.769863:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244812680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605046272:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.769866:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e6303551c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4487 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.769894:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255488380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605046400:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.769898:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733936c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.769921:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.770459:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244812680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605046272:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.770470:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005d10d3aa/0xaf35132e63035515 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4480 expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.770478:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005d10d3aa/0xaf35132e63035515 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4480 expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.770484:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baec00, rc 0) 00000100:00100000:0.0:1780298644.770504:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343966300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609626496:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 679us (749us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.770508:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10046 00000100:00100000:2.0:1780298644.770641:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255488380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605046400:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.770655:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e6303551c lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4487 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.770666:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e6303551c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4487 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.770672:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2000, rc 0) 00000100:00100000:2.0:1780298644.770700:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343965880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609626624:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 851us (915us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.770707:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10047 00000100:00100000:1.0:1780298644.771507:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.771528:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609626752 00000100:00100000:3.0:1780298644.771533:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609626752 00000100:00100000:1.0:1780298644.771539:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.771562:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609626880 00000100:00100000:3.0:1780298644.771565:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609626880 00000100:00100000:3.0:1780298644.771579:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10048 00000100:00100000:3.0:1780298644.771587:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609626752:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298644.771591:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10049 00010000:00010000:3.0:1780298644.771595:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.771603:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009c841b52/0xaf35132e63035523 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d448e expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.771605:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481ffb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609626880:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.771618:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.771627:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e6303552a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4495 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.771658:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244812680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605046528:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.771663:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255488380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605046656:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.771712:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.771713:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733936c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.772387:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255488380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605046656:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.772403:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009c841b52/0xaf35132e63035523 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d448e expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.772414:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009c841b52/0xaf35132e63035523 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d448e expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.772418:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244812680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605046528:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.772424:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157600, rc 0) 00010000:00010000:0.0:1780298644.772435:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e6303552a lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4495 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.772448:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e6303552a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4495 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.772452:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609626752:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 865us (945us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.772466:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10048 00010000:00010000:0.0:1780298644.772467:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf200, rc 0) 00000100:00100000:0.0:1780298644.772488:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481ffb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609626880:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 890us (950us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.772493:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10049 00000100:00100000:1.0:1780298644.773351:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.773407:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609627008 00000100:00100000:3.0:1780298644.773410:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609627008 00000100:00100000:3.0:1780298644.773436:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10050 00000100:00100000:3.0:1780298644.773445:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609627008:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.773453:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298644.773460:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:3.0:1780298644.773462:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e63035531 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d449c expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.773483:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609627136 00000100:00100000:0.0:1780298644.773488:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609627136 00000100:00100000:3.0:1780298644.773494:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343964a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605046784:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.773510:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10051 00000100:00100000:0.0:1780298644.773516:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255489500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609627136:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.773522:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.773528:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63035538 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d44a3 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.773548:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244812680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605046912:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.773585:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733936c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.773588:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.774233:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343964a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605046784:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.774236:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244812680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605046912:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.774259:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e63035531 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d449c expref: 7 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.774261:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63035538 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44a3 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.774294:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e63035531 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d449c expref: 7 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.774299:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63035538 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44a3 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.774314:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157000, rc 0) 00010000:00010000:0.0:1780298644.774318:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae000, rc 0) 00000100:00100000:2.0:1780298644.774348:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609627008:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 904us (996us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.774349:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255489500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609627136:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 832us (891us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.774362:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10051 00000100:00100000:2.0:1780298644.774371:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10050 00000100:00100000:1.0:1780298644.775191:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.775214:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609627264 00000100:00100000:2.0:1780298644.775218:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609627264 00000100:00100000:2.0:1780298644.775238:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10052 00000100:00100000:2.0:1780298644.775244:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:29360:x1866777609627264:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.775249:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.775255:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e6303553f lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d44aa expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.775267:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e6303531d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4296 expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.775273:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(27 169) 00010000:00010000:2.0:1780298644.775284:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(28 169) 00010000:00010000:2.0:1780298644.775288:0:20252:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e6303553f lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2371584->2379775] (req 2371584->2379775) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44aa expref: 7 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.775303:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e6303531d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4296 expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.775314:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e6303531d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4296 expref: 8 pid: 8356 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.775321:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605047040:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298644.775358:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1480 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.775866:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605047040:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298644.775878:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e6303553f lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2371584->2379775] (req 2371584->2379775) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44aa expref: 7 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.775887:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3c00, rc 0) 00000100:00100000:2.0:1780298644.775908:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609627264:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 665us (720us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.775913:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10052 00000100:00100000:1.0:1780298644.776089:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.776119:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609627392 00000100:00100000:2.0:1780298644.776123:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609627392 00010000:00010000:2.0:1780298644.776154:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e6303531d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4296 expref: 9 pid: 8356 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.776173:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15224 00000100:00100000:2.0:1780298644.776182:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609627392:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298644.776188:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298644.776195:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e6303531d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4296 expref: 9 pid: 8356 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.776202:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e6303531d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4296 expref: 9 pid: 8356 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.776215:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e6303531d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4296 expref: 9 pid: 8356 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.776229:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(28 169) 00010000:00010000:2.0:1780298644.776233:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(28 169) 00010000:00010000:2.0:1780298644.776241:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e6303553f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2371584->35926015] (req 2371584->2379775) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44aa expref: 6 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.776250:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e6303553f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2371584->35926015] (req 2371584->2379775) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44aa expref: 6 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.776270:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e6303553f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2371584->35926015] (req 2371584->2379775) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44aa expref: 6 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.776288:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548ad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777605047168:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298644.776318:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1480 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.776941:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548ad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777605047168:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298644.776949:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298644.776966:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609627392:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 788us (879us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.776971:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15224 00000100:00100000:1.0:1780298644.777041:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.777059:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609627520 00000100:00100000:2.0:1780298644.777062:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609627520 00000100:00100000:2.0:1780298644.777090:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6357 00000100:00100000:2.0:1780298644.777096:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609627520:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298644.777236:0:19951:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:2.0:1780298644.777239:0:19951:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298644.777242:0:19951:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298644.777245:0:19951:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298644.777247:0:19951:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 400 -> 400 00000100:00100000:2.0:1780298644.777270:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609627520:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 175us (231us total) trans 4294969739 rc 0/0 00000100:00100000:2.0:1780298644.777284:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6357 00000100:00100000:2.0:1780298644.778042:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.778060:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.778079:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609627776 00000100:00100000:0.0:1780298644.778083:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609627776 00000100:00100000:1.0:1780298644.778098:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609627648 00000100:00100000:1.0:1780298644.778103:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609627648 00000100:00100000:0.0:1780298644.778118:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10053 00000100:00100000:0.0:1780298644.778146:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609627776:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298644.778155:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10054 00010000:00010000:0.0:1780298644.778155:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.778166:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e63035546 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d44b8 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.778169:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609627648:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.778183:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.778194:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e6303554d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d44b1 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.778205:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244812680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605047296:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.778231:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343965f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605047424:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.778253:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733936c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.778290:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.778907:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244812680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605047296:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.778920:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e63035546 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44b8 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.778936:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e63035546 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44b8 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.778944:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baec00, rc 0) 00000100:00100000:0.0:1780298644.778967:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609627776:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 844us (925us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.778972:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10053 00000100:00100000:3.0:1780298644.779090:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343965f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605047424:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.779111:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e6303554d lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44b1 expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.779156:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e6303554d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44b1 expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.779169:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156a00, rc 0) 00000100:00100000:2.0:1780298644.779220:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609627648:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1054us (1156us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.779228:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10054 00000100:00100000:1.0:1780298644.780081:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.780151:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609627904 00000100:00100000:2.0:1780298644.780157:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609627904 00000100:00100000:3.0:1780298644.780171:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.780198:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609628032 00000100:00100000:2.0:1780298644.780201:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609628032 00000100:00100000:1.0:1780298644.780208:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10055 00000100:00100000:1.0:1780298644.780220:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254888000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609627904:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.780231:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298644.780232:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10056 00010000:00010000:1.0:1780298644.780241:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63035554 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d44bf expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.780242:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343965f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609628032:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.780256:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.780265:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e6303555b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d44c6 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.780296:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254888380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605047552:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.780305:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605047680:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.780350:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1480 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.780352:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a540 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.781014:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254888380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605047552:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.781035:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605047680:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.781036:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63035554 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44bf expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.781053:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63035554 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44bf expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.781059:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e6303555b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44c6 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.781079:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279c00, rc 0) 00010000:00010000:0.0:1780298644.781079:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e6303555b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44c6 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.781094:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2000, rc 0) 00000100:00100000:2.0:1780298644.781104:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254888000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609627904:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 889us (1024us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.781109:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10055 00000100:00100000:0.0:1780298644.781147:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343965f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609628032:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 879us (949us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.781153:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10056 00000100:00100000:0.0:1780298644.782269:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.782318:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609628160 00000100:00100000:2.0:1780298644.782323:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609628160 00000100:00100000:0.0:1780298644.782328:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.782361:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609628288 00000100:00100000:2.0:1780298644.782365:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609628288 00000100:00100000:1.0:1780298644.782369:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10057 00000100:00100000:1.0:1780298644.782382:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244812680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609628160:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.782391:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.782402:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63035562 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d44cd expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.782422:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10058 00000100:00100000:3.0:1780298644.782435:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244812d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609628288:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298644.782438:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25488a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605047808:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.782467:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.782476:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e63035569 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d44d4 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.782487:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a540 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.782501:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255488a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605047936:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.782525:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.783225:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25488a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605047808:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.783246:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63035562 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44cd expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.783263:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63035562 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44cd expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.783286:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278c00, rc 0) 00000100:00100000:1.0:1780298644.783317:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244812680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609628160:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 940us (1049us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.783326:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10057 00000100:00100000:2.0:1780298644.783355:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255488a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605047936:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.783373:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e63035569 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44d4 expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.783387:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e63035569 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44d4 expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.783396:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3000, rc 0) 00000100:00100000:2.0:1780298644.783421:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244812d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609628288:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 992us (1094us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.783429:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10058 00000100:00100000:0.0:1780298644.784051:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.784073:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609628416 00000100:00100000:3.0:1780298644.784079:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609628416 00000100:00100000:3.0:1780298644.784109:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10059 00000100:00100000:3.0:1780298644.784116:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244810e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:29360:x1866777609628416:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.784148:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.784158:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eae09c9b/0xaf35132e63035570 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d44db expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.784178:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000009e0b8e7b/0xaf35132e630354ba lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2306048->35860479] (req 2306048->2318335) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4425 expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.784187:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(27 122) 00010000:00010000:3.0:1780298644.784191:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(28 122) 00010000:00010000:3.0:1780298644.784195:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eae09c9b/0xaf35132e63035570 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3989504->4001791] (req 3989504->4001791) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44db expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.784217:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000009e0b8e7b/0xaf35132e630354ba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2306048->35860479] (req 2306048->2318335) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4425 expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.784232:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000009e0b8e7b/0xaf35132e630354ba lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2306048->35860479] (req 2306048->2318335) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4425 expref: 11 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.784243:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343964380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605048064:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298644.784292:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.784819:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343964380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605048064:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298644.784832:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000eae09c9b/0xaf35132e63035570 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3989504->4001791] (req 3989504->4001791) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44db expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.784840:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157a00, rc 0) 00000100:00100000:1.0:1780298644.784866:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244810e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:29360:x1866777609628416:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 750us (814us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.784873:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10059 00000100:00100000:1.0:1780298644.785192:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.785229:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609628544 00000100:00100000:0.0:1780298644.785232:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609628544 00002000:00100000:0.0:1780298644.785265:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (2306048->2318335) req@ffff9cf25488a680 x1866777609628544/t0(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 12776/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298644.785285:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25488a680 with x1866777609628544 ext(2306048->2318335) 00010000:00010000:0.0:1780298644.785292:0:8364:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000009e0b8e7b/0xaf35132e630354ba lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2306048->35860479] (req 2306048->2318335) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4425 expref: 12 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.785305:0:8364:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000009e0b8e7b/0xaf35132e630354ba lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2306048->35860479] (req 2306048->2318335) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4425 expref: 12 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1780298644.785316:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9cf25488a680 00000100:00100000:0.0:1780298644.785335:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 210 00000100:00100000:0.0:1780298644.785343:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25488a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2399:x1866777609628544:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:0.0:1780298644.785820:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25488a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609628544:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 479us (629us total) trans 4294969288 rc 0/0 00000100:00100000:0.0:1780298644.785826:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 210 00002000:00100000:0.0:1780298644.785831:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (2306048->2318335) req@ffff9cf25488a680 x1866777609628544/t4294969288(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 12776/448 e 0 to 0 dl 1780298655 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298644.785837:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25488a680 with x1866777609628544 ext(2306048->2318335) 00010000:00010000:0.0:1780298644.785841:0:8364:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000009e0b8e7b/0xaf35132e630354ba lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2306048->35860479] (req 2306048->2318335) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4425 expref: 14 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.785850:0:8364:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000009e0b8e7b/0xaf35132e630354ba lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2306048->35860479] (req 2306048->2318335) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4425 expref: 14 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1780298644.785855:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9cf25488a680 00000100:00100000:0.0:1780298644.786675:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.786720:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609628672 00000100:00100000:3.0:1780298644.786726:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609628672 00010000:00010000:3.0:1780298644.786747:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000009e0b8e7b/0xaf35132e630354ba lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2306048->35860479] (req 2306048->2318335) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4425 expref: 14 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.786777:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15225 00000100:00100000:3.0:1780298644.786787:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244811880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609628672:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298644.786797:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:3.0:1780298644.786806:0:7400:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 128 -> 152 00010000:00010000:3.0:1780298644.786812:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000009e0b8e7b/0xaf35132e630354ba lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [2306048->35860479] (req 2306048->2318335) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4425 expref: 14 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.786824:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000009e0b8e7b/0xaf35132e630354ba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [2306048->35860479] (req 2306048->2318335) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4425 expref: 14 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.786847:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e0b8e7b/0xaf35132e630354ba lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [2306048->35860479] (req 2306048->2318335) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4425 expref: 14 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.786859:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(28 122) 00010000:00010000:3.0:1780298644.786863:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(28 122) 00010000:00010000:3.0:1780298644.786871:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000eae09c9b/0xaf35132e63035570 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3989504->37543935] (req 3989504->4001791) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44db expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.786881:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eae09c9b/0xaf35132e63035570 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3989504->37543935] (req 3989504->4001791) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44db expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.786908:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000eae09c9b/0xaf35132e63035570 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3989504->37543935] (req 3989504->4001791) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44db expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.786920:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343964380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:7400:1866777605048192:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298644.786964:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.787829:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343964380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:7400:1866777605048192:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298644.787839:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:0.0:1780298644.787848:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298644.787885:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244811880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609628672:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1100us (1211us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.787895:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15225 00000100:00100000:1.0:1780298644.787935:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609628800 00000100:00100000:1.0:1780298644.787948:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609628800 00000100:00100000:1.0:1780298644.788021:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6358 00000100:00100000:1.0:1780298644.788035:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343964a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609628800:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298644.788288:0:19950:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4157841 -> 4157841 00002000:00010000:1.0:1780298644.788293:0:19950:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298644.788295:0:19950:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298644.788298:0:19950:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298644.788300:0:19950:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 152 -> 176 00000100:00100000:1.0:1780298644.788356:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343964a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609628800:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 327us (521us total) trans 4294969289 rc 0/0 00000100:00100000:1.0:1780298644.788363:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6358 00000100:00100000:3.0:1780298644.789449:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.789503:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609628928 00000100:00100000:3.0:1780298644.789508:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.789509:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609628928 00000100:00100000:0.0:1780298644.789544:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609629056 00000100:00100000:0.0:1780298644.789546:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609629056 00000100:00100000:0.0:1780298644.789585:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10060 00000100:00100000:2.0:1780298644.789605:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10061 00000100:00100000:0.0:1780298644.789606:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343964380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609628928:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.789626:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298644.789637:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f746680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609629056:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.789641:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e63035577 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d44e2 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.789650:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.789665:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e6303557e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d44e9 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.789709:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244813b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605048320:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.789711:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255488a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605048448:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.789794:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1a80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.789796:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733936c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.790664:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255488a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605048448:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.790665:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244813b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605048320:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.790706:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e6303557e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44e9 expref: 14 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.790706:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e63035577 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44e2 expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.790741:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e6303557e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44e9 expref: 14 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.790741:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e63035577 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44e2 expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.790769:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3600, rc 0) 00010000:00010000:0.0:1780298644.790769:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf600, rc 0) 00000100:00100000:0.0:1780298644.790815:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343964380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609628928:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1220us (1369us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.790816:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f746680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609629056:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1191us (1307us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.790834:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10061 00000100:00100000:0.0:1780298644.790842:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10060 00000100:00100000:3.0:1780298644.791905:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.791928:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.791954:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609629184 00000100:00100000:2.0:1780298644.791960:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609629184 00000100:00100000:1.0:1780298644.791983:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609629312 00000100:00100000:1.0:1780298644.791988:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609629312 00000100:00100000:2.0:1780298644.792000:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10062 00000100:00100000:2.0:1780298644.792010:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2398:x1866777609629184:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298644.792020:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10063 00010000:00010000:2.0:1780298644.792025:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.792036:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e63035585 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d44f0 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.792043:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f744000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609629312:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.792062:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.792074:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a58d48e9/0xaf35132e6303558c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d44f7 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.792074:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255488a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605048576:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.792168:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f747800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605048704:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.792168:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1a80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.792307:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.792929:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255488a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605048576:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.792929:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f747800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605048704:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.792961:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a58d48e9/0xaf35132e6303558c lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44f7 expref: 14 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.792962:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e63035585 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44f0 expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.792986:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a58d48e9/0xaf35132e6303558c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44f7 expref: 14 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.792990:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e63035585 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44f0 expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.793000:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156a00, rc 0) 00010000:00010000:2.0:1780298644.793005:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2e00, rc 0) 00000100:00100000:1.0:1780298644.793045:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f744000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609629312:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1014us (1114us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.793054:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2398:x1866777609629184:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1044us (1148us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.793057:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10063 00000100:00100000:2.0:1780298644.793066:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10062 00000100:00100000:2.0:1780298644.794057:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.794106:0:7411:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609629568 00000100:00100000:1.0:1780298644.794111:0:7411:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609629568 00000100:00100000:1.0:1780298644.794191:0:7411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62042 00000100:00100000:1.0:1780298644.794202:0:7411:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255488a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+176:29360:x1866777609629568:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:1.0:1780298644.794249:0:7411:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298644.794286:0:7411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2400/0xaf35132e63035363 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 97 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42ce expref: 176 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.794305:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.794309:0:7411:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298644.794480:0:7411:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279800/0xaf35132e63035593 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 96 type: IBT flags: 0x40000000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.794494:0:7411:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278000/0xaf35132e63035332 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 96 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42a4 expref: 141 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.794518:0:7411:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279800/0xaf35132e63035593 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 96 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.794543:0:7411:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278000/0xaf35132e63035332 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 96 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42a4 expref: 141 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.794557:0:7411:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278000/0xaf35132e63035332 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 96 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42a4 expref: 141 pid: 17578 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.794569:0:7411:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25488aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7411:1866777605048832:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298644.794619:0:7411:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.795270:0:7411:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25488aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7411:1866777605048832:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298644.795298:0:7411:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279800/0xaf35132e63035593 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 96 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.795444:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.795482:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609629696 00000100:00100000:1.0:1780298644.795487:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609629696 00010000:00010000:1.0:1780298644.795508:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278000/0xaf35132e63035332 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 96 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42a4 expref: 142 pid: 17578 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.795539:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15226 00000100:00100000:1.0:1780298644.795548:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+142:2400:x1866777609629696:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298644.795558:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298644.795562:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278000/0xaf35132e63035332 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 97 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42a4 expref: 142 pid: 17578 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.795572:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278000/0xaf35132e63035332 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 97 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42a4 expref: 142 pid: 17578 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.795585:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278000/0xaf35132e63035332 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 97 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d42a4 expref: 142 pid: 17578 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.795595:0:6463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.795599:0:6463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279800/0xaf35132e63035593 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 96 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.795611:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279800/0xaf35132e63035593 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 96 type: IBT flags: 0x40290000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.795619:0:6463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279800/0xaf35132e63035593 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 96 type: IBT flags: 0x40290000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.795640:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298644.795651:0:7411:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279800/0xaf35132e63035593 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 95 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.795659:0:7411:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279800/0xaf35132e63035593 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 95 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.795671:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+141:2400:x1866777609629696:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 124us (226us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.795680:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15226 00000001:00080000:2.0:1780298644.795702:0:7411:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991094 00000004:00010000:2.0:1780298644.795729:0:7411:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 000000004049350f reply state 000000008e273309 transno 4294991095 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279800/0xaf35132e63035593 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 95 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.795738:0:7411:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279800/0xaf35132e63035593 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 95 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.795743:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298644.795744:0:7411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1780298644.795746:0:7411:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1780298644.795800:0:7411:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255488a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+178:29360:x1866777609629568:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1600us (1744us total) trans 4294991095 rc 0/0 00000100:00100000:2.0:1780298644.795805:0:7411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62042 00000020:00080000:2.0:1780298644.795809:0:7411:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1780298644.796549:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.796574:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609629824 00000100:00100000:3.0:1780298644.796581:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609629824 00000100:00100000:3.0:1780298644.796617:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10064 00000100:00100000:3.0:1780298644.796625:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255488380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:29360:x1866777609629824:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.796637:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.796651:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e6303559a lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d44fe expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.796677:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e6303553f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2371584->35926015] (req 2371584->2379775) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44aa expref: 7 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.796691:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(28 169) 00010000:00010000:3.0:1780298644.796696:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(29 169) 00010000:00010000:3.0:1780298644.796700:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e6303559a lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44fe expref: 9 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.796723:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e6303553f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2371584->35926015] (req 2371584->2379775) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44aa expref: 7 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.796733:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e6303553f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2371584->35926015] (req 2371584->2379775) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44aa expref: 7 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.796742:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f745180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605048960:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298644.796791:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.797454:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f745180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605048960:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298644.797475:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e6303559a lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44fe expref: 9 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.797489:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156400, rc 0) 00000100:00100000:1.0:1780298644.797530:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255488380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609629824:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 904us (984us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.797539:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10064 00000100:00100000:3.0:1780298644.797694:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.797744:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609629952 00000100:00100000:0.0:1780298644.797751:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609629952 00010000:00010000:0.0:1780298644.797773:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e6303553f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2371584->35926015] (req 2371584->2379775) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44aa expref: 8 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.797803:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15227 00000100:00100000:0.0:1780298644.797812:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609629952:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298644.797824:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298644.797838:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e6303553f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2371584->35926015] (req 2371584->2379775) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44aa expref: 8 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.797849:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e6303553f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2371584->35926015] (req 2371584->2379775) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44aa expref: 8 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.797865:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a36d96d0/0xaf35132e6303553f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2371584->35926015] (req 2371584->2379775) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44aa expref: 8 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.797877:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(29 169) 00010000:00010000:0.0:1780298644.797881:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(29 169) 00010000:00010000:0.0:1780298644.797886:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e6303559a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44fe expref: 8 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.797896:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e6303559a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44fe expref: 8 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.797922:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e6303559a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44fe expref: 8 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.797933:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244813b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777605049088:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298644.797979:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733936c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.798728:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244813b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777605049088:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298644.798742:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298644.798771:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609629952:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 960us (1117us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.798779:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15227 00000100:00100000:3.0:1780298644.798918:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.798958:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609630080 00000100:00100000:0.0:1780298644.798964:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609630080 00000100:00100000:0.0:1780298644.799001:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10065 00000100:00100000:0.0:1780298644.799009:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f744700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:29360:x1866777609630080:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.799030:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.799042:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e630355a1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4505 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.799062:0:19992:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e630354e4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [651264->2306047] (req 651264->659455) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d444f expref: 14 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.799071:0:19992:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000eae09c9b/0xaf35132e63035570 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3989504->37543935] (req 3989504->4001791) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44db expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.799083:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 2 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(28 122) 00010000:00010000:0.0:1780298644.799087:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(30 122) 00010000:00010000:0.0:1780298644.799091:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e630355a1 lrc: 5/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4505 expref: 14 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.799110:0:19992:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000eae09c9b/0xaf35132e63035570 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3989504->37543935] (req 3989504->4001791) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44db expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.799121:0:19992:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000eae09c9b/0xaf35132e63035570 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3989504->37543935] (req 3989504->4001791) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44db expref: 11 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.799156:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244813b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605049216:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298644.799205:0:19992:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e630354e4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [651264->2306047] (req 651264->659455) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d444f expref: 14 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.799216:0:19992:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e630354e4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [651264->2306047] (req 651264->659455) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d444f expref: 14 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.799226:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244813480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605049344:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298644.799252:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf3733936c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.799927:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244813b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605049216:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298644.799938:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244813480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605049344:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298644.799952:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e630355a1 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4505 expref: 14 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.799963:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf600, rc 0) 00000100:00100000:0.0:1780298644.799990:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f744700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:29360:x1866777609630080:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 983us (1074us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.799997:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10065 00000100:00100000:3.0:1780298644.800237:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.800294:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609630208 00000100:00100000:1.0:1780298644.800301:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609630208 00010000:00010000:1.0:1780298644.800324:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000eae09c9b/0xaf35132e63035570 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3989504->37543935] (req 3989504->4001791) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44db expref: 12 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.800356:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15228 00000100:00100000:1.0:1780298644.800366:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f747100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609630208:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298644.800375:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1780298644.800384:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:1.0:1780298644.800393:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000eae09c9b/0xaf35132e63035570 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [3989504->37543935] (req 3989504->4001791) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44db expref: 12 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.800413:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000eae09c9b/0xaf35132e63035570 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [3989504->37543935] (req 3989504->4001791) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44db expref: 12 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.800428:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609630336 00010000:00010000:1.0:1780298644.800432:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eae09c9b/0xaf35132e63035570 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [3989504->37543935] (req 3989504->4001791) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44db expref: 12 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.800433:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609630336 00010000:00010000:1.0:1780298644.800445:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298644.800451:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e630354e4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [651264->2306047] (req 651264->659455) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d444f expref: 14 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.800471:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f747100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609630208:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 108us (236us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.800477:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15229 00000100:00100000:1.0:1780298644.800483:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15228 00000100:00100000:2.0:1780298644.800485:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f746300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609630336:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298644.800493:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298644.800509:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e630354e4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [651264->2306047] (req 651264->659455) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d444f expref: 14 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.800518:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e630354e4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [651264->2306047] (req 651264->659455) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d444f expref: 14 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.800528:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000488d1c2a/0xaf35132e630354e4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [651264->2306047] (req 651264->659455) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d444f expref: 14 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.800538:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(30 122) 00010000:00010000:2.0:1780298644.800540:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(30 122) 00010000:00010000:2.0:1780298644.800543:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e630355a1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4505 expref: 13 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.800549:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e630355a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4505 expref: 13 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.800565:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e630355a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4505 expref: 13 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.800580:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255488000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777605049472:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298644.800625:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1480 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.801201:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255488000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777605049472:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298644.801214:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298644.801246:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f746300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609630336:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 762us (913us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.801254:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15229 00000100:00100000:3.0:1780298644.801385:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.801405:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.801423:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609630592 00000100:00100000:1.0:1780298644.801429:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609630592 00000100:00100000:0.0:1780298644.801438:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609630464 00000100:00100000:0.0:1780298644.801443:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609630464 00002000:00010000:1.0:1780298644.801450:0:8362:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (1580978->18446744073709551615) 00002000:00100000:1.0:1780298644.801454:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f745c00 with x1866777609630592 ext(1580978->18446744073709551615) 00002000:00010000:1.0:1780298644.801460:0:8362:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f745c00. 00002000:00010000:0.0:1780298644.801462:0:19943:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:0.0:1780298644.801466:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f745180 with x1866777609630464 ext(4194304->18446744073709551615) 00002000:00010000:0.0:1780298644.801471:0:19943:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f745180. 00000100:00100000:1.0:1780298644.801486:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6359 00000100:00100000:1.0:1780298644.801496:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609630592:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:0.0:1780298644.801504:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6360 00000100:00100000:0.0:1780298644.801517:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609630464:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:0.0:1780298644.801604:0:19943:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:0.0:1780298644.801607:0:19943:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298644.801610:0:19943:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298644.801612:0:19943:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298644.801615:0:19943:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 400 -> 400 00000100:00100000:0.0:1780298644.801651:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609630464:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 138us (244us total) trans 4294969740 rc 0/0 00000100:00100000:0.0:1780298644.801660:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6360 00002000:00010000:0.0:1780298644.801663:0:19943:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:0.0:1780298644.801665:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f745180 with x1866777609630464 ext(4194304->18446744073709551615) 00002000:00010000:0.0:1780298644.801670:0:19943:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f745180. 00002000:00010000:1.0:1780298644.801769:0:8362:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4157841 -> 1580978 00002000:00010000:1.0:1780298644.801772:0:8362:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298644.801773:0:8362:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298644.801774:0:8362:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298644.801776:0:8362:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 176 -> 88 00000100:00100000:1.0:1780298644.801798:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609630592:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 307us (414us total) trans 4294969290 rc 0/0 00000100:00100000:1.0:1780298644.801806:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6359 00002000:00010000:1.0:1780298644.801808:0:8362:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (1580978->18446744073709551615) 00002000:00100000:1.0:1780298644.801810:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f745c00 with x1866777609630592 ext(1580978->18446744073709551615) 00002000:00010000:1.0:1780298644.801813:0:8362:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f745c00. 00000100:00100000:3.0:1780298644.802556:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.802599:0:6476:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609630720 00000100:00100000:2.0:1780298644.802604:0:6476:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609630720 00000100:00100000:2.0:1780298644.802642:0:6476:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62043 00000100:00100000:2.0:1780298644.802651:0:6476:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f747800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+178:29360:x1866777609630720:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.802664:0:6476:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.802676:0:6476:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2000/0xaf35132e630355a8 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 96 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d450c expref: -99 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.802747:0:6476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3000/0xaf35132e630355af lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 97 type: IBT flags: 0x40000000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.802761:0:6476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3000/0xaf35132e630355af lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 97 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.802769:0:6476:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3000/0xaf35132e630355af lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 97 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298644.802793:0:6476:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 5775282 00000004:00200000:2.0:1780298644.802798:0:6476:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298644.802814:0:6476:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3000/0xaf35132e630355af lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 97 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.802837:0:6476:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2000/0xaf35132e630355a8 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 97 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d450c expref: 180 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.802852:0:6476:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3000/0xaf35132e630355af lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 96 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d450c expref: 179 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.802860:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298644.802863:0:6476:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298644.802866:0:6476:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3000, rc 0) 00000100:00100000:2.0:1780298644.802899:0:6476:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f747800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+179:29360:x1866777609630720:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 250us (345us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.802906:0:6476:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62043 00000100:00100000:2.0:1780298644.804832:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.804862:0:17578:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609630976 00000100:00100000:1.0:1780298644.804871:0:17578:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609630976 00000100:00100000:1.0:1780298644.804900:0:17578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62044 00000100:00100000:1.0:1780298644.804905:0:17578:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255488000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+141:29360:x1866777609630976:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:1.0:1780298644.805022:0:17578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278000/0xaf35132e630355b6 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 97 type: IBT flags: 0x40000000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.805031:0:17578:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3000/0xaf35132e630355af lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 97 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d450c expref: 178 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.805048:0:17578:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278000/0xaf35132e630355b6 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 97 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.805075:0:17578:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3000/0xaf35132e630355af lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 97 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d450c expref: 178 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.805084:0:17578:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3000/0xaf35132e630355af lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 97 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d450c expref: 178 pid: 6476 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.805092:0:17578:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25488aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000_UUID:17578:1866777605049600:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298644.805154:0:17578:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837ae40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.805877:0:17578:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25488aa00 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000_UUID:17578:1866777605049600:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298644.805898:0:17578:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278000/0xaf35132e630355b6 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 97 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.805995:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.806040:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609631104 00000100:00100000:0.0:1780298644.806046:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609631104 00010000:00010000:0.0:1780298644.806067:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3000/0xaf35132e630355af lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 97 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d450c expref: 179 pid: 6476 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.806103:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15230 00000100:00100000:0.0:1780298644.806113:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348af5450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+179:2399:x1866777609631104:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298644.806141:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298644.806148:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3000/0xaf35132e630355af lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 98 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d450c expref: 179 pid: 6476 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.806162:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3000/0xaf35132e630355af lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 98 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d450c expref: 179 pid: 6476 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.806179:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3000/0xaf35132e630355af lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 98 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d450c expref: 179 pid: 6476 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.806194:0:6461:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298644.806199:0:6461:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278000/0xaf35132e630355b6 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 97 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.806213:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278000/0xaf35132e630355b6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 97 type: IBT flags: 0x40290000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.806220:0:6461:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278000/0xaf35132e630355b6 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 97 type: IBT flags: 0x40290000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.806240:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298644.806259:0:17578:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278000/0xaf35132e630355b6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 96 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.806268:0:17578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278000/0xaf35132e630355b6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 96 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.806288:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348af5450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+178:2399:x1866777609631104:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 164us (280us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.806301:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15230 00000001:00080000:1.0:1780298644.806330:0:17578:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991096 00000004:00010000:1.0:1780298644.806365:0:17578:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 0000000073e95522 reply state 00000000a4784873 transno 4294991097 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278000/0xaf35132e630355b6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 96 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.806375:0:17578:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278000/0xaf35132e630355b6 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 96 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.806380:0:17578:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.806382:0:17578:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1780298644.806385:0:17578:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1780298644.806433:0:17578:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255488000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+144:29360:x1866777609630976:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1528us (1601us total) trans 4294991097 rc 0/0 00000100:00100000:1.0:1780298644.806438:0:17578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62044 00000020:00080000:1.0:1780298644.806440:0:17578:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1780298644.807123:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.807181:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609631232 00000100:00100000:0.0:1780298644.807186:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609631232 00000100:00100000:0.0:1780298644.807219:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10066 00000100:00100000:0.0:1780298644.807227:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609631232:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.807237:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.807249:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630355bd lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4513 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.807267:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e6303559a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44fe expref: 9 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.807290:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(29 169) 00010000:00010000:0.0:1780298644.807295:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(30 169) 00010000:00010000:0.0:1780298644.807300:0:20252:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630355bd lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4513 expref: 8 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.807322:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e6303559a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44fe expref: 9 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.807333:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e6303559a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44fe expref: 9 pid: 8359 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.807344:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244811880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605049728:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298644.807385:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.807995:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244811880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605049728:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298644.808010:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630355bd lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4513 expref: 8 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.808036:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf000, rc 0) 00000100:00100000:0.0:1780298644.808068:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:29360:x1866777609631232:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 842us (946us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.808074:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10066 00000100:00100000:2.0:1780298644.808233:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.808266:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609631360 00000100:00100000:0.0:1780298644.808270:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609631360 00010000:00010000:0.0:1780298644.808295:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e6303559a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44fe expref: 10 pid: 8359 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.808317:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15231 00000100:00100000:0.0:1780298644.808322:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b43b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609631360:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298644.808329:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298644.808342:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e6303559a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44fe expref: 10 pid: 8359 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.808349:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e6303559a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44fe expref: 10 pid: 8359 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.808361:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e6303559a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d44fe expref: 10 pid: 8359 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.808373:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(30 169) 00010000:00010000:0.0:1780298644.808375:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(30 169) 00010000:00010000:0.0:1780298644.808379:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630355bd lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4513 expref: 7 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.808384:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630355bd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4513 expref: 7 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.808398:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630355bd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4513 expref: 7 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.808404:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244811880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777605049856:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298644.808433:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.809055:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244811880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777605049856:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298644.809067:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298644.809101:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609631360:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 777us (868us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.809109:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15231 00000100:00100000:3.0:1780298644.809267:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.809322:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609631488 00000100:00100000:2.0:1780298644.809328:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609631488 00000100:00100000:2.0:1780298644.809368:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10067 00000100:00100000:2.0:1780298644.809381:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:29360:x1866777609631488:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.809393:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.809406:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e630355c4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d451a expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.809436:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e630355a1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4505 expref: 13 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.809445:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(30 122) 00010000:00010000:2.0:1780298644.809449:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(31 122) 00010000:00010000:2.0:1780298644.809453:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e630355c4 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d451a expref: 12 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.809475:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e630355a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4505 expref: 13 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.809488:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e630355a1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4505 expref: 13 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.809499:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605049984:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298644.809552:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb16c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.810189:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605049984:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298644.810215:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e630355c4 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d451a expref: 12 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.810229:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3e00, rc 0) 00000100:00100000:0.0:1780298644.810286:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:29360:x1866777609631488:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 896us (1012us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.810296:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10067 00000100:00100000:1.0:1780298644.810398:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.810435:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609631616 00000100:00100000:3.0:1780298644.810444:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609631616 00010000:00010000:3.0:1780298644.810478:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e630355a1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4505 expref: 14 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.810523:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15232 00000100:00100000:3.0:1780298644.810533:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25488aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609631616:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298644.810545:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298644.810564:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e630355a1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4505 expref: 14 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.810577:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e630355a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4505 expref: 14 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.810598:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e630355a1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4505 expref: 14 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.810612:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(31 122) 00010000:00010000:3.0:1780298644.810617:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(31 122) 00010000:00010000:3.0:1780298644.810623:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e630355c4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d451a expref: 11 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.810632:0:6463:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e630355c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d451a expref: 11 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.810665:0:6463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e630355c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d451a expref: 11 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.810684:0:6463:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f747100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777605050112:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298644.810748:0:6463:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.811410:0:6463:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f747100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777605050112:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298644.811422:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298644.811448:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25488aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609631616:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 919us (1052us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.811456:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15232 00000100:00100000:3.0:1780298644.811589:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.811714:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.811724:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609631744 00000100:00100000:2.0:1780298644.811730:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609631744 00002000:00010000:2.0:1780298644.811753:0:19951:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:2.0:1780298644.811757:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f746300 with x1866777609631744 ext(4194304->18446744073709551615) 00000100:00100000:1.0:1780298644.811762:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609631872 00002000:00010000:2.0:1780298644.811765:0:19951:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f746300. 00000100:00100000:1.0:1780298644.811769:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609631872 00002000:00010000:1.0:1780298644.811792:0:8363:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4103694->18446744073709551615) 00000100:00100000:2.0:1780298644.811794:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6361 00002000:00100000:1.0:1780298644.811799:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f747100 with x1866777609631872 ext(4103694->18446744073709551615) 00002000:00010000:1.0:1780298644.811806:0:8363:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f747100. 00000100:00100000:2.0:1780298644.811807:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609631744:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:1.0:1780298644.811835:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6362 00000100:00100000:1.0:1780298644.811844:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f747100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609631872:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:2.0:1780298644.811938:0:19951:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:2.0:1780298644.811941:0:19951:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298644.811944:0:19951:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298644.811946:0:19951:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298644.811947:0:19951:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 400 -> 400 00000100:00100000:2.0:1780298644.811979:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609631744:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 175us (391us total) trans 4294969741 rc 0/0 00000100:00100000:2.0:1780298644.811990:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6361 00002000:00010000:2.0:1780298644.811994:0:19951:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:2.0:1780298644.811997:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f746300 with x1866777609631744 ext(4194304->18446744073709551615) 00002000:00010000:2.0:1780298644.812004:0:19951:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f746300. 00002000:00010000:1.0:1780298644.812018:0:8363:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 1580978 -> 4103694 00002000:00010000:1.0:1780298644.812023:0:8363:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298644.812026:0:8363:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298644.812029:0:8363:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298644.812032:0:8363:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 88 -> 88 00000100:00100000:1.0:1780298644.812067:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f747100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609631872:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 225us (353us total) trans 4294969291 rc 0/0 00000100:00100000:1.0:1780298644.812074:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6362 00002000:00010000:1.0:1780298644.812077:0:8363:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4103694->18446744073709551615) 00002000:00100000:1.0:1780298644.812080:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f747100 with x1866777609631872 ext(4103694->18446744073709551615) 00002000:00010000:1.0:1780298644.812086:0:8363:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f747100. 00000100:00100000:3.0:1780298644.812880:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.812932:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609632000 00000100:00100000:2.0:1780298644.812939:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609632000 00000100:00100000:2.0:1780298644.812987:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62045 00000100:00100000:2.0:1780298644.812999:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348318e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+144:29360:x1866777609632000:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.813029:0:6477:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.813045:0:6477:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2600/0xaf35132e630355cb lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 97 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4521 expref: -99 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.813174:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2000/0xaf35132e630355d2 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 98 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.813197:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2000/0xaf35132e630355d2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 98 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.813208:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2000/0xaf35132e630355d2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 98 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298644.813237:0:6477:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 8297998 00000004:00200000:2.0:1780298644.813243:0:6477:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298644.813263:0:6477:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2000/0xaf35132e630355d2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 98 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.813307:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2600/0xaf35132e630355cb lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 98 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4521 expref: 146 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.813325:0:6477:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2000/0xaf35132e630355d2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 97 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4521 expref: 145 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.813334:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298644.813337:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298644.813340:0:6477:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2000, rc 0) 00000100:00100000:2.0:1780298644.813373:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348318e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+145:29360:x1866777609632000:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 378us (493us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.813382:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62045 00000100:00100000:2.0:1780298644.815026:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.815087:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609632128 00000100:00100000:1.0:1780298644.815093:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609632128 00002000:00100000:1.0:1780298644.815167:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (2805760->2809855) req@ffff9cf242b41180 x1866777609632128/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.815178:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b41180 with x1866777609632128 ext(2805760->2809855) 00002000:00010000:1.0:1780298644.815184:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf242b41180 00000100:00100000:1.0:1780298644.815209:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6363 00000100:00100000:1.0:1780298644.815217:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b41180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2399:x1866777609632128:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298644.815413:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b41180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2399:x1866777609632128:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 196us (390us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298644.815423:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6363 00002000:00100000:1.0:1780298644.815430:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (2805760->2809855) req@ffff9cf242b41180 x1866777609632128/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/4536 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.815439:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b41180 with x1866777609632128 ext(2805760->2809855) 00002000:00010000:1.0:1780298644.815444:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf242b41180 00000100:00100000:3.0:1780298644.817265:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.817367:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609632256 00000100:00100000:1.0:1780298644.817373:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609632256 00002000:00100000:1.0:1780298644.817399:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (3866624->3870719) req@ffff9cf348318000 x1866777609632256/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.817408:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf348318000 with x1866777609632256 ext(3866624->3870719) 00002000:00010000:1.0:1780298644.817412:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf348318000 00000100:00100000:1.0:1780298644.817436:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6364 00000100:00100000:1.0:1780298644.817443:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348318000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609632256:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298644.817553:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348318000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609632256:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 110us (295us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298644.817560:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6364 00002000:00100000:1.0:1780298644.817567:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (3866624->3870719) req@ffff9cf348318000 x1866777609632256/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/4536 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.817576:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf348318000 with x1866777609632256 ext(3866624->3870719) 00002000:00010000:1.0:1780298644.817579:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf348318000 00000100:00100000:3.0:1780298644.818578:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.818632:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609632384 00000100:00100000:1.0:1780298644.818639:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609632384 00002000:00100000:1.0:1780298644.818661:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (3870720->3874815) req@ffff9cf34831b480 x1866777609632384/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.818670:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf34831b480 with x1866777609632384 ext(3870720->3874815) 00002000:00010000:1.0:1780298644.818674:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf34831b480 00000100:00100000:1.0:1780298644.818695:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6365 00000100:00100000:1.0:1780298644.818704:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34831b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609632384:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298644.818798:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34831b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609632384:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 96us (223us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298644.818806:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6365 00002000:00100000:1.0:1780298644.818814:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (3870720->3874815) req@ffff9cf34831b480 x1866777609632384/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/4536 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.818823:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf34831b480 with x1866777609632384 ext(3870720->3874815) 00002000:00010000:1.0:1780298644.818827:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf34831b480 00000100:00100000:2.0:1780298644.819827:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.819866:0:7411:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609632512 00000100:00100000:1.0:1780298644.819872:0:7411:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609632512 00000100:00100000:1.0:1780298644.819903:0:7411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62046 00000100:00100000:1.0:1780298644.819908:0:7411:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b40000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+178:29360:x1866777609632512:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.819918:0:7411:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.819935:0:7411:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279a00/0xaf35132e630355d9 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 98 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4528 expref: -99 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.820025:0:7411:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279c00/0xaf35132e630355e0 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 99 type: IBT flags: 0x40000000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.820041:0:7411:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279c00/0xaf35132e630355e0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 99 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.820052:0:7411:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279c00/0xaf35132e630355e0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 99 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1780298644.820074:0:7411:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 8297998 00000004:00200000:1.0:1780298644.820078:0:7411:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1780298644.820092:0:7411:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279c00/0xaf35132e630355e0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 99 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.820111:0:7411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279a00/0xaf35132e630355d9 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 99 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4528 expref: 180 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.820164:0:7411:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279c00/0xaf35132e630355e0 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 98 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4528 expref: 179 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.820170:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.820173:0:7411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1780298644.820176:0:7411:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279c00, rc 0) 00000100:00100000:1.0:1780298644.820224:0:7411:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b40000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+179:29360:x1866777609632512:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 315us (400us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.820229:0:7411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62046 00000100:00100000:1.0:1780298644.821250:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.821348:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609632640 00000100:00100000:3.0:1780298644.821359:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609632640 00000100:00100000:2.0:1780298644.821375:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.821412:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609632768 00000100:00100000:1.0:1780298644.821416:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609632768 00000100:00100000:3.0:1780298644.821447:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10068 00000100:00100000:1.0:1780298644.821467:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10069 00000100:00100000:3.0:1780298644.821468:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25488b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609632640:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298644.821486:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609632768:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.821490:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.821498:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.821509:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e630355e7 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d452f expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.821512:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e630355ee lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4536 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.821623:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25488ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605050240:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.821632:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf348318380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605050368:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.821715:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.821716:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837af00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.822642:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348318380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605050368:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.822647:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25488ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605050240:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.822681:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e630355e7 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d452f expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.822681:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e630355ee lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4536 expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.822714:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e630355e7 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d452f expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.822731:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e630355ee lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4536 expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.822736:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156a00, rc 0) 00010000:00010000:2.0:1780298644.822753:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278400, rc 0) 00000100:00100000:3.0:1780298644.822797:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25488b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609632640:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1332us (1538us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.822797:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609632768:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1309us (1416us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.822830:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10069 00000100:00100000:3.0:1780298644.822839:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10068 00000100:00100000:2.0:1780298644.823771:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.823817:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609632896 00000100:00100000:0.0:1780298644.823822:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609632896 00000100:00100000:0.0:1780298644.823853:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10070 00000100:00100000:0.0:1780298644.823862:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609632896:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.823871:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298644.823880:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:0.0:1780298644.823880:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e630355f5 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d453d expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.823914:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244811880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605050496:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.823924:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609633024 00000100:00100000:2.0:1780298644.823928:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609633024 00000100:00100000:2.0:1780298644.823962:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10071 00000100:00100000:2.0:1780298644.823974:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254888a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609633024:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298644.823978:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00010000:00010000:2.0:1780298644.823983:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.823993:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e630355fc lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4544 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.824025:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605050624:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.824064:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb10c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.824605:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244811880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605050496:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.824619:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e630355f5 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d453d expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.824629:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e630355f5 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d453d expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.824638:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae400, rc 0) 00000100:00100000:0.0:1780298644.824659:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609632896:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 800us (890us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.824671:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10070 00000100:00100000:0.0:1780298644.824834:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605050624:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.824863:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e630355fc lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4544 expref: 14 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.824882:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d4cf86a3/0xaf35132e630355fc lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4544 expref: 14 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.824895:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2600, rc 0) 00000100:00100000:0.0:1780298644.824931:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254888a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609633024:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 963us (1051us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.824939:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10071 00000100:00100000:3.0:1780298644.825892:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.825936:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609633152 00000100:00100000:0.0:1780298644.825942:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609633152 00000100:00100000:0.0:1780298644.825990:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6366 00000100:00100000:0.0:1780298644.826001:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348318380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609633152:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298644.826293:0:16972:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:0.0:1780298644.826298:0:16972:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298644.826300:0:16972:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298644.826302:0:16972:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298644.826305:0:16972:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 400 -> 424 00000100:00100000:0.0:1780298644.826343:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348318380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609633152:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 345us (452us total) trans 4294969742 rc 0/0 00000100:00100000:0.0:1780298644.826353:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6366 00000100:00100000:1.0:1780298644.827335:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.827383:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609633280 00000100:00100000:0.0:1780298644.827390:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609633280 00000100:00100000:0.0:1780298644.827433:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10072 00000100:00100000:0.0:1780298644.827442:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254889c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609633280:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.827453:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.827464:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005d10d3aa/0xaf35132e63035603 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d454b expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.827487:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.827504:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244811880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605050752:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.827535:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609633408 00000100:00100000:1.0:1780298644.827540:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609633408 00000100:00100000:0.0:1780298644.827565:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.827578:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10073 00000100:00100000:1.0:1780298644.827588:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34831a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609633408:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.827597:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.827607:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e6303560a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4552 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.827636:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254889180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605050880:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.827674:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a840 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.828306:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244811880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605050752:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.828329:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005d10d3aa/0xaf35132e63035603 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d454b expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.828345:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005d10d3aa/0xaf35132e63035603 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d454b expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.828356:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baec00, rc 0) 00000100:00100000:0.0:1780298644.828392:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254889c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609633280:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 951us (1059us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.828404:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10072 00000100:00100000:1.0:1780298644.828456:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254889180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605050880:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.828475:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e6303560a lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4552 expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.828489:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e6303560a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4552 expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.828497:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279a00, rc 0) 00000100:00100000:1.0:1780298644.828525:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34831a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609633408:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 940us (1042us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.828531:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10073 00000100:00100000:2.0:1780298644.829469:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.829501:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.829523:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609633536 00000100:00100000:1.0:1780298644.829528:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609633536 00000100:00100000:0.0:1780298644.829540:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609633664 00000100:00100000:0.0:1780298644.829545:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609633664 00000100:00100000:1.0:1780298644.829567:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10074 00000100:00100000:0.0:1780298644.829575:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10075 00000100:00100000:1.0:1780298644.829580:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609633536:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298644.829586:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609633664:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.829589:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.829594:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.829599:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63035611 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4559 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.829602:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c37b15a2/0xaf35132e63035618 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4560 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.829651:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244811880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605051136:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.829653:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254889180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605051008:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.829686:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.829689:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.830466:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254889180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605051008:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.830485:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63035611 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4559 expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.830489:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244811880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605051136:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.830509:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63035611 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4559 expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.830515:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c37b15a2/0xaf35132e63035618 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4560 expref: 14 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.830523:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279200, rc 0) 00010000:00010000:0.0:1780298644.830537:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c37b15a2/0xaf35132e63035618 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4560 expref: 14 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.830548:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baee00, rc 0) 00000100:00100000:1.0:1780298644.830559:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609633536:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 984us (1090us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.830567:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10074 00000100:00100000:0.0:1780298644.830573:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609633664:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 989us (1071us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.830582:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10075 00000100:00100000:3.0:1780298644.832094:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.832164:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609633792 00000100:00100000:1.0:1780298644.832170:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609633792 00002000:00100000:1.0:1780298644.832205:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (16384->20479) req@ffff9cf348319180 x1866777609633792/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.832217:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf348319180 with x1866777609633792 ext(16384->20479) 00002000:00010000:1.0:1780298644.832222:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf348319180 00000100:00100000:1.0:1780298644.832245:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6367 00000100:00100000:1.0:1780298644.832255:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348319180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609633792:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298644.832402:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348319180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609633792:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 151us (312us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298644.832409:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6367 00002000:00100000:1.0:1780298644.832422:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (16384->20479) req@ffff9cf348319180 x1866777609633792/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/4536 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.832431:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf348319180 with x1866777609633792 ext(16384->20479) 00002000:00010000:1.0:1780298644.832435:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf348319180 00000100:00100000:2.0:1780298644.833513:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.833555:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609633920 00000100:00100000:1.0:1780298644.833560:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609633920 00002000:00100000:1.0:1780298644.833597:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (20480->24575) req@ffff9cf34372dc00 x1866777609633920/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.833606:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf34372dc00 with x1866777609633920 ext(20480->24575) 00002000:00010000:1.0:1780298644.833610:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf34372dc00 00000100:00100000:1.0:1780298644.833631:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6368 00000100:00100000:1.0:1780298644.833638:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609633920:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298644.833703:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609633920:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 65us (192us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298644.833710:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6368 00002000:00100000:1.0:1780298644.833718:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (20480->24575) req@ffff9cf34372dc00 x1866777609633920/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/4536 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.833725:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf34372dc00 with x1866777609633920 ext(20480->24575) 00002000:00010000:1.0:1780298644.833727:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf34372dc00 00000100:00100000:3.0:1780298644.834771:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.834796:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.834813:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609634048 00000100:00100000:2.0:1780298644.834817:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609634048 00000100:00100000:0.0:1780298644.834834:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609634176 00000100:00100000:0.0:1780298644.834840:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609634176 00000100:00100000:2.0:1780298644.834850:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10076 00000100:00100000:2.0:1780298644.834858:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348318a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609634048:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.834867:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.834877:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e6303561f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4567 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.834885:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10077 00000100:00100000:1.0:1780298644.834912:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34831aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609634176:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298644.834930:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605051264:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.834935:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.834955:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e63035626 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d456e expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.834985:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1480 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.835044:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254889180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605051392:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.835104:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a540 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.835773:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605051264:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.835804:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e6303561f lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4567 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.835822:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e6303561f lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4567 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.835834:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3000, rc 0) 00000100:00100000:1.0:1780298644.835834:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254889180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605051392:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.835856:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e63035626 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d456e expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.835870:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348318a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609634048:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1012us (1098us total) trans 0 rc 301/301 00010000:00010000:1.0:1780298644.835872:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e63035626 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d456e expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.835887:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10076 00010000:00010000:1.0:1780298644.835890:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279a00, rc 0) 00000100:00100000:1.0:1780298644.835916:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34831aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609634176:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1007us (1119us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.835923:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10077 00000100:00100000:2.0:1780298644.836938:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.837012:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609634304 00000100:00100000:3.0:1780298644.837018:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609634304 00000100:00100000:3.0:1780298644.837048:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10078 00000100:00100000:3.0:1780298644.837054:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609634304:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.837063:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298644.837068:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:3.0:1780298644.837074:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e6303562d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4575 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.837106:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf348318700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605051520:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.837112:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609634432 00000100:00100000:0.0:1780298644.837116:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609634432 00000100:00100000:0.0:1780298644.837167:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10079 00000100:00100000:0.0:1780298644.837174:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254889180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609634432:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.837181:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.837188:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63035634 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d457c expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.837212:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244811880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605051648:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.837254:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.837256:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.837956:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244811880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605051648:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.837977:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63035634 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d457c expref: 14 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.838008:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63035634 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d457c expref: 14 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.838020:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae400, rc 0) 00000100:00100000:0.0:1780298644.838057:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254889180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609634432:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 882us (991us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.838066:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10079 00000100:00100000:2.0:1780298644.838077:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348318700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605051520:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.838101:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e6303562d lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4575 expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.838116:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a58d48e9/0xaf35132e6303562d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4575 expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.838142:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156a00, rc 0) 00000100:00100000:2.0:1780298644.838196:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609634304:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1142us (1258us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.838203:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10078 00000100:00100000:1.0:1780298644.839092:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.839117:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.839147:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609634560 00000100:00100000:3.0:1780298644.839152:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609634560 00000100:00100000:3.0:1780298644.839177:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609634688 00000100:00100000:3.0:1780298644.839181:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609634688 00000100:00100000:1.0:1780298644.839194:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10080 00000100:00100000:3.0:1780298644.839201:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10081 00000100:00100000:1.0:1780298644.839203:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25488ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609634560:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298644.839206:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254888380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609634688:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.839211:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.839212:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.839219:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e0b8e7b/0xaf35132e6303563b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d458a expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.839219:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63035642 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4583 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.839246:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433dca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605051776:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.839247:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34831bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605051904:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.839293:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a540 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.839328:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.839989:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433dca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605051776:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.840011:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63035642 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4583 expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.840028:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34831bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605051904:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.840032:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63035642 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4583 expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.840045:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279200, rc 0) 00010000:00010000:2.0:1780298644.840053:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009e0b8e7b/0xaf35132e6303563b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d458a expref: 14 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.840070:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e0b8e7b/0xaf35132e6303563b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d458a expref: 14 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.840073:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25488ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609634560:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 870us (979us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.840088:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10080 00010000:00010000:2.0:1780298644.840091:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156c00, rc 0) 00000100:00100000:2.0:1780298644.840115:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254888380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609634688:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 907us (996us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.840122:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10081 00000100:00100000:3.0:1780298644.840975:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.840994:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.841015:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609634816 00000100:00100000:2.0:1780298644.841021:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609634816 00000100:00100000:0.0:1780298644.841024:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609634944 00000100:00100000:0.0:1780298644.841027:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609634944 00000100:00100000:0.0:1780298644.841052:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10082 00000100:00100000:2.0:1780298644.841057:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10083 00000100:00100000:0.0:1780298644.841060:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348318700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609634944:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298644.841064:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34831bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609634816:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.841068:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.841070:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.841076:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63035650 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4591 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.841076:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e63035649 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4598 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.841103:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605052160:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.841103:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244811880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605052032:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.841160:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1480 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.841160:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.841846:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244811880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605052032:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.841864:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605052160:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.841867:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e63035649 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4598 expref: 14 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.841887:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63035650 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4591 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.841891:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e63035649 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4598 expref: 14 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.841901:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baec00, rc 0) 00010000:00010000:2.0:1780298644.841925:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63035650 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4591 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.841936:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348318700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609634944:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 874us (936us total) trans 0 rc 301/301 00010000:00010000:2.0:1780298644.841939:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3000, rc 0) 00000100:00100000:0.0:1780298644.841946:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10082 00000100:00100000:2.0:1780298644.841970:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34831bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609634816:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 905us (993us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.841979:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10083 00000100:00100000:1.0:1780298644.842863:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.842899:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609635072 00000100:00100000:2.0:1780298644.842905:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609635072 00000100:00100000:2.0:1780298644.842935:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10084 00000100:00100000:2.0:1780298644.842944:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609635072:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.842953:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.842961:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63035657 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d459f expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.842981:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630355bd lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4513 expref: 9 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.842991:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(30 169) 00010000:00010000:2.0:1780298644.842995:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(31 169) 00010000:00010000:2.0:1780298644.843001:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63035657 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2973696->2977791] (req 2973696->2977791) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d459f expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.843023:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630355bd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4513 expref: 9 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.843038:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630355bd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4513 expref: 9 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.843049:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605052288:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298644.843086:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1480 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.843645:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605052288:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298644.843656:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63035657 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2973696->2977791] (req 2973696->2977791) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d459f expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.843674:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2600, rc 0) 00000100:00100000:2.0:1780298644.843705:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609635072:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 762us (843us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.843712:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10084 00000100:00100000:3.0:1780298644.844055:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.844097:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609635200 00000100:00100000:1.0:1780298644.844103:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609635200 00002000:00100000:1.0:1780298644.844164:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (3866624->3874815) req@ffff9cf2548b8700 x1866777609635200/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 8680/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.844181:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2548b8700 with x1866777609635200 ext(3866624->3874815) 00010000:00010000:1.0:1780298644.844188:0:19949:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630355bd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4513 expref: 10 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.844202:0:19949:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630355bd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4513 expref: 10 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1780298644.844210:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf2548b8700 00000100:00100000:1.0:1780298644.844233:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 211 00000100:00100000:1.0:1780298644.844241:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609635200:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:1.0:1780298644.844745:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609635200:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 505us (689us total) trans 4294969743 rc 0/0 00000100:00100000:1.0:1780298644.844753:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 211 00002000:00100000:1.0:1780298644.844761:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (3866624->3874815) req@ffff9cf2548b8700 x1866777609635200/t4294969743(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 8680/448 e 0 to 0 dl 1780298655 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.844770:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2548b8700 with x1866777609635200 ext(3866624->3874815) 00010000:00010000:1.0:1780298644.844776:0:19949:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630355bd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4513 expref: 12 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.844790:0:19949:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630355bd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4513 expref: 12 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1780298644.844797:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf2548b8700 00000100:00100000:1.0:1780298644.845575:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.845611:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609635328 00000100:00100000:2.0:1780298644.845616:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609635328 00010000:00010000:2.0:1780298644.845631:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630355bd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4513 expref: 12 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.845652:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15233 00000100:00100000:2.0:1780298644.845657:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433df480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609635328:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298644.845664:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:2.0:1780298644.845672:0:20253:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 424 -> 440 00010000:00010000:2.0:1780298644.845676:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630355bd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4513 expref: 12 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.845684:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630355bd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4513 expref: 12 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.845698:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630355bd lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4513 expref: 12 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.845710:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(31 169) 00010000:00010000:2.0:1780298644.845714:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(31 169) 00010000:00010000:2.0:1780298644.845722:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63035657 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2973696->36528127] (req 2973696->2977791) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d459f expref: 9 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.845731:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63035657 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2973696->36528127] (req 2973696->2977791) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d459f expref: 9 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.845755:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63035657 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2973696->36528127] (req 2973696->2977791) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d459f expref: 9 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.845766:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372fb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777605052416:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298644.845804:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1480 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.846657:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372fb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777605052416:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298644.846671:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298644.846701:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433df480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609635328:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1043us (1125us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.846713:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15233 00000100:00100000:3.0:1780298644.846881:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.846917:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609635456 00000100:00100000:1.0:1780298644.846922:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609635456 00000100:00100000:1.0:1780298644.846959:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6369 00000100:00100000:1.0:1780298644.846968:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609635456:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298644.847079:0:8362:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:1.0:1780298644.847082:0:8362:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298644.847085:0:8362:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298644.847087:0:8362:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298644.847089:0:8362:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 440 -> 440 00000100:00100000:1.0:1780298644.847117:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609635456:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 151us (236us total) trans 4294969744 rc 0/0 00000100:00100000:1.0:1780298644.847139:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6369 00000100:00100000:1.0:1780298644.848092:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.848137:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609635584 00000100:00100000:2.0:1780298644.848142:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609635584 00000100:00100000:2.0:1780298644.848174:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10085 00000100:00100000:2.0:1780298644.848183:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609635584:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.848193:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.848203:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e6303565e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d45a6 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.848233:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605052544:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.848264:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1480 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.849151:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605052544:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.849175:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e6303565e lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45a6 expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.849192:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e6303565e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45a6 expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.849204:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2e00, rc 0) 00000100:00100000:2.0:1780298644.849237:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609635584:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1056us (1147us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.849245:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10085 00000100:00100000:2.0:1780298644.850238:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.850294:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609635712 00000100:00100000:3.0:1780298644.850306:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609635712 00000100:00100000:3.0:1780298644.850363:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10086 00000100:00100000:3.0:1780298644.850374:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609635712:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.850391:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.850407:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004ba675d2/0xaf35132e63035665 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d45ad expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.850469:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548bbb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605052672:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.850542:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.851349:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548bbb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605052672:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.851375:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004ba675d2/0xaf35132e63035665 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45ad expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.851393:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004ba675d2/0xaf35132e63035665 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45ad expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.851406:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156e00, rc 0) 00000100:00100000:3.0:1780298644.851447:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609635712:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1076us (1209us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.851454:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10086 00000100:00100000:0.0:1780298644.852311:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.852343:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609635840 00000100:00100000:2.0:1780298644.852348:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609635840 00000100:00100000:2.0:1780298644.852374:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10087 00000100:00100000:2.0:1780298644.852380:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e4f0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:29360:x1866777609635840:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.852388:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.852397:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e6303566c lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d45b4 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.852410:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(31 169) 00010000:00010000:2.0:1780298644.852413:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(31 169) 00010000:00010000:2.0:1780298644.852420:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e6303566c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1097728->2973695] (req 1097728->1105919) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45b4 expref: 12 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.852430:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e6303566c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1097728->2973695] (req 1097728->1105919) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45b4 expref: 12 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.852436:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3600, rc 0) 00000100:00100000:2.0:1780298644.852462:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e4f0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:29360:x1866777609635840:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 83us (153us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.852467:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10087 00000100:00100000:2.0:1780298644.853178:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.853219:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609635968 00000100:00100000:0.0:1780298644.853226:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609635968 00000100:00100000:0.0:1780298644.853268:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6370 00000100:00100000:0.0:1780298644.853286:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609635968:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298644.853407:0:16972:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:0.0:1780298644.853410:0:16972:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298644.853412:0:16972:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298644.853413:0:16972:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298644.853415:0:16972:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 440 -> 440 00000100:00100000:0.0:1780298644.853447:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2399:x1866777609635968:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 174us (270us total) trans 4294969745 rc 0/0 00000100:00100000:0.0:1780298644.853458:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6370 00000100:00100000:1.0:1780298644.854601:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.854645:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609636096 00000100:00100000:2.0:1780298644.854651:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609636096 00000100:00100000:2.0:1780298644.854680:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10088 00000100:00100000:2.0:1780298644.854687:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609636096:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.854697:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.854709:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e63035673 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d45bb expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.854752:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.854755:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605052800:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.854815:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1480 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.854821:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609636224 00000100:00100000:0.0:1780298644.854829:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609636224 00000100:00100000:0.0:1780298644.854873:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10089 00000100:00100000:0.0:1780298644.854885:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609636224:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.854906:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.854919:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e6303567a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d45c2 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.854959:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244811880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605052928:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.855006:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.855584:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605052800:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.855610:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e63035673 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45bb expref: 12 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.855628:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e63035673 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45bb expref: 12 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.855638:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2400, rc 0) 00000100:00100000:2.0:1780298644.855665:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609636096:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 980us (1065us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.855671:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10088 00000100:00100000:0.0:1780298644.855814:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244811880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605052928:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.855836:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e6303567a lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45c2 expref: 14 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.855849:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d10d3aa/0xaf35132e6303567a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45c2 expref: 14 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.855857:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baec00, rc 0) 00000100:00100000:0.0:1780298644.855883:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609636224:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1004us (1135us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.855889:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10089 00000100:00100000:1.0:1780298644.856941:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.857032:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609636352 00000100:00100000:3.0:1780298644.857038:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609636352 00000100:00100000:3.0:1780298644.857081:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10090 00000100:00100000:3.0:1780298644.857094:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609636352:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.857105:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298644.857111:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:3.0:1780298644.857119:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e63035681 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d45c9 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.857174:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609636480 00000100:00100000:2.0:1780298644.857179:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609636480 00000100:00100000:2.0:1780298644.857207:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10091 00000100:00100000:2.0:1780298644.857215:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244811880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609636480:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.857222:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.857231:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e63035688 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d45d0 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.857243:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548b9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605053056:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.857257:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605053184:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.857317:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.857318:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.858056:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605053184:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.858060:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548b9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605053056:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.858077:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e63035688 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45d0 expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.858085:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e63035681 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45c9 expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.858098:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000887cd623/0xaf35132e63035688 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45d0 expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.858107:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e63035681 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45c9 expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.858107:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3000, rc 0) 00010000:00010000:3.0:1780298644.858121:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157000, rc 0) 00000100:00100000:2.0:1780298644.858157:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244811880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609636480:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 943us (1048us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.858162:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10091 00000100:00100000:3.0:1780298644.858181:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609636352:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1090us (1238us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.858190:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10090 00000100:00100000:1.0:1780298644.859119:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.859181:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609636608 00000100:00100000:0.0:1780298644.859187:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609636608 00000100:00100000:0.0:1780298644.859219:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10092 00000100:00100000:0.0:1780298644.859226:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:29360:x1866777609636608:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.859235:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.859244:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e6303568f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d45d7 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.859270:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e630355c4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d451a expref: 12 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.859292:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(31 122) 00010000:00010000:0.0:1780298644.859296:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(32 122) 00010000:00010000:0.0:1780298644.859302:0:20252:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e6303568f lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3088384->3092479] (req 3088384->3092479) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45d7 expref: 14 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.859328:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e630355c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d451a expref: 12 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.859339:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e630355c4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d451a expref: 12 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.859348:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244813480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605053312:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298644.859397:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.860072:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244813480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605053312:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298644.860091:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e6303568f lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3088384->3092479] (req 3088384->3092479) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45d7 expref: 14 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.860102:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae000, rc 0) 00000100:00100000:1.0:1780298644.860163:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:29360:x1866777609636608:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 935us (1042us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.860174:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10092 00000100:00100000:1.0:1780298644.860470:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.860508:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609636736 00000100:00100000:2.0:1780298644.860513:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609636736 00010000:00010000:2.0:1780298644.860533:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e630355c4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d451a expref: 13 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.860556:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15234 00000100:00100000:2.0:1780298644.860561:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dc000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609636736:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298644.860571:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298644.860583:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e630355c4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d451a expref: 13 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.860591:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e630355c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d451a expref: 13 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.860607:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc4b2e1a/0xaf35132e630355c4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d451a expref: 13 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.860620:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(32 122) 00010000:00010000:2.0:1780298644.860623:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(32 122) 00010000:00010000:2.0:1780298644.860629:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e6303568f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3088384->36642815] (req 3088384->3092479) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45d7 expref: 13 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.860636:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e6303568f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3088384->36642815] (req 3088384->3092479) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45d7 expref: 13 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.860655:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e6303568f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3088384->36642815] (req 3088384->3092479) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45d7 expref: 13 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.860663:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372f100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:8374:1866777605053440:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298644.860702:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1480 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.861422:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372f100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:8374:1866777605053440:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298644.861433:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298644.861458:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433dc000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609636736:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 897us (989us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.861463:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15234 00000100:00100000:0.0:1780298644.861560:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.861597:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609636864 00000100:00100000:1.0:1780298644.861600:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609636864 00000100:00100000:1.0:1780298644.861629:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6371 00000100:00100000:1.0:1780298644.861634:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244813b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609636864:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298644.861803:0:8363:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4103694 -> 4103694 00002000:00010000:1.0:1780298644.861806:0:8363:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298644.861808:0:8363:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298644.861809:0:8363:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298644.861811:0:8363:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 88 -> 96 00000100:00100000:1.0:1780298644.861833:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244813b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2400:x1866777609636864:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 199us (274us total) trans 4294969292 rc 0/0 00000100:00100000:1.0:1780298644.861837:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6371 00000100:00100000:0.0:1780298644.862958:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.862978:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.863001:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609636992 00000100:00100000:2.0:1780298644.863007:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609636992 00000100:00100000:2.0:1780298644.863041:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609637120 00000100:00100000:2.0:1780298644.863044:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609637120 00000100:00100000:1.0:1780298644.863053:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10093 00000100:00100000:1.0:1780298644.863064:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244810700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609636992:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.863075:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298644.863080:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10094 00010000:00010000:1.0:1780298644.863088:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63035696 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d45de expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.863094:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244813480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2400:x1866777609637120:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.863109:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.863120:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004ba675d2/0xaf35132e6303569d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d45e5 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.863157:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433de300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605053568:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.863188:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548b9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605053696:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.863214:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a540 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.863223:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.863991:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548b9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605053696:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.864010:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433de300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605053568:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.864020:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004ba675d2/0xaf35132e6303569d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45e5 expref: 16 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.864026:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63035696 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45de expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.864044:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63035696 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45de expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.864047:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004ba675d2/0xaf35132e6303569d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45e5 expref: 16 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.864054:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279200, rc 0) 00010000:00010000:3.0:1780298644.864064:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156e00, rc 0) 00000100:00100000:1.0:1780298644.864082:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244810700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609636992:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1021us (1124us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.864088:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10093 00000100:00100000:3.0:1780298644.864093:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244813480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2400:x1866777609637120:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1002us (1111us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.864100:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10094 00000100:00100000:0.0:1780298644.865328:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.865418:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609637376 00000100:00100000:3.0:1780298644.865424:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609637376 00000100:00100000:3.0:1780298644.865466:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10095 00000100:00100000:3.0:1780298644.865475:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244812680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2397:x1866777609637376:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298644.865482:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:3.0:1780298644.865489:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.865501:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bed20494/0xaf35132e630356a4 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d45f3 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.865528:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609637248 00000100:00100000:1.0:1780298644.865533:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609637248 00000100:00100000:1.0:1780298644.865561:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10096 00000100:00100000:1.0:1780298644.865567:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609637248:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.865580:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298644.865581:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548b9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605053824:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.865590:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f4922155/0xaf35132e630356ab lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d45ec expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.865623:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433de300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605053952:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.865633:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.865652:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a540 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.866445:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548b9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605053824:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.866465:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bed20494/0xaf35132e630356a4 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45f3 expref: 16 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.866481:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bed20494/0xaf35132e630356a4 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45f3 expref: 16 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.866491:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157000, rc 0) 00000100:00100000:1.0:1780298644.866521:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433de300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605053952:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.866539:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f4922155/0xaf35132e630356ab lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45ec expref: 12 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.866560:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f4922155/0xaf35132e630356ab lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45ec expref: 12 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.866564:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244812680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2397:x1866777609637376:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1088us (1234us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.866581:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10095 00010000:00010000:1.0:1780298644.866582:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278600, rc 0) 00000100:00100000:1.0:1780298644.866608:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609637248:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1040us (1130us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.866616:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10096 00000100:00100000:0.0:1780298644.868329:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.868359:0:6476:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609637632 00000100:00100000:2.0:1780298644.868365:0:6476:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609637632 00000100:00100000:2.0:1780298644.868400:0:6476:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62047 00000100:00100000:2.0:1780298644.868406:0:6476:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244812d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+179:29360:x1866777609637632:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:2.0:1780298644.868446:0:6476:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298644.868462:0:6476:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279c00/0xaf35132e630355e0 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 99 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4528 expref: 179 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.868477:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298644.868481:0:6476:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298644.868608:0:6476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3000/0xaf35132e630356b2 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 98 type: IBT flags: 0x40000000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.868620:0:6476:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2000/0xaf35132e630355d2 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 98 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4521 expref: 144 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.868645:0:6476:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3000/0xaf35132e630356b2 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 98 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.868665:0:6476:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2000/0xaf35132e630355d2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 98 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4521 expref: 144 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.868677:0:6476:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2000/0xaf35132e630355d2 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 98 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4521 expref: 144 pid: 6477 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.868686:0:6476:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372d500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6476:1866777605054080:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298644.868738:0:6476:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1480 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.869491:0:6476:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372d500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6476:1866777605054080:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298644.869504:0:6476:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3000/0xaf35132e630356b2 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 98 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.869779:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.869807:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609637760 00000100:00100000:2.0:1780298644.869811:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609637760 00010000:00010000:2.0:1780298644.869828:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2000/0xaf35132e630355d2 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 98 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4521 expref: 145 pid: 6477 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.869856:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15235 00000100:00100000:2.0:1780298644.869861:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244810e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+145:2399:x1866777609637760:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298644.869869:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298644.869872:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2000/0xaf35132e630355d2 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 99 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4521 expref: 145 pid: 6477 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.869881:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2000/0xaf35132e630355d2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 99 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4521 expref: 145 pid: 6477 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.869892:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2000/0xaf35132e630355d2 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 99 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4521 expref: 145 pid: 6477 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.869903:0:6462:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298644.869905:0:6462:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3000/0xaf35132e630356b2 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 98 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.869916:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3000/0xaf35132e630356b2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 98 type: IBT flags: 0x40290000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.869921:0:6462:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3000/0xaf35132e630356b2 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 98 type: IBT flags: 0x40290000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.869936:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298644.869949:0:6476:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3000/0xaf35132e630356b2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 97 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.869956:0:6476:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3000/0xaf35132e630356b2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 97 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.869962:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244810e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+144:2399:x1866777609637760:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 100us (183us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.869967:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15235 00000001:00080000:1.0:1780298644.870020:0:6476:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991098 00000004:00010000:1.0:1780298644.870049:0:6476:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 000000008ac38357 reply state 0000000007f83b15 transno 4294991099 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3000/0xaf35132e630356b2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 97 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.870061:0:6476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3000/0xaf35132e630356b2 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 97 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.870066:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.870069:0:6476:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1780298644.870071:0:6476:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1780298644.870160:0:6476:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244812d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+181:29360:x1866777609637632:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1752us (1833us total) trans 4294991099 rc 0/0 00000100:00100000:1.0:1780298644.870166:0:6476:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62047 00000020:00080000:1.0:1780298644.870170:0:6476:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1780298644.870857:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.870887:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609637888 00000100:00100000:2.0:1780298644.870891:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609637888 00000100:00100000:2.0:1780298644.870918:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10097 00000100:00100000:2.0:1780298644.870926:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244811f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:29360:x1866777609637888:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.870936:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.870948:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e630356b9 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d45fa expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.870967:0:19992:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e6303566c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1097728->2973695] (req 1097728->1105919) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45b4 expref: 12 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.870973:0:19992:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63035657 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2973696->36528127] (req 2973696->2977791) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d459f expref: 12 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.870978:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 2 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(31 169) 00010000:00010000:2.0:1780298644.870981:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(33 169) 00010000:00010000:2.0:1780298644.870984:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e630356b9 lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45fa expref: 12 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.871009:0:19992:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63035657 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2973696->36528127] (req 2973696->2977791) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d459f expref: 12 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.871020:0:19992:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63035657 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2973696->36528127] (req 2973696->2977791) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d459f expref: 12 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.871032:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605054208:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298644.871082:0:19992:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e6303566c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1097728->2973695] (req 1097728->1105919) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45b4 expref: 12 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.871091:0:19992:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e6303566c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1097728->2973695] (req 1097728->1105919) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45b4 expref: 12 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.871097:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605054336:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298644.871113:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1480 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.872497:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605054208:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298644.872507:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605054336:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298644.872523:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e630356b9 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45fa expref: 12 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.872534:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3e00, rc 0) 00000100:00100000:2.0:1780298644.872560:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244811f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:29360:x1866777609637888:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1636us (1704us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.872565:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10097 00000100:00100000:3.0:1780298644.872846:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.872892:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.872893:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609638016 00000100:00100000:0.0:1780298644.872900:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609638016 00010000:00010000:0.0:1780298644.872924:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e6303566c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1097728->2973695] (req 1097728->1105919) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45b4 expref: 13 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.872928:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609638144 00000100:00100000:2.0:1780298644.872933:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609638144 00010000:00010000:2.0:1780298644.872951:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63035657 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2973696->36528127] (req 2973696->2977791) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d459f expref: 12 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.872960:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15236 00000100:00100000:0.0:1780298644.872971:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609638016:12345-192.168.203.25@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1780298644.872979:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15237 00010000:00010000:0.0:1780298644.872987:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1780298644.872992:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348af7850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2399:x1866777609638144:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298644.873002:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298644.873015:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e6303566c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [1097728->2973695] (req 1097728->1105919) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45b4 expref: 13 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.873017:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63035657 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [2973696->36528127] (req 2973696->2977791) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d459f expref: 12 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.873041:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e6303566c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [1097728->2973695] (req 1097728->1105919) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45b4 expref: 13 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.873055:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63035657 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [2973696->36528127] (req 2973696->2977791) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d459f expref: 12 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.873081:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e6303566c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [1097728->2973695] (req 1097728->1105919) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45b4 expref: 13 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.873089:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63035657 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [2973696->36528127] (req 2973696->2977791) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d459f expref: 12 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.873095:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(33 169) 00010000:00010000:0.0:1780298644.873102:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(33 169) 00010000:00010000:0.0:1780298644.873110:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e630356b9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45fa expref: 11 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.873118:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e630356b9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45fa expref: 11 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.873151:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1780298644.873167:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e630356b9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45fa expref: 11 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.873186:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348af7850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609638144:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 196us (292us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.873186:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fd500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777605054464:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298644.873198:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15237 00000100:00100000:0.0:1780298644.873232:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.873984:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fd500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777605054464:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298644.873997:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298644.874026:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609638016:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1058us (1180us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.874037:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15236 00000100:00100000:3.0:1780298644.874168:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.874215:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609638272 00000100:00100000:2.0:1780298644.874221:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609638272 00000100:00100000:2.0:1780298644.874259:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10098 00000100:00100000:2.0:1780298644.874285:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:29360:x1866777609638272:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.874299:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.874314:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e630356c0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4601 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.874336:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e6303568f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3088384->36642815] (req 3088384->3092479) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45d7 expref: 16 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.874345:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(32 122) 00010000:00010000:2.0:1780298644.874349:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(33 122) 00010000:00010000:2.0:1780298644.874353:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e630356c0 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4601 expref: 16 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.874374:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e6303568f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3088384->36642815] (req 3088384->3092479) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45d7 expref: 16 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.874388:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e6303568f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3088384->36642815] (req 3088384->3092479) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45d7 expref: 16 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.874398:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605054592:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298644.874441:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb10c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.875177:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605054592:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298644.875196:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e630356c0 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4601 expref: 16 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.875208:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2400, rc 0) 00000100:00100000:2.0:1780298644.875236:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:29360:x1866777609638272:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 969us (1071us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.875243:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10098 00000100:00100000:0.0:1780298644.875492:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.875554:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609638400 00000100:00100000:1.0:1780298644.875563:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609638400 00010000:00010000:1.0:1780298644.875602:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e6303568f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3088384->36642815] (req 3088384->3092479) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45d7 expref: 16 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.875641:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15238 00000100:00100000:1.0:1780298644.875650:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609638400:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298644.875662:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298644.875680:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e6303568f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3088384->36642815] (req 3088384->3092479) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45d7 expref: 16 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.875690:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e6303568f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3088384->36642815] (req 3088384->3092479) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45d7 expref: 16 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.875708:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e6303568f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3088384->36642815] (req 3088384->3092479) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45d7 expref: 16 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.875725:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(33 122) 00010000:00010000:1.0:1780298644.875730:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(33 122) 00010000:00010000:1.0:1780298644.875736:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e630356c0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4601 expref: 15 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.875745:0:6463:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e630356c0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4601 expref: 15 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.875778:0:6463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e630356c0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4601 expref: 15 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.875792:0:6463:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433de300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777605054720:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298644.875873:0:6463:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.876721:0:6463:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433de300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777605054720:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298644.876739:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298644.876783:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2400:x1866777609638400:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1134us (1296us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.876793:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15238 00000100:00100000:0.0:1780298644.876810:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.876875:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609638528 00000100:00100000:0.0:1780298644.876899:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.876907:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609638528 00002000:00010000:3.0:1780298644.876932:0:19950:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (5375863->18446744073709551615) 00002000:00100000:3.0:1780298644.876937:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481ff800 with x1866777609638528 ext(5375863->18446744073709551615) 00002000:00010000:3.0:1780298644.876945:0:19950:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481ff800. 00000100:00100000:2.0:1780298644.876946:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609638656 00000100:00100000:2.0:1780298644.876951:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609638656 00002000:00010000:2.0:1780298644.876966:0:8364:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4194304->18446744073709551615) 00002000:00100000:2.0:1780298644.876969:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fe680 with x1866777609638656 ext(4194304->18446744073709551615) 00000100:00100000:3.0:1780298644.876973:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6372 00002000:00010000:2.0:1780298644.876976:0:8364:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf3481fe680. 00000100:00100000:3.0:1780298644.876982:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481ff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609638528:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:1.0:1780298644.877009:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6373 00000100:00100000:1.0:1780298644.877020:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2400:x1866777609638656:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:1.0:1780298644.877174:0:19949:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4103694 -> 4194304 00002000:00010000:1.0:1780298644.877177:0:19949:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298644.877179:0:19949:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298644.877186:0:19949:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298644.877188:0:19949:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 96 -> 96 00000100:00100000:1.0:1780298644.877212:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609638656:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 196us (314us total) trans 4294969293 rc 0/0 00000100:00100000:1.0:1780298644.877218:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6373 00002000:00010000:1.0:1780298644.877221:0:19949:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4194304->18446744073709551615) 00002000:00100000:1.0:1780298644.877223:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fe680 with x1866777609638656 ext(4194304->18446744073709551615) 00002000:00010000:1.0:1780298644.877226:0:19949:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf3481fe680. 00002000:00010000:2.0:1780298644.877236:0:19950:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 5375863 00002000:00010000:2.0:1780298644.877239:0:19950:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298644.877242:0:19950:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298644.877244:0:19950:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298644.877246:0:19950:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 440 -> 440 00000100:00100000:2.0:1780298644.877287:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481ff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609638528:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 296us (468us total) trans 4294969746 rc 0/0 00000100:00100000:2.0:1780298644.877293:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6372 00002000:00010000:2.0:1780298644.877296:0:19950:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (5375863->18446744073709551615) 00002000:00100000:2.0:1780298644.877298:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481ff800 with x1866777609638528 ext(5375863->18446744073709551615) 00002000:00010000:2.0:1780298644.877303:0:19950:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481ff800. 00000100:00100000:3.0:1780298644.878207:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.878243:0:17578:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609638784 00000100:00100000:1.0:1780298644.878247:0:17578:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609638784 00000100:00100000:1.0:1780298644.878291:0:17578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62048 00000100:00100000:1.0:1780298644.878301:0:17578:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548bb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+181:29360:x1866777609638784:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.878320:0:17578:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.878334:0:17578:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279200/0xaf35132e630356c7 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 98 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4608 expref: -99 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.878407:0:17578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278400/0xaf35132e630356ce lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 99 type: IBT flags: 0x40000000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.878418:0:17578:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278400/0xaf35132e630356ce lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 99 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.878425:0:17578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278400/0xaf35132e630356ce lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 99 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1780298644.878444:0:17578:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 9570167 00000004:00200000:1.0:1780298644.878448:0:17578:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1780298644.878462:0:17578:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278400/0xaf35132e630356ce lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 99 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.878479:0:17578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279200/0xaf35132e630356c7 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 99 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4608 expref: 183 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.878489:0:17578:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278400/0xaf35132e630356ce lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 98 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4608 expref: 182 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.878494:0:17578:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.878496:0:17578:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1780298644.878498:0:17578:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278400, rc 0) 00000100:00100000:1.0:1780298644.878521:0:17578:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548bb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+182:29360:x1866777609638784:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 224us (315us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.878528:0:17578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62048 00000100:00100000:1.0:1780298644.880163:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.880198:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609638912 00000100:00100000:0.0:1780298644.880202:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609638912 00000100:00100000:0.0:1780298644.880241:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6374 00000100:00100000:0.0:1780298644.880251:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2399:x1866777609638912:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298644.880416:0:8362:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 5375863 -> 5375863 00002000:00010000:0.0:1780298644.880419:0:8362:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298644.880422:0:8362:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298644.880424:0:8362:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298644.880426:0:8362:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 440 -> 456 00000100:00100000:0.0:1780298644.880460:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609638912:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 211us (298us total) trans 4294969747 rc 0/0 00000100:00100000:0.0:1780298644.880468:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6374 00000100:00100000:0.0:1780298644.882033:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.882079:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609639040 00000100:00100000:2.0:1780298644.882085:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609639040 00000100:00100000:2.0:1780298644.882119:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6375 00000100:00100000:2.0:1780298644.882148:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481ffb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609639040:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298644.882252:0:16972:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 5375863 -> 5375863 00002000:00010000:2.0:1780298644.882255:0:16972:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298644.882256:0:16972:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298644.882258:0:16972:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298644.882259:0:16972:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 456 -> 464 00000100:00100000:2.0:1780298644.882296:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481ffb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609639040:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 171us (264us total) trans 4294969748 rc 0/0 00000100:00100000:2.0:1780298644.882301:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6375 00000100:00100000:3.0:1780298644.883115:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.883162:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609639168 00000100:00100000:2.0:1780298644.883167:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609639168 00000100:00100000:2.0:1780298644.883199:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62049 00000100:00100000:2.0:1780298644.883207:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548bad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+144:29360:x1866777609639168:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.883218:0:6477:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.883231:0:6477:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3c00/0xaf35132e630356d5 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 99 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d460f expref: -99 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.883307:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2600/0xaf35132e630356dc lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 100 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.883320:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2600/0xaf35132e630356dc lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 100 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.883331:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2600/0xaf35132e630356dc lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 100 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298644.883353:0:6477:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 9570167 00000004:00200000:2.0:1780298644.883358:0:6477:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298644.883371:0:6477:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2600/0xaf35132e630356dc lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 100 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.883394:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3c00/0xaf35132e630356d5 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 100 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d460f expref: 146 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.883410:0:6477:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2600/0xaf35132e630356dc lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 99 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d460f expref: 145 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.883418:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298644.883421:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298644.883424:0:6477:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2600, rc 0) 00000100:00100000:2.0:1780298644.883452:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548bad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+145:29360:x1866777609639168:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 246us (337us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.883458:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62049 00000100:00100000:1.0:1780298644.884418:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.884450:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609639296 00000100:00100000:2.0:1780298644.884454:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609639296 00000100:00100000:2.0:1780298644.884478:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10099 00000100:00100000:2.0:1780298644.884483:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609639296:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.884489:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.884496:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e630356e3 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4616 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.884536:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605054848:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.884592:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb10c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.884604:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.884641:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609639424 00000100:00100000:2.0:1780298644.884645:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609639424 00000100:00100000:2.0:1780298644.884679:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10100 00000100:00100000:2.0:1780298644.884687:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609639424:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.884696:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.884706:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e630356ea lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d461d expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.884730:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605054976:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.884757:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb16c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.885568:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605054848:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.885607:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e630356e3 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4616 expref: 13 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.885622:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000cdbb289/0xaf35132e630356e3 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4616 expref: 13 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.885630:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3600, rc 0) 00000100:00100000:2.0:1780298644.885663:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609639296:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1179us (1245us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.885670:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10099 00000100:00100000:1.0:1780298644.885681:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605054976:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.885699:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e630356ea lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d461d expref: 13 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.885712:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001eb6350c/0xaf35132e630356ea lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d461d expref: 13 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.885721:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2000, rc 0) 00000100:00100000:1.0:1780298644.885738:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609639424:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1054us (1138us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.885743:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10100 00000100:00100000:1.0:1780298644.886848:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.886882:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609639552 00000100:00100000:2.0:1780298644.886886:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609639552 00000100:00100000:2.0:1780298644.886921:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10101 00000100:00100000:2.0:1780298644.886930:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609639552:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.886941:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.886950:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e630356f1 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4624 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.886979:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605055104:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.887025:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb10c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.887200:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.887243:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609639680 00000100:00100000:2.0:1780298644.887248:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609639680 00000100:00100000:2.0:1780298644.887283:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10102 00000100:00100000:2.0:1780298644.887292:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609639680:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.887301:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.887311:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000fa1945a/0xaf35132e630356f8 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d462b expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.887339:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605055232:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.887379:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb16c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.887908:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605055104:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.887926:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e630356f1 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4624 expref: 13 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.887944:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000909e5905/0xaf35132e630356f1 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4624 expref: 13 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.887956:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2e00, rc 0) 00000100:00100000:2.0:1780298644.887992:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609639552:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1062us (1142us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.888000:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10101 00000100:00100000:0.0:1780298644.888051:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605055232:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.888073:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000fa1945a/0xaf35132e630356f8 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d462b expref: 13 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.888097:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000fa1945a/0xaf35132e630356f8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d462b expref: 13 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.888107:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3000, rc 0) 00000100:00100000:0.0:1780298644.888165:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609639680:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 875us (965us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.888173:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10102 00000100:00100000:1.0:1780298644.889117:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.889166:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609639808 00000100:00100000:3.0:1780298644.889170:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609639808 00000100:00100000:2.0:1780298644.889233:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10103 00000100:00100000:2.0:1780298644.889244:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:29360:x1866777609639808:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.889253:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.889262:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000075e99a4b/0xaf35132e630356ff lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4632 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.889291:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e630356c0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4601 expref: 15 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.889301:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(33 122) 00010000:00010000:2.0:1780298644.889304:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(34 122) 00010000:00010000:2.0:1780298644.889309:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000075e99a4b/0xaf35132e630356ff lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2764800->2777087] (req 2764800->2777087) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4632 expref: 13 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.889331:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e630356c0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4601 expref: 15 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.889344:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e630356c0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4601 expref: 15 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.889355:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605055360:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298644.889396:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb10c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.890000:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605055360:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298644.890013:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000075e99a4b/0xaf35132e630356ff lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2764800->2777087] (req 2764800->2777087) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4632 expref: 13 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.890025:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3800, rc 0) 00000100:00100000:2.0:1780298644.890049:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:29360:x1866777609639808:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 807us (930us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.890055:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10103 00000100:00100000:2.0:1780298644.890288:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.890319:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609639936 00000100:00100000:0.0:1780298644.890322:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609639936 00010000:00010000:0.0:1780298644.890340:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e630356c0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4601 expref: 16 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.890362:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15239 00000100:00100000:0.0:1780298644.890368:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2399:x1866777609639936:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298644.890376:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298644.890384:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e630356c0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4601 expref: 16 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.890392:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e630356c0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4601 expref: 16 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.890404:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005e996b61/0xaf35132e630356c0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4601 expref: 16 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.890418:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(34 122) 00010000:00010000:0.0:1780298644.890422:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(34 122) 00010000:00010000:0.0:1780298644.890430:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000075e99a4b/0xaf35132e630356ff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2764800->36319231] (req 2764800->2777087) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4632 expref: 12 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.890439:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000075e99a4b/0xaf35132e630356ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2764800->36319231] (req 2764800->2777087) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4632 expref: 12 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.890465:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000075e99a4b/0xaf35132e630356ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2764800->36319231] (req 2764800->2777087) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4632 expref: 12 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.890473:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fed80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:20253:1866777605055488:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298644.890506:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.891193:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fed80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:20253:1866777605055488:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298644.891207:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298644.891233:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372c380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2399:x1866777609639936:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 865us (958us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.891242:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15239 00000100:00100000:3.0:1780298644.891498:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.891534:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609640064 00000100:00100000:1.0:1780298644.891538:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609640064 00002000:00100000:1.0:1780298644.891568:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x12:0x0] (2764800->2768895) req@ffff9cf2548b8e00 x1866777609640064/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.891579:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2548b8e00 with x1866777609640064 ext(2764800->2768895) 00002000:00010000:1.0:1780298644.891582:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2548b8e00 00000100:00100000:1.0:1780298644.891608:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6376 00000100:00100000:1.0:1780298644.891616:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2399:x1866777609640064:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298644.891795:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2399:x1866777609640064:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 179us (297us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298644.891802:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6376 00002000:00100000:1.0:1780298644.891808:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x12:0x0] (2764800->2768895) req@ffff9cf2548b8e00 x1866777609640064/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/4536 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.891816:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2548b8e00 with x1866777609640064 ext(2764800->2768895) 00002000:00010000:1.0:1780298644.891820:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2548b8e00 00000100:00100000:2.0:1780298644.892764:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.892807:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609640192 00000100:00100000:0.0:1780298644.892813:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609640192 00002000:00100000:0.0:1780298644.892837:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (2772992->2777087) req@ffff9cf34372e300 x1866777609640192/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298644.892848:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf34372e300 with x1866777609640192 ext(2772992->2777087) 00002000:00010000:0.0:1780298644.892851:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf34372e300 00000100:00100000:0.0:1780298644.892875:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6377 00000100:00100000:0.0:1780298644.892883:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2399:x1866777609640192:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298644.892977:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2399:x1866777609640192:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 95us (214us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298644.892984:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6377 00002000:00100000:0.0:1780298644.892991:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (2772992->2777087) req@ffff9cf34372e300 x1866777609640192/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/4536 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298644.892999:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf34372e300 with x1866777609640192 ext(2772992->2777087) 00002000:00010000:0.0:1780298644.893003:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf34372e300 00000100:00100000:0.0:1780298644.894188:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.894215:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.894232:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609640320 00000100:00100000:1.0:1780298644.894236:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609640320 00000100:00100000:2.0:1780298644.894252:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609640448 00000100:00100000:2.0:1780298644.894259:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609640448 00000100:00100000:1.0:1780298644.894263:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10104 00000100:00100000:1.0:1780298644.894272:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609640320:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.894289:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.894302:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63035706 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4639 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.894309:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10105 00000100:00100000:2.0:1780298644.894322:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2399:x1866777609640448:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.894331:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.894342:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002c458c77/0xaf35132e6303570d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4640 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.894350:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433df480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605055616:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.894384:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605055744:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.894408:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a840 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.894426:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb10c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.895178:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433df480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605055616:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.895203:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63035706 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4639 expref: 13 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.895220:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63035706 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4639 expref: 13 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.895238:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279c00, rc 0) 00000100:00100000:2.0:1780298644.895255:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605055744:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.895271:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609640320:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1000us (1080us total) trans 0 rc 301/301 00010000:00010000:2.0:1780298644.895288:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002c458c77/0xaf35132e6303570d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4640 expref: 14 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.895293:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10104 00010000:00010000:2.0:1780298644.895312:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002c458c77/0xaf35132e6303570d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4640 expref: 14 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.895327:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2a00, rc 0) 00000100:00100000:2.0:1780298644.895353:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2399:x1866777609640448:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1034us (1136us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.895360:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10105 00000100:00100000:3.0:1780298644.896118:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.896227:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609640576 00000100:00100000:0.0:1780298644.896231:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609640576 00000100:00100000:3.0:1780298644.896239:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.896295:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609640704 00000100:00100000:2.0:1780298644.896300:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609640704 00000100:00100000:1.0:1780298644.896306:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10106 00000100:00100000:1.0:1780298644.896318:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609640576:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.896327:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298644.896331:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10107 00010000:00010000:1.0:1780298644.896339:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63035714 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4647 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.896344:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2399:x1866777609640704:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.896357:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.896368:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003540747e/0xaf35132e6303571b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d464e expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.896378:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433df480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605055872:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.896403:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605056000:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.896417:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a840 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.896435:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb10c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.897055:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605056000:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.897076:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003540747e/0xaf35132e6303571b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d464e expref: 14 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.897091:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003540747e/0xaf35132e6303571b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d464e expref: 14 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.897102:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3c00, rc 0) 00000100:00100000:2.0:1780298644.897150:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2399:x1866777609640704:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 809us (909us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.897151:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433df480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605055872:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.897164:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10107 00010000:00010000:1.0:1780298644.897181:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63035714 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4647 expref: 13 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.897198:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63035714 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4647 expref: 13 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.897209:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279a00, rc 0) 00000100:00100000:1.0:1780298644.897230:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609640576:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 918us (1115us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.897235:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10106 00000100:00100000:3.0:1780298644.898100:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.898157:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609640832 00000100:00100000:0.0:1780298644.898162:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609640832 00000100:00100000:0.0:1780298644.898192:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10108 00000100:00100000:0.0:1780298644.898202:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:29360:x1866777609640832:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.898211:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.898220:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e63035722 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4655 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.898235:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(34 122) 00010000:00010000:0.0:1780298644.898239:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(34 122) 00010000:00010000:0.0:1780298644.898245:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e63035722 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [69632->2764799] (req 69632->73727) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4655 expref: 14 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.898258:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e63035722 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [69632->2764799] (req 69632->73727) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4655 expref: 14 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.898265:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf600, rc 0) 00000100:00100000:0.0:1780298644.898299:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:29360:x1866777609640832:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 101us (200us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.898303:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10108 00000100:00100000:3.0:1780298644.898963:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.899020:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609640960 00000100:00100000:2.0:1780298644.899025:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609640960 00000100:00100000:2.0:1780298644.899066:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6378 00000100:00100000:2.0:1780298644.899075:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609640960:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298644.899388:0:19950:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:2.0:1780298644.899391:0:19950:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298644.899392:0:19950:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298644.899394:0:19950:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298644.899395:0:19950:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 96 -> 104 00000100:00100000:2.0:1780298644.899418:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2400:x1866777609640960:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 346us (456us total) trans 4294969294 rc 0/0 00000100:00100000:2.0:1780298644.899423:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6378 00000100:00100000:0.0:1780298644.900269:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.900303:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.900320:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609641088 00000100:00100000:2.0:1780298644.900325:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609641088 00000100:00100000:3.0:1780298644.900341:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609641216 00000100:00100000:3.0:1780298644.900347:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609641216 00000100:00100000:2.0:1780298644.900358:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10109 00000100:00100000:2.0:1780298644.900366:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609641088:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.900374:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.900381:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000052b89172/0xaf35132e63035729 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d465c expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.900385:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10110 00000100:00100000:3.0:1780298644.900399:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2399:x1866777609641216:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.900408:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298644.900413:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605056128:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.900418:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bed20494/0xaf35132e63035730 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4663 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.900452:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605056256:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.900456:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb10c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.900506:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.901170:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605056128:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.901191:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000052b89172/0xaf35132e63035729 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d465c expref: 13 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.901207:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000052b89172/0xaf35132e63035729 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d465c expref: 13 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.901218:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2800, rc 0) 00000100:00100000:2.0:1780298644.901246:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609641088:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 882us (977us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.901256:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10109 00000100:00100000:2.0:1780298644.901379:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605056256:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.901395:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bed20494/0xaf35132e63035730 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4663 expref: 16 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.901408:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bed20494/0xaf35132e63035730 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4663 expref: 16 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.901417:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157000, rc 0) 00000100:00100000:2.0:1780298644.901443:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2399:x1866777609641216:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1048us (1139us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.901452:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10110 00000100:00100000:3.0:1780298644.902356:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.902393:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609641344 00000100:00100000:1.0:1780298644.902398:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609641344 00000100:00100000:1.0:1780298644.902438:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10111 00000100:00100000:1.0:1780298644.902448:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609641344:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.902458:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.902469:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63035737 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d466a expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.902481:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.902505:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433df480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605056384:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.902510:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609641472 00000100:00100000:2.0:1780298644.902516:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609641472 00000100:00100000:2.0:1780298644.902551:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10112 00000100:00100000:2.0:1780298644.902563:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2398:x1866777609641472:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298644.902566:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a840 going to sleep for 11 seconds 00010000:00010000:2.0:1780298644.902573:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.902582:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d828f4f8/0xaf35132e6303573e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4671 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.902612:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605056512:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.902645:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb10c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.903210:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433df480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605056384:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.903227:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63035737 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d466a expref: 13 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.903243:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63035737 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d466a expref: 13 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.903254:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279200, rc 0) 00000100:00100000:2.0:1780298644.903298:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609641344:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 853us (943us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.903305:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10111 00000100:00100000:1.0:1780298644.903360:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605056512:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.903377:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d828f4f8/0xaf35132e6303573e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4671 expref: 16 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.903391:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d828f4f8/0xaf35132e6303573e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4671 expref: 16 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.903401:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2e00, rc 0) 00000100:00100000:1.0:1780298644.903433:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2398:x1866777609641472:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 874us (953us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.903440:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10112 00000100:00100000:0.0:1780298644.904492:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.904532:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609641600 00000100:00100000:1.0:1780298644.904537:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609641600 00000100:00100000:1.0:1780298644.904574:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6379 00000100:00100000:1.0:1780298644.904582:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2400:x1866777609641600:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298644.904714:0:8362:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:1.0:1780298644.904717:0:8362:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298644.904719:0:8362:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298644.904721:0:8362:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298644.904722:0:8362:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 104 -> 104 00000100:00100000:1.0:1780298644.904745:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609641600:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 165us (255us total) trans 4294969295 rc 0/0 00000100:00100000:1.0:1780298644.904749:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6379 00000100:00100000:0.0:1780298644.905841:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.905870:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.905889:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609641856 00000100:00100000:2.0:1780298644.905895:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609641856 00000100:00100000:1.0:1780298644.905915:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609641728 00000100:00100000:1.0:1780298644.905921:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609641728 00000100:00100000:2.0:1780298644.905939:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10113 00000100:00100000:2.0:1780298644.905950:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2399:x1866777609641856:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.905961:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298644.905969:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10114 00010000:00010000:2.0:1780298644.905971:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e63035745 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d467f expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.905985:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609641728:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.906001:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298644.906008:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605056640:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.906015:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004ba675d2/0xaf35132e6303574c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4678 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.906049:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605056768:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.906062:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1a80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.906078:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.906910:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605056640:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.906925:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605056768:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.906934:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e63035745 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d467f expref: 17 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.906945:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004ba675d2/0xaf35132e6303574c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4678 expref: 13 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.906955:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e63035745 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d467f expref: 17 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.906972:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3200, rc 0) 00010000:00010000:0.0:1780298644.906978:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004ba675d2/0xaf35132e6303574c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4678 expref: 13 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.906991:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156e00, rc 0) 00000100:00100000:2.0:1780298644.907003:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2399:x1866777609641856:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1055us (1161us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.907011:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10113 00000100:00100000:0.0:1780298644.907015:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609641728:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1038us (1145us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.907022:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10114 00000100:00100000:0.0:1780298644.908163:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.908195:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.908214:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609641984 00000100:00100000:2.0:1780298644.908221:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609641984 00000100:00100000:1.0:1780298644.908233:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609642112 00000100:00100000:1.0:1780298644.908239:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609642112 00000100:00100000:2.0:1780298644.908252:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10115 00000100:00100000:2.0:1780298644.908261:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609641984:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.908270:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298644.908285:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10116 00010000:00010000:2.0:1780298644.908288:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003540747e/0xaf35132e63035753 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4686 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.908295:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2397:x1866777609642112:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.908305:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.908313:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e6303575a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d468d expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.908324:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605056896:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.908341:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433dc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605057024:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.908381:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.908381:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837af00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.909209:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605056896:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.909235:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003540747e/0xaf35132e63035753 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4686 expref: 13 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.909251:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433dc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605057024:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.909253:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003540747e/0xaf35132e63035753 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4686 expref: 13 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.909284:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3c00, rc 0) 00010000:00010000:1.0:1780298644.909288:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e6303575a lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d468d expref: 17 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.909305:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e6303575a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d468d expref: 17 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.909317:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279a00, rc 0) 00000100:00100000:2.0:1780298644.909319:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609641984:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1058us (1162us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.909335:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10115 00000100:00100000:1.0:1780298644.909346:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2397:x1866777609642112:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1052us (1149us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.909353:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10116 00000100:00100000:3.0:1780298644.910697:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.910746:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609642240 00000100:00100000:2.0:1780298644.910753:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609642240 00002000:00100000:2.0:1780298644.910786:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (2895872->2904063) req@ffff9cf24519c700 x1866777609642240/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.910797:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24519c700 with x1866777609642240 ext(2895872->2904063) 00002000:00010000:2.0:1780298644.910801:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24519c700 00000100:00100000:2.0:1780298644.910826:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6380 00000100:00100000:2.0:1780298644.910836:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609642240:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298644.911039:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609642240:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 205us (342us total) trans 0 rc 8192/8192 00000100:00100000:2.0:1780298644.911046:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6380 00002000:00100000:2.0:1780298644.911053:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (2895872->2904063) req@ffff9cf24519c700 x1866777609642240/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:350/0 lens 488/8632 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 8192/8192 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.911070:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24519c700 with x1866777609642240 ext(2895872->2904063) 00002000:00010000:2.0:1780298644.911073:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24519c700 00000100:00100000:0.0:1780298644.912486:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.912537:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609642368 00000100:00100000:2.0:1780298644.912544:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609642368 00000100:00100000:2.0:1780298644.912581:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10117 00000100:00100000:2.0:1780298644.912591:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2400:x1866777609642368:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.912603:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.912616:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000052b89172/0xaf35132e63035761 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4694 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.912663:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605057152:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.912720:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.913523:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605057152:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.913544:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000052b89172/0xaf35132e63035761 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4694 expref: 16 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.913558:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000052b89172/0xaf35132e63035761 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4694 expref: 16 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.913566:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2800, rc 0) 00000100:00100000:2.0:1780298644.913593:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2400:x1866777609642368:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1006us (1111us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.913598:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10117 00000100:00100000:3.0:1780298644.914695:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.914735:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609642496 00000100:00100000:0.0:1780298644.914740:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609642496 00000100:00100000:0.0:1780298644.914777:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10118 00000100:00100000:0.0:1780298644.914788:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2397:x1866777609642496:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.914799:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.914810:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a3f03a/0xaf35132e63035768 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d469b expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.914848:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605057280:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.914892:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.915726:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605057280:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298644.915750:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000001a3f03a/0xaf35132e63035768 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d469b expref: 16 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.915767:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a3f03a/0xaf35132e63035768 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d469b expref: 16 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.915779:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae600, rc 0) 00000100:00100000:3.0:1780298644.915812:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2397:x1866777609642496:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1027us (1120us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.915820:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10118 00000100:00100000:3.0:1780298644.917067:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.917171:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609642624 00000100:00100000:1.0:1780298644.917177:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609642624 00000100:00100000:1.0:1780298644.917228:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6381 00000100:00100000:1.0:1780298644.917237:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2397:x1866777609642624:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298644.917533:0:8363:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 5375863 -> 5375863 00002000:00010000:1.0:1780298644.917538:0:8363:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298644.917541:0:8363:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298644.917543:0:8363:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298644.917545:0:8363:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 464 -> 488 00000100:00100000:1.0:1780298644.917587:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2397:x1866777609642624:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 350us (521us total) trans 4294969749 rc 0/0 00000100:00100000:1.0:1780298644.917595:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6381 00000100:00100000:2.0:1780298644.918652:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.918705:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609642752 00000100:00100000:0.0:1780298644.918710:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609642752 00000100:00100000:0.0:1780298644.918739:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10119 00000100:00100000:0.0:1780298644.918746:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2397:x1866777609642752:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.918756:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.918766:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b35ce980/0xaf35132e6303576f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d46a2 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.918803:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605057408:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.918846:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.919724:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605057408:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.919768:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b35ce980/0xaf35132e6303576f lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d46a2 expref: 16 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.919781:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b35ce980/0xaf35132e6303576f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d46a2 expref: 16 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.919794:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bafa00, rc 0) 00000100:00100000:1.0:1780298644.919818:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2397:x1866777609642752:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1073us (1169us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.919823:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10119 00000100:00100000:3.0:1780298644.920901:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.920934:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609642880 00000100:00100000:1.0:1780298644.920938:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609642880 00000100:00100000:1.0:1780298644.920967:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10120 00000100:00100000:1.0:1780298644.920973:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2397:x1866777609642880:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.920981:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.920989:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e63035776 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d46a9 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.921025:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433dc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605057536:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.921079:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837af00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.921859:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433dc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605057536:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.921879:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e63035776 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d46a9 expref: 16 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.921896:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e63035776 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d46a9 expref: 16 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.921906:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279a00, rc 0) 00000100:00100000:1.0:1780298644.921937:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2397:x1866777609642880:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 963us (1038us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.921949:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10120 00000100:00100000:0.0:1780298644.923225:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.923248:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.923268:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609643008 00000100:00100000:2.0:1780298644.923283:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609643008 00000100:00100000:3.0:1780298644.923291:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609643136 00000100:00100000:3.0:1780298644.923298:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609643136 00000100:00100000:2.0:1780298644.923326:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10121 00000100:00100000:2.0:1780298644.923339:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481faa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609643008:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298644.923344:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10122 00010000:00010000:2.0:1780298644.923355:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298644.923358:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2398:x1866777609643136:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.923368:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.923369:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003540747e/0xaf35132e6303577d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d46b0 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.923378:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042c16ab0/0xaf35132e63035784 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d46b7 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.923413:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605057664:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.923417:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2556a9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605057792:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.923477:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.923478:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.924322:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605057664:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.924346:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003540747e/0xaf35132e6303577d lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d46b0 expref: 13 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.924352:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2556a9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605057792:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.924368:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003540747e/0xaf35132e6303577d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d46b0 expref: 13 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.924380:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3c00, rc 0) 00010000:00010000:1.0:1780298644.924382:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000042c16ab0/0xaf35132e63035784 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d46b7 expref: 17 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.924400:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042c16ab0/0xaf35132e63035784 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d46b7 expref: 17 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.924413:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481faa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609643008:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1077us (1188us total) trans 0 rc 301/301 00010000:00010000:1.0:1780298644.924413:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156800, rc 0) 00000100:00100000:2.0:1780298644.924424:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10121 00000100:00100000:1.0:1780298644.924444:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2398:x1866777609643136:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1090us (1193us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.924453:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10122 00000100:00100000:0.0:1780298644.925626:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.925645:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.925668:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609643264 00000100:00100000:1.0:1780298644.925674:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609643264 00000100:00100000:2.0:1780298644.925690:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609643392 00000100:00100000:2.0:1780298644.925696:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609643392 00000100:00100000:1.0:1780298644.925714:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10123 00000100:00100000:1.0:1780298644.925725:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609643264:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.925735:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298644.925740:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10124 00010000:00010000:1.0:1780298644.925746:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e6303578b lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d46be expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.925753:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2397:x1866777609643392:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.925771:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.925781:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000024bbcf41/0xaf35132e63035792 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d46c5 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.925789:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433dc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605057920:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.925819:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605058048:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.925843:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837af00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.925853:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.926726:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433dc000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605057920:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.926733:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605058048:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.926755:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000024bbcf41/0xaf35132e63035792 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d46c5 expref: 17 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.926757:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e6303578b lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d46be expref: 13 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.926774:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000024bbcf41/0xaf35132e63035792 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d46c5 expref: 17 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.926783:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2c00, rc 0) 00010000:00010000:1.0:1780298644.926783:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e6303578b lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d46be expref: 13 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.926797:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279c00, rc 0) 00000100:00100000:2.0:1780298644.926815:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2397:x1866777609643392:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1066us (1167us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.926826:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10124 00000100:00100000:1.0:1780298644.926826:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609643264:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1102us (1197us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.926837:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10123 00000100:00100000:3.0:1780298644.928411:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.928462:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609643520 00000100:00100000:0.0:1780298644.928469:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609643520 00000100:00100000:3.0:1780298644.928491:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.928521:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609643648 00000100:00100000:0.0:1780298644.928526:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609643648 00000100:00100000:2.0:1780298644.928531:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10125 00000100:00100000:2.0:1780298644.928542:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556ab480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609643520:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.928552:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298644.928554:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10126 00010000:00010000:2.0:1780298644.928563:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000aa41763f/0xaf35132e63035799 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d46cc expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.928567:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2398:x1866777609643648:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.928578:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.928587:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e630357a0 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d46d3 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.928599:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605058176:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.928615:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243845180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605058304:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.928653:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.928654:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.929585:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605058176:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.929611:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000aa41763f/0xaf35132e63035799 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d46cc expref: 13 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.929631:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000aa41763f/0xaf35132e63035799 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d46cc expref: 13 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.929639:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243845180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605058304:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.929649:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3600, rc 0) 00010000:00010000:3.0:1780298644.929692:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e630357a0 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d46d3 expref: 17 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.929692:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556ab480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609643520:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1148us (1284us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.929716:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10125 00010000:00010000:3.0:1780298644.929734:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c47ea6a1/0xaf35132e630357a0 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d46d3 expref: 17 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.929753:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf200, rc 0) 00000100:00100000:3.0:1780298644.929806:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2398:x1866777609643648:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1245us (1318us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.929884:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10126 00000100:00100000:0.0:1780298644.931016:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.931082:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609643776 00000100:00100000:3.0:1780298644.931090:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609643776 00000100:00100000:3.0:1780298644.931172:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10127 00000100:00100000:3.0:1780298644.931185:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243846d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:29360:x1866777609643776:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.931201:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.931216:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e630357a7 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d46da expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.931241:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e630356b9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45fa expref: 14 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.931250:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(33 169) 00010000:00010000:3.0:1780298644.931254:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(34 169) 00010000:00010000:3.0:1780298644.931259:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e630357a7 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [5169152->5177343] (req 5169152->5177343) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d46da expref: 13 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.931301:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e630356b9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45fa expref: 14 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.931316:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e630356b9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45fa expref: 14 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.931331:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2556abb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605058432:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298644.931408:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.932203:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2556abb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605058432:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298644.932230:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e630357a7 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [5169152->5177343] (req 5169152->5177343) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d46da expref: 13 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.932244:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157c00, rc 0) 00000100:00100000:3.0:1780298644.932354:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243846d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:29360:x1866777609643776:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1109us (1277us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298644.932363:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10127 00000100:00100000:1.0:1780298644.932424:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.932515:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609643904 00000100:00100000:3.0:1780298644.932521:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609643904 00010000:00010000:3.0:1780298644.932594:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e630356b9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45fa expref: 15 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.932623:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15240 00000100:00100000:3.0:1780298644.932632:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dc000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2399:x1866777609643904:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298644.932645:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298644.932660:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e630356b9 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45fa expref: 15 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.932670:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e630356b9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45fa expref: 15 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.932688:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000cc4b2e1a/0xaf35132e630356b9 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d45fa expref: 15 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.932705:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(34 169) 00010000:00010000:3.0:1780298644.932710:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(34 169) 00010000:00010000:3.0:1780298644.932716:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e630357a7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [5169152->38723583] (req 5169152->5177343) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d46da expref: 12 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.932723:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e630357a7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [5169152->38723583] (req 5169152->5177343) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d46da expref: 12 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.932745:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e630357a7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [5169152->38723583] (req 5169152->5177343) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d46da expref: 12 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.932752:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2556abb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777605058560:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298644.932797:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.933563:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2556abb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777605058560:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298644.933574:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298644.933598:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433dc000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609643904:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 969us (1179us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298644.933604:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15240 00000100:00100000:0.0:1780298644.933829:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.933877:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609644032 00000100:00100000:2.0:1780298644.933883:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609644032 00000100:00100000:2.0:1780298644.933934:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6382 00000100:00100000:2.0:1780298644.933942:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243847480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609644032:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298644.934189:0:8364:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 5375863 -> 5375863 00002000:00010000:2.0:1780298644.934194:0:8364:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298644.934200:0:8364:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298644.934203:0:8364:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298644.934205:0:8364:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 488 -> 504 00000100:00100000:2.0:1780298644.934233:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243847480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609644032:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 293us (406us total) trans 4294969750 rc 0/0 00000100:00100000:2.0:1780298644.934238:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6382 00000100:00100000:1.0:1780298644.935228:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.935292:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609644288 00000100:00100000:0.0:1780298644.935299:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609644288 00000100:00100000:1.0:1780298644.935304:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.935340:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609644160 00000100:00100000:0.0:1780298644.935344:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609644160 00000100:00100000:2.0:1780298644.935362:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10128 00000100:00100000:2.0:1780298644.935374:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2397:x1866777609644288:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.935386:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298644.935391:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10129 00010000:00010000:2.0:1780298644.935400:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000052b89172/0xaf35132e630357ae lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d46e8 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.935400:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2397:x1866777609644160:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.935413:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.935423:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e630357b5 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d46e1 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.935447:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605058688:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.935460:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243846a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605058816:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.935502:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.935503:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.936370:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605058688:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.936386:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243846a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605058816:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.936392:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000052b89172/0xaf35132e630357ae lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d46e8 expref: 17 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.936410:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000052b89172/0xaf35132e630357ae lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d46e8 expref: 17 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.936412:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e630357b5 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d46e1 expref: 15 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.936438:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2800, rc 0) 00010000:00010000:1.0:1780298644.936442:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e630357b5 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d46e1 expref: 15 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.936455:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae600, rc 0) 00000100:00100000:2.0:1780298644.936476:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2397:x1866777609644288:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1104us (1248us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.936484:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2397:x1866777609644160:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1084us (1179us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.936495:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10128 00000100:00100000:1.0:1780298644.936503:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10129 00000100:00100000:3.0:1780298644.937925:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.937945:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.937977:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609644544 00000100:00100000:0.0:1780298644.937979:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609644416 00000100:00100000:2.0:1780298644.937982:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609644544 00000100:00100000:0.0:1780298644.937986:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609644416 00000100:00100000:2.0:1780298644.938013:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10130 00000100:00100000:2.0:1780298644.938023:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556abb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2399:x1866777609644544:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298644.938030:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10131 00010000:00010000:2.0:1780298644.938032:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298644.938043:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2399:x1866777609644416:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.938045:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e630357bc lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d46f6 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.938053:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.938063:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630357c3 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d46ef expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.938101:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605058944:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.938110:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243845c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605059072:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.938189:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.938189:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf373393300 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.939227:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605058944:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.939254:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e630357bc lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d46f6 expref: 17 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.939267:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e630357bc lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d46f6 expref: 17 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.939284:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3200, rc 0) 00000100:00100000:2.0:1780298644.939319:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556abb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2399:x1866777609644544:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1297us (1392us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.939323:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243845c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605059072:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.939335:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10130 00010000:00010000:3.0:1780298644.939371:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630357c3 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d46ef expref: 15 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.939395:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630357c3 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d46ef expref: 15 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.939409:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf000, rc 0) 00000100:00100000:3.0:1780298644.939518:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2399:x1866777609644416:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1479us (1569us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298644.939527:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10131 00000100:00100000:0.0:1780298644.940519:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.940563:0:7411:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609644800 00000100:00100000:1.0:1780298644.940570:0:7411:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609644800 00000100:00100000:1.0:1780298644.940621:0:7411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62050 00000100:00100000:1.0:1780298644.940632:0:7411:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243844700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+182:29360:x1866777609644800:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:1.0:1780298644.940684:0:7411:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298644.940705:0:7411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278400/0xaf35132e630356ce lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 100 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4608 expref: 182 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.940721:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.940726:0:7411:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298644.940925:0:7411:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279400/0xaf35132e630357ca lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 99 type: IBT flags: 0x40000000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.940941:0:7411:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2600/0xaf35132e630356dc lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 99 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d460f expref: 144 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.940968:0:7411:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279400/0xaf35132e630357ca lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 99 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.940997:0:7411:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2600/0xaf35132e630356dc lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 99 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d460f expref: 144 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.941011:0:7411:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2600/0xaf35132e630356dc lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 99 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d460f expref: 144 pid: 6477 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.941025:0:7411:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433df480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7411:1866777605059200:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298644.941089:0:7411:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a840 going to sleep for 11 seconds 00000001:00080000:2.0:1780298644.941861:0:6455:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-MDT0000: transno 4294991100 is committed 00010000:00010000:1.0:1780298644.941891:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c000/0xaf35132e63030c21 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 99 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.941901:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c000/0xaf35132e63030c21 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 99 type: IBT flags: 0x50210400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1780298644.941903:0:6455:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-MDT0000: transno 4294991097 is committed 00010000:00010000:1.0:1780298644.941912:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c000/0xaf35132e63030c21 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 99 type: IBT flags: 0x40210400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.941920:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf34830c000 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.941925:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c000/0xaf35132e63030c21 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 99 type: IBT flags: 0x40210400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.941935:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c000/0xaf35132e63030c21 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 99 type: IBT flags: 0x40218400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.941944:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37c00/0xaf35132e63030af4 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 99 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.941961:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37c00/0xaf35132e63030af4 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 99 type: IBT flags: 0x50210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.941969:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37c00/0xaf35132e63030af4 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 99 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.941976:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.941981:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.941985:0:3609:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279400/0xaf35132e630357ca lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 99 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.941999:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf244b37c00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.942000:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c000/0xaf35132e63030c21 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 99 type: IBT flags: 0x44a19400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.942008:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37c00/0xaf35132e63030af4 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 100 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.942011:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c000/0xaf35132e63030c21 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 100 type: IBT flags: 0x44a19400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.942038:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37c00/0xaf35132e63030af4 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 99 type: IBT flags: 0x40218400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.942060:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243377400/0xaf35132e63030cad lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 98 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.942069:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243377400/0xaf35132e63030cad lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 98 type: IBT flags: 0x50210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.942076:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243377400/0xaf35132e63030cad lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 98 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.942077:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298644.942088:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298644.942097:0:3610:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279400/0xaf35132e630357ca lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 98 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.942119:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf243377400 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298644.942120:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37c00/0xaf35132e63030af4 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 98 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.942163:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37c00/0xaf35132e63030af4 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 98 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.942170:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243377400/0xaf35132e63030cad lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 99 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.942188:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243377400/0xaf35132e63030cad lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 97 type: IBT flags: 0x40218400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.942200:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.942205:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.942208:0:3609:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279400/0xaf35132e630357ca lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 97 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.942219:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243377400/0xaf35132e63030cad lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 97 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.942220:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09200/0xaf35132e63031280 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 98 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.942228:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243377400/0xaf35132e63030cad lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 98 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.942253:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09200/0xaf35132e63031280 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 97 type: IBT flags: 0x50210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.942262:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09200/0xaf35132e63031280 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 96 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.942265:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3821e7400/0xaf35132e63030cc9 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 97 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.942290:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3821e7400/0xaf35132e63030cc9 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 96 type: IBT flags: 0x50210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.942295:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3821e7400/0xaf35132e63030cc9 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 96 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.942296:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf244d09200 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.942301:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf3821e7400 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298644.942304:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09200/0xaf35132e63031280 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 96 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.942305:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3821e7400/0xaf35132e63030cc9 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 97 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.942332:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3821e7400/0xaf35132e63030cc9 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 97 type: IBT flags: 0x40218400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.942335:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09200/0xaf35132e63031280 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 97 type: IBT flags: 0x40218400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.942371:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.942374:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.942375:0:3609:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279400/0xaf35132e630357ca lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 96 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.942384:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.942384:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3821e7400/0xaf35132e63030cc9 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 96 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.942396:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.942398:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3821e7400/0xaf35132e63030cc9 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 96 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.942401:0:3610:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279400/0xaf35132e630357ca lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 97 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.942415:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09200/0xaf35132e63031280 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 95 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.942421:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3821e6e00/0xaf35132e63030d78 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 96 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.942425:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09200/0xaf35132e63031280 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 96 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.942442:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3821e6e00/0xaf35132e63030d78 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 96 type: IBT flags: 0x50210400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.942448:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3821e6e00/0xaf35132e63030d78 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 94 type: IBT flags: 0x40210400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.942451:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf3821e6e00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.942453:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3821e6e00/0xaf35132e63030d78 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 94 type: IBT flags: 0x40210400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.942455:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2c00/0xaf35132e63031416 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 95 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.942465:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2c00/0xaf35132e63031416 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 94 type: IBT flags: 0x50210400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.942472:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2c00/0xaf35132e63031416 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 95 type: IBT flags: 0x40210400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.942472:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3821e6e00/0xaf35132e63030d78 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 95 type: IBT flags: 0x40218400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.942493:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298644.942494:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf244eb2c00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.942502:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.942504:0:3609:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279400/0xaf35132e630357ca lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 94 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.942516:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3821e6e00/0xaf35132e63030d78 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 94 type: IBT flags: 0x44a19400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.942517:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2c00/0xaf35132e63031416 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 95 type: IBT flags: 0x40210400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.942529:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3821e6e00/0xaf35132e63030d78 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 95 type: IBT flags: 0x44a19400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.942554:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2c00/0xaf35132e63031416 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 95 type: IBT flags: 0x40218400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.942561:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08000/0xaf35132e6303114c lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 94 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.942567:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08000/0xaf35132e6303114c lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 93 type: IBT flags: 0x50210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.942572:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08000/0xaf35132e6303114c lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 93 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.942577:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf244d08000 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298644.942579:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298644.942583:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298644.942586:0:3610:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279400/0xaf35132e630357ca lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 93 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.942597:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08000/0xaf35132e6303114c lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 93 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.942598:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2c00/0xaf35132e63031416 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 94 type: IBT flags: 0x44a19400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.942616:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08000/0xaf35132e6303114c lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 94 type: IBT flags: 0x40218400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.942631:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2c00/0xaf35132e63031416 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 94 type: IBT flags: 0x44a19400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.942634:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.942640:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.942641:0:3609:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279400/0xaf35132e630357ca lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 94 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.942652:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08000/0xaf35132e6303114c lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 93 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.942658:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08000/0xaf35132e6303114c lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 92 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.942666:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09800/0xaf35132e63031677 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 91 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.942676:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09800/0xaf35132e63031677 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 91 type: IBT flags: 0x50210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.942684:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09a00/0xaf35132e63031375 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 92 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.942685:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09800/0xaf35132e63031677 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 92 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.942709:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09a00/0xaf35132e63031375 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 92 type: IBT flags: 0x50210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.942722:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09a00/0xaf35132e63031375 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 91 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.942723:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf244d09800 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298644.942733:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09800/0xaf35132e63031677 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 92 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.942733:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf244d09a00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.942736:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09a00/0xaf35132e63031375 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 92 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.942761:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09a00/0xaf35132e63031375 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 92 type: IBT flags: 0x40218400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.942770:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09800/0xaf35132e63031677 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 92 type: IBT flags: 0x40218400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.942772:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.942780:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.942783:0:3609:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279400/0xaf35132e630357ca lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 92 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.942802:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09a00/0xaf35132e63031375 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 91 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.942806:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09a00/0xaf35132e63031375 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 91 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.942808:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298644.942814:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298644.942818:0:3610:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279400/0xaf35132e630357ca lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 90 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.942828:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09400/0xaf35132e63031535 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 90 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.942829:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09800/0xaf35132e63031677 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 91 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.942863:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09400/0xaf35132e63031535 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 91 type: IBT flags: 0x50210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.942864:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09800/0xaf35132e63031677 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 91 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.942900:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09400/0xaf35132e63031535 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 90 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.942905:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf244d09400 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.942908:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09400/0xaf35132e63031535 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 89 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.942911:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692200/0xaf35132e63031956 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 90 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.942920:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692200/0xaf35132e63031956 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 89 type: IBT flags: 0x50210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.942928:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692200/0xaf35132e63031956 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 89 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.942929:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09400/0xaf35132e63031535 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 90 type: IBT flags: 0x40218400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.942957:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf341692200 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.942958:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.942965:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.942968:0:3609:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279400/0xaf35132e630357ca lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 89 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.942978:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09400/0xaf35132e63031535 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 89 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.942985:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09400/0xaf35132e63031535 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 89 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.943070:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692200/0xaf35132e63031956 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 88 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.943070:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08e00/0xaf35132e63031757 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 89 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.943084:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08e00/0xaf35132e63031757 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 89 type: IBT flags: 0x50210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.943101:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08e00/0xaf35132e63031757 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 88 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.943103:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692200/0xaf35132e63031956 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 89 type: IBT flags: 0x40218400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.943109:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf244d08e00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.943115:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08e00/0xaf35132e63031757 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 89 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.943144:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08e00/0xaf35132e63031757 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 88 type: IBT flags: 0x40218400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.943151:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.943154:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.943155:0:3609:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279400/0xaf35132e630357ca lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 88 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.943162:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08e00/0xaf35132e63031757 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 88 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.943166:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08e00/0xaf35132e63031757 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 88 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.943181:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298644.943186:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298644.943189:0:3610:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279400/0xaf35132e630357ca lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 87 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.943193:0:7411:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433df480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:7411:1866777605059200:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298644.943211:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08c00/0xaf35132e63031814 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 88 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.943212:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692200/0xaf35132e63031956 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 88 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.943229:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692200/0xaf35132e63031956 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 88 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.943245:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08c00/0xaf35132e63031814 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 88 type: IBT flags: 0x50210400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.943254:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08c00/0xaf35132e63031814 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 87 type: IBT flags: 0x40210400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.943259:0:7411:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279400/0xaf35132e630357ca lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 88 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.943260:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf244d08c00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.943265:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08c00/0xaf35132e63031814 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 88 type: IBT flags: 0x40210400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.943289:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08c00/0xaf35132e63031814 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 87 type: IBT flags: 0x40218400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.943304:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.943307:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.943310:0:3609:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279400/0xaf35132e630357ca lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 87 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.943323:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08c00/0xaf35132e63031814 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 87 type: IBT flags: 0x44a19400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.943330:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08c00/0xaf35132e63031814 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 86 type: IBT flags: 0x44a19400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.943337:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244f3be00/0xaf35132e63031d93 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 86 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.943346:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244f3be00/0xaf35132e63031d93 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 85 type: IBT flags: 0x50210400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.943354:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244f3be00/0xaf35132e63031d93 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 85 type: IBT flags: 0x40210400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.943355:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09000/0xaf35132e630318bc lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 86 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.943380:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09000/0xaf35132e630318bc lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 85 type: IBT flags: 0x50210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.943385:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09000/0xaf35132e630318bc lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 85 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.943386:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf244f3be00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.943396:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf244d09000 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298644.943397:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244f3be00/0xaf35132e63031d93 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 86 type: IBT flags: 0x40210400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.943400:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09000/0xaf35132e630318bc lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 86 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.943422:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09000/0xaf35132e630318bc lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 86 type: IBT flags: 0x40218400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.943446:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:3.0:1780298644.943462:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244f3be00/0xaf35132e63031d93 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 85 type: IBT flags: 0x40218400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.943462:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.943472:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.943478:0:3609:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279400/0xaf35132e630357ca lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 86 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.943494:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09000/0xaf35132e630318bc lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 85 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.943497:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609644928 00010000:00010000:1.0:1780298644.943504:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d09000/0xaf35132e630318bc lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 85 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.943505:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609644928 00010000:00010000:2.0:1780298644.943536:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2600/0xaf35132e630356dc lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 84 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d460f expref: 61 pid: 6477 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.943544:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08800/0xaf35132e63031902 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 85 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.943553:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08800/0xaf35132e63031902 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 84 type: IBT flags: 0x50210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.943561:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08800/0xaf35132e63031902 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 84 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.943567:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298644.943573:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298644.943576:0:3610:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279400/0xaf35132e630357ca lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 84 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.943579:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15241 00010000:00010000:1.0:1780298644.943594:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf244d08800 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298644.943595:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244f3be00/0xaf35132e63031d93 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 84 type: IBT flags: 0x44a19400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.943595:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243844a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+61:2400:x1866777609644928:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298644.943598:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08800/0xaf35132e63031902 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 85 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.943625:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298644.943636:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08800/0xaf35132e63031902 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 86 type: IBT flags: 0x40218400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.943640:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244f3be00/0xaf35132e63031d93 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 87 type: IBT flags: 0x44a19400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.943640:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2600/0xaf35132e630356dc lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 87 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d460f expref: 61 pid: 6477 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.943655:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.943669:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.943675:0:3609:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279400/0xaf35132e630357ca lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 87 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.943696:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08800/0xaf35132e63031902 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 84 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.943701:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08800/0xaf35132e63031902 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 84 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.943707:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2600/0xaf35132e630356dc lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 85 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d460f expref: 60 pid: 6477 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.943719:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d78000/0xaf35132e63031f6f lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 83 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.943728:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d78000/0xaf35132e63031f6f lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 83 type: IBT flags: 0x50210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.943736:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d78000/0xaf35132e63031f6f lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 83 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.943736:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841da00/0xaf35132e63031917 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 84 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.943759:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841da00/0xaf35132e63031917 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 84 type: IBT flags: 0x50210400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.943768:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841da00/0xaf35132e63031917 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 83 type: IBT flags: 0x40210400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.943779:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf34841da00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298644.943780:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf375d78000 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:2.0:1780298644.943781:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2600/0xaf35132e630356dc lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 83 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d460f expref: 60 pid: 6477 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.943783:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841da00/0xaf35132e63031917 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 84 type: IBT flags: 0x40210400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.943790:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d78000/0xaf35132e63031f6f lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 85 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.943812:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841da00/0xaf35132e63031917 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 85 type: IBT flags: 0x40218400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.943835:0:6462:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298644.943841:0:6462:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279400/0xaf35132e630357ca lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 82 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.943854:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279400/0xaf35132e630357ca lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 82 type: IBT flags: 0x40290000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.943861:0:6462:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279400/0xaf35132e630357ca lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 82 type: IBT flags: 0x40290000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.943871:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d78000/0xaf35132e63031f6f lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 82 type: IBT flags: 0x40218400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.943875:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.943884:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1780298644.943892:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298644.943893:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841da00/0xaf35132e63031917 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 81 type: IBT flags: 0x44a19400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.943897:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.943900:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841da00/0xaf35132e63031917 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 81 type: IBT flags: 0x44a19400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.943906:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298644.943914:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d78000/0xaf35132e63031f6f lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 80 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.943916:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36600/0xaf35132e63031a7c lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 81 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.943937:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243844a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+59:2400:x1866777609644928:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 344us (499us total) trans 0 rc 0/0 00010000:00010000:1.0:1780298644.943958:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36600/0xaf35132e63031a7c lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 81 type: IBT flags: 0x50210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.943959:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d78000/0xaf35132e63031f6f lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 81 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.943969:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15241 00010000:00010000:1.0:1780298644.944000:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36600/0xaf35132e63031a7c lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 80 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.944008:0:7411:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279400/0xaf35132e630357ca lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 81 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.944010:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf244b36600 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298644.944015:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be200/0xaf35132e630321a6 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 80 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.944028:0:7411:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279400/0xaf35132e630357ca lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 80 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.944031:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be200/0xaf35132e630321a6 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 80 type: IBT flags: 0x50210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.944058:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be200/0xaf35132e630321a6 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 79 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.944059:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36600/0xaf35132e63031a7c lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 80 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.944078:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf2445be200 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.944081:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36600/0xaf35132e63031a7c lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 79 type: IBT flags: 0x40218400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.944083:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be200/0xaf35132e630321a6 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 80 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.944093:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.944097:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.944100:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36600/0xaf35132e63031a7c lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 79 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.944101:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be200/0xaf35132e630321a6 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 80 type: IBT flags: 0x40218400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.944116:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36600/0xaf35132e63031a7c lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 80 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.944151:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244f3aa00/0xaf35132e63031c19 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 78 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1780298644.944152:0:7411:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991100 00010000:00010000:1.0:1780298644.944159:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244f3aa00/0xaf35132e63031c19 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 78 type: IBT flags: 0x50210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.944165:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244f3aa00/0xaf35132e63031c19 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 78 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.944170:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf244f3aa00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298644.944171:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298644.944175:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.944181:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244f3aa00/0xaf35132e63031c19 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 79 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.944182:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be200/0xaf35132e630321a6 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 78 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1780298644.944195:0:7411:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000cc6dff89 reply state 000000006bc1844b transno 4294991101 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279400/0xaf35132e630357ca lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 80 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.944205:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244f3aa00/0xaf35132e63031c19 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 80 type: IBT flags: 0x40218400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.944225:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be200/0xaf35132e630321a6 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 80 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.944230:0:7411:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279400/0xaf35132e630357ca lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 80 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.944291:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be400/0xaf35132e630321c9 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 77 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.944305:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be400/0xaf35132e630321c9 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 77 type: IBT flags: 0x50210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.944311:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be400/0xaf35132e630321c9 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 77 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.944312:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298644.944321:0:7411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1780298644.944326:0:7411:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1780298644.944331:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf2445be400 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.944332:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.944339:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.944342:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244f3aa00/0xaf35132e63031c19 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 77 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.944343:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be400/0xaf35132e630321c9 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 78 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.944357:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244f3aa00/0xaf35132e63031c19 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 78 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.944370:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be400/0xaf35132e630321c9 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 78 type: IBT flags: 0x40218400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.944375:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b200/0xaf35132e63031fed lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 77 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.944383:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b200/0xaf35132e63031fed lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 76 type: IBT flags: 0x50210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.944389:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b200/0xaf35132e63031fed lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 76 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.944396:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf243d9b200 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298644.944398:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00000100:00100000:0.0:1780298644.944398:0:7411:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243844700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+61:29360:x1866777609644800:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 3767us (3880us total) trans 4294991101 rc 0/0 00010000:00010000:3.0:1780298644.944408:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.944415:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b200/0xaf35132e63031fed lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 77 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.944416:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be400/0xaf35132e630321c9 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 76 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.944421:0:7411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62050 00000020:00080000:0.0:1780298644.944430:0:7411:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:1.0:1780298644.944437:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b200/0xaf35132e63031fed lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 77 type: IBT flags: 0x40218400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.944454:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be400/0xaf35132e630321c9 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 76 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.944457:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.944461:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.944465:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b200/0xaf35132e63031fed lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 77 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.944486:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b200/0xaf35132e63031fed lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 77 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.944503:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf600/0xaf35132e63032320 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 74 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.944512:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf600/0xaf35132e63032320 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 74 type: IBT flags: 0x50210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.944519:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf600/0xaf35132e63032320 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 74 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.944519:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d78a00/0xaf35132e63032087 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 75 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.944534:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d78a00/0xaf35132e63032087 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 75 type: IBT flags: 0x50210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.944556:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf2445bf600 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.944556:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d78a00/0xaf35132e63032087 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 74 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.944567:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf600/0xaf35132e63032320 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 75 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.944568:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf375d78a00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.944574:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d78a00/0xaf35132e63032087 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 75 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.944578:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d78a00/0xaf35132e63032087 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 75 type: IBT flags: 0x40218400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.944583:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.944584:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.944585:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d78a00/0xaf35132e63032087 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 75 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.944589:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375d78a00/0xaf35132e63032087 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 75 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.944595:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a600/0xaf35132e630320c6 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 74 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.944597:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a600/0xaf35132e630320c6 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 74 type: IBT flags: 0x50210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.944601:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a600/0xaf35132e630320c6 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 74 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.944605:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf243d9a600 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.944606:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a600/0xaf35132e630320c6 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 74 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.944610:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a600/0xaf35132e630320c6 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 74 type: IBT flags: 0x40218400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.944614:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.944625:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.944630:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a600/0xaf35132e630320c6 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 74 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.944634:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf600/0xaf35132e63032320 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 74 type: IBT flags: 0x40218400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.944644:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a600/0xaf35132e630320c6 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 74 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.944653:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298644.944657:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.944660:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37a00/0xaf35132e63032160 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 72 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.944661:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf600/0xaf35132e63032320 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 73 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.944698:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37a00/0xaf35132e63032160 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 72 type: IBT flags: 0x50210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.944699:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf600/0xaf35132e63032320 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 73 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.944720:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37a00/0xaf35132e63032160 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 73 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.944727:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf244b37a00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298644.944731:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ba00/0xaf35132e63032343 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 71 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.944738:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ba00/0xaf35132e63032343 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 71 type: IBT flags: 0x50210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.944745:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ba00/0xaf35132e63032343 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 71 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.944746:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37a00/0xaf35132e63032160 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 72 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.944765:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf243d9ba00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.944770:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37a00/0xaf35132e63032160 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 71 type: IBT flags: 0x40218400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.944771:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ba00/0xaf35132e63032343 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 72 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.944810:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ba00/0xaf35132e63032343 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 72 type: IBT flags: 0x40218400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.944844:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.944848:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.944852:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37a00/0xaf35132e63032160 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 71 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.944855:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298644.944860:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.944860:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37a00/0xaf35132e63032160 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 71 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.944866:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ba00/0xaf35132e63032343 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 72 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.944877:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ba00/0xaf35132e63032343 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 70 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.944880:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36400/0xaf35132e63032240 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 71 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.944889:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36400/0xaf35132e63032240 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 69 type: IBT flags: 0x50210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.944896:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bee00/0xaf35132e63032397 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 69 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.944897:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36400/0xaf35132e63032240 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 70 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.944932:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bee00/0xaf35132e63032397 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 70 type: IBT flags: 0x50210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.944937:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bee00/0xaf35132e63032397 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 69 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.944938:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf244b36400 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298644.944944:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf2445bee00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.944947:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36400/0xaf35132e63032240 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 69 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.944948:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bee00/0xaf35132e63032397 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 70 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.944973:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bee00/0xaf35132e63032397 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 70 type: IBT flags: 0x40218400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.944980:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36400/0xaf35132e63032240 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 70 type: IBT flags: 0x40218400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.944983:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298644.944994:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298644.944997:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bee00/0xaf35132e63032397 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 70 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.945003:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bee00/0xaf35132e63032397 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 69 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.945007:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.945012:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298644.945015:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36000/0xaf35132e630325e3 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 68 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.945016:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36400/0xaf35132e63032240 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 69 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.945037:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36400/0xaf35132e63032240 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 68 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.945038:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36000/0xaf35132e630325e3 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 69 type: IBT flags: 0x50210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.945067:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36000/0xaf35132e630325e3 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 69 type: IBT flags: 0x40210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.945076:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf244b36000 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298644.945079:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36000/0xaf35132e630325e3 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 67 type: IBT flags: 0x40210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.945082:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9bc00/0xaf35132e63032542 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 68 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.945094:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9bc00/0xaf35132e63032542 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 67 type: IBT flags: 0x50210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.945101:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9bc00/0xaf35132e63032542 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 67 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.945102:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36000/0xaf35132e630325e3 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 68 type: IBT flags: 0x40218400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.945155:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf243d9bc00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298644.945162:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.945162:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9bc00/0xaf35132e63032542 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 67 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.945179:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.945183:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9bc00/0xaf35132e63032542 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 67 type: IBT flags: 0x40218400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.945184:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36000/0xaf35132e630325e3 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 68 type: IBT flags: 0x44a19400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.945223:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36000/0xaf35132e630325e3 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 67 type: IBT flags: 0x44a19400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.945229:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.945233:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298644.945239:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244f3bc00/0xaf35132e63032606 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 66 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.945239:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9bc00/0xaf35132e63032542 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 67 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.945261:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9bc00/0xaf35132e63032542 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 67 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.945265:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244f3bc00/0xaf35132e63032606 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 67 type: IBT flags: 0x50210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.945284:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244f3bc00/0xaf35132e63032606 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 65 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.945285:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244f3a400/0xaf35132e6303272c lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 66 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.945312:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244f3a400/0xaf35132e6303272c lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 66 type: IBT flags: 0x50210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.945320:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244f3a400/0xaf35132e6303272c lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 65 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.945321:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf244f3bc00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298644.945331:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244f3bc00/0xaf35132e63032606 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 66 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.945332:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf244f3a400 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.945338:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244f3a400/0xaf35132e6303272c lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 66 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.945359:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244f3a400/0xaf35132e6303272c lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 66 type: IBT flags: 0x40218400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.945392:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298644.945393:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244f3bc00/0xaf35132e63032606 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 65 type: IBT flags: 0x40218400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.945398:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.945404:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244f3a400/0xaf35132e6303272c lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 66 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.945419:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244f3a400/0xaf35132e6303272c lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 65 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.945422:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298644.945426:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298644.945429:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244f3bc00/0xaf35132e63032606 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 64 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.945430:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafe00/0xaf35132e63032b69 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 65 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.945445:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafe00/0xaf35132e63032b69 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 65 type: IBT flags: 0x50210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.945452:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafe00/0xaf35132e63032b69 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 65 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.945458:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf242bafe00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.945461:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafe00/0xaf35132e63032b69 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 65 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.945469:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafe00/0xaf35132e63032b69 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 65 type: IBT flags: 0x40218400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.945481:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.945483:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.945485:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafe00/0xaf35132e63032b69 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 65 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.945489:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafe00/0xaf35132e63032b69 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 65 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.945497:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a200/0xaf35132e63032c42 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 64 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.945501:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a200/0xaf35132e63032c42 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 64 type: IBT flags: 0x50210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.945505:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a200/0xaf35132e63032c42 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 64 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.945510:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf243d9a200 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.945512:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a200/0xaf35132e63032c42 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 64 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.945515:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a200/0xaf35132e63032c42 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 64 type: IBT flags: 0x40218400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.945520:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.945521:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.945523:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a200/0xaf35132e63032c42 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 64 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.945543:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a200/0xaf35132e63032c42 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 64 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.945545:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244f3bc00/0xaf35132e63032606 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 64 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.945583:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be800/0xaf35132e63032d53 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 61 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.945593:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be800/0xaf35132e63032d53 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 61 type: IBT flags: 0x50210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.945600:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be800/0xaf35132e63032d53 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 61 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.945601:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b600/0xaf35132e63032645 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 62 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.945614:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:3.0:1780298644.945618:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b600/0xaf35132e63032645 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 61 type: IBT flags: 0x50210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.945626:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b600/0xaf35132e63032645 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 61 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.945626:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf2445be800 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.945633:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be800/0xaf35132e63032d53 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 62 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.945643:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf243d9b600 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.945649:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be800/0xaf35132e63032d53 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 61 type: IBT flags: 0x40218400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.945651:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b600/0xaf35132e63032645 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 62 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.945660:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609645056 00000100:00100000:2.0:1780298644.945667:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609645056 00010000:00010000:1.0:1780298644.945678:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298644.945679:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b600/0xaf35132e63032645 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 61 type: IBT flags: 0x40218400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.945684:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.945688:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be800/0xaf35132e63032d53 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 62 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.945708:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10132 00010000:00010000:1.0:1780298644.945710:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445be800/0xaf35132e63032d53 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 61 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.945718:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00000100:00100000:2.0:1780298644.945728:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243845c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:29360:x1866777609645056:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.945729:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.945737:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfe00/0xaf35132e63032e48 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 60 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.945738:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b600/0xaf35132e63032645 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 61 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.945744:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.945755:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000aa41763f/0xaf35132e630357d1 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d46fd expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.945756:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfe00/0xaf35132e63032e48 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 61 type: IBT flags: 0x50210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.945797:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfe00/0xaf35132e63032e48 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 61 type: IBT flags: 0x40210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.945799:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b600/0xaf35132e63032645 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 61 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.945816:0:19992:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e630357a7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [5169152->38723583] (req 5169152->5177343) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d46da expref: 13 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.945826:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf2445bfe00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.945837:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfe00/0xaf35132e63032e48 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 61 type: IBT flags: 0x40210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.945847:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(34 169) 00010000:00010000:3.0:1780298644.945848:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36200/0xaf35132e63032709 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 60 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.945859:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(35 169) 00010000:00010000:3.0:1780298644.945866:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36200/0xaf35132e63032709 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 59 type: IBT flags: 0x50210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.945871:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000aa41763f/0xaf35132e630357d1 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d46fd expref: 15 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.945892:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36200/0xaf35132e63032709 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 59 type: IBT flags: 0x40210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.945893:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfe00/0xaf35132e63032e48 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 60 type: IBT flags: 0x40218400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.945921:0:19992:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e630357a7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [5169152->38723583] (req 5169152->5177343) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d46da expref: 13 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.945939:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf244b36200 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.945939:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298644.945946:0:19992:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e630357a7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [5169152->38723583] (req 5169152->5177343) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d46da expref: 13 pid: 8359 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.945947:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298644.945952:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36200/0xaf35132e63032709 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 59 type: IBT flags: 0x40210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.945952:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfe00/0xaf35132e63032e48 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 60 type: IBT flags: 0x44a19400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.945998:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfe00/0xaf35132e63032e48 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 60 type: IBT flags: 0x44a19400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.945999:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36200/0xaf35132e63032709 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 60 type: IBT flags: 0x40218400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.946008:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605059328:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298644.946044:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298644.946047:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.946052:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36800/0xaf35132e630330b0 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 59 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.946053:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36200/0xaf35132e63032709 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 58 type: IBT flags: 0x44a19400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.946077:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36800/0xaf35132e630330b0 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 59 type: IBT flags: 0x50210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.946084:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36200/0xaf35132e63032709 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 59 type: IBT flags: 0x44a19400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.946084:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36800/0xaf35132e630330b0 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 59 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.946092:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00010000:00010000:1.0:1780298644.946109:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf244b36800 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.946115:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36800/0xaf35132e630330b0 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 59 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.946122:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36800/0xaf35132e630330b0 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 57 type: IBT flags: 0x40218400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.946169:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37e00/0xaf35132e6303299b lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 58 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.946198:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37e00/0xaf35132e6303299b lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 57 type: IBT flags: 0x50210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.946207:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37e00/0xaf35132e6303299b lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 57 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.946216:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf244b37e00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.946217:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.946222:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298644.946226:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37e00/0xaf35132e6303299b lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 57 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.946226:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36800/0xaf35132e630330b0 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 58 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.946249:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b36800/0xaf35132e630330b0 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 58 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.946252:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37e00/0xaf35132e6303299b lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 58 type: IBT flags: 0x40218400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.946326:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157800/0xaf35132e6303327e lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 56 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.946334:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157800/0xaf35132e6303327e lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 56 type: IBT flags: 0x50210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.946341:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157800/0xaf35132e6303327e lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 56 type: IBT flags: 0x40210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.946342:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298644.946352:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298644.946358:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37e00/0xaf35132e6303299b lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 56 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.946358:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf243157800 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.946366:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157800/0xaf35132e6303327e lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 57 type: IBT flags: 0x40210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.946371:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37e00/0xaf35132e6303299b lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 57 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.946383:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157800/0xaf35132e6303327e lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 57 type: IBT flags: 0x40218400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.946398:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae200/0xaf35132e630329e8 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 55 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.946406:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae200/0xaf35132e630329e8 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 55 type: IBT flags: 0x50210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.946412:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae200/0xaf35132e630329e8 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 55 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.946413:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.946420:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.946428:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157800/0xaf35132e6303327e lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 56 type: IBT flags: 0x44a19400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.946430:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf242bae200 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298644.946436:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae200/0xaf35132e630329e8 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 56 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.946445:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157800/0xaf35132e6303327e lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 56 type: IBT flags: 0x44a19400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.946479:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae200/0xaf35132e630329e8 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 56 type: IBT flags: 0x40218400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.946494:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298644.946497:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298644.946500:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae200/0xaf35132e630329e8 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 55 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.946501:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37400/0xaf35132e63033334 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 55 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.946526:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae200/0xaf35132e630329e8 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 55 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.946529:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37400/0xaf35132e63033334 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 55 type: IBT flags: 0x50210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.946543:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37400/0xaf35132e63033334 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 53 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.946548:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37600/0xaf35132e63032a27 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 54 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.946555:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37600/0xaf35132e63032a27 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 53 type: IBT flags: 0x50210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.946561:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37600/0xaf35132e63032a27 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 53 type: IBT flags: 0x40210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.946562:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf244b37400 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.946570:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37400/0xaf35132e63033334 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 54 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.946572:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf244b37600 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298644.946580:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37600/0xaf35132e63032a27 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 54 type: IBT flags: 0x40210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.946584:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37400/0xaf35132e63033334 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 54 type: IBT flags: 0x40218400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.946598:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37600/0xaf35132e63032a27 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 54 type: IBT flags: 0x40218400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.946602:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.946609:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.946615:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37400/0xaf35132e63033334 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 53 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.946618:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.946622:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37400/0xaf35132e63033334 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 53 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.946626:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298644.946633:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37600/0xaf35132e63032a27 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 52 type: IBT flags: 0x44a19400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.946636:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156600/0xaf35132e6303354f lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 53 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.946661:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156600/0xaf35132e6303354f lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 53 type: IBT flags: 0x50210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.946683:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37600/0xaf35132e63032a27 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 53 type: IBT flags: 0x44a19400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.946684:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156600/0xaf35132e6303354f lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 53 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.946708:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf243156600 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.946714:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156600/0xaf35132e6303354f lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 52 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.946720:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156600/0xaf35132e6303354f lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 52 type: IBT flags: 0x40218400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.946730:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.946732:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.946735:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156600/0xaf35132e6303354f lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 52 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.946741:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156600/0xaf35132e6303354f lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 52 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.946753:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693400/0xaf35132e630335cd lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 51 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.946758:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693400/0xaf35132e630335cd lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 51 type: IBT flags: 0x50210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.946762:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693400/0xaf35132e630335cd lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 51 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.946765:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf341693400 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.946766:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693400/0xaf35132e630335cd lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 51 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.946771:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693400/0xaf35132e630335cd lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 51 type: IBT flags: 0x40218400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.946781:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.946783:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.946787:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693400/0xaf35132e630335cd lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 50 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.946792:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693400/0xaf35132e630335cd lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 50 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.946796:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b800/0xaf35132e63032c1f lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 51 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.946806:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b800/0xaf35132e63032c1f lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 49 type: IBT flags: 0x50210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.946814:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b800/0xaf35132e63032c1f lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 49 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.946815:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841c600/0xaf35132e630336ec lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 50 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.946846:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841c600/0xaf35132e630336ec lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 49 type: IBT flags: 0x50210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.946855:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841c600/0xaf35132e630336ec lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 49 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.946856:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf243d9b800 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298644.946863:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b800/0xaf35132e63032c1f lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 50 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.946868:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf34841c600 already unused, calling callback (ffffffffc169a4f0) 00000100:00100000:2.0:1780298644.946876:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605059328:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298644.946877:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841c600/0xaf35132e630336ec lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 50 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.946895:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b800/0xaf35132e63032c1f lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 50 type: IBT flags: 0x40218400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.946899:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841c600/0xaf35132e630336ec lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 50 type: IBT flags: 0x40218400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.946918:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000aa41763f/0xaf35132e630357d1 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d46fd expref: 15 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.946920:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298644.946930:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298644.946937:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b800/0xaf35132e63032c1f lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 49 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.946942:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298644.946944:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3600, rc 0) 00010000:00010000:3.0:1780298644.946949:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b800/0xaf35132e63032c1f lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 49 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.946950:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.946956:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841c600/0xaf35132e630336ec lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 50 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.946967:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34841c600/0xaf35132e630336ec lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 48 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.946987:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243845c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:29360:x1866777609645056:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1270us (1379us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.947010:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10132 00010000:00010000:3.0:1780298644.947016:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692e00/0xaf35132e63032cc7 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 47 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.947025:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692e00/0xaf35132e63032cc7 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 47 type: IBT flags: 0x50210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.947033:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692e00/0xaf35132e63032cc7 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 47 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.947035:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafc00/0xaf35132e63033827 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 48 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.947063:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafc00/0xaf35132e63033827 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 47 type: IBT flags: 0x50210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.947071:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafc00/0xaf35132e63033827 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 47 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.947083:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:3.0:1780298644.947098:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf341692e00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.947099:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf242bafc00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.947105:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafc00/0xaf35132e63033827 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 47 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.947108:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692e00/0xaf35132e63032cc7 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 48 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.947150:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafc00/0xaf35132e63033827 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 48 type: IBT flags: 0x40218400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.947165:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692e00/0xaf35132e63032cc7 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 48 type: IBT flags: 0x40218400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.947171:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609645184 00010000:00010000:1.0:1780298644.947175:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00000100:00100000:2.0:1780298644.947177:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609645184 00010000:00010000:1.0:1780298644.947179:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.947183:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafc00/0xaf35132e63033827 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 47 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.947186:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.947189:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafc00/0xaf35132e63033827 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 47 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.947195:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298644.947200:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692e00/0xaf35132e63032cc7 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 48 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.947215:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692e00/0xaf35132e63032cc7 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 47 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.947219:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e630357a7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [5169152->38723583] (req 5169152->5177343) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d46da expref: 14 pid: 8359 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.947226:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499e00/0xaf35132e63033866 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 45 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.947238:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499e00/0xaf35132e63033866 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 45 type: IBT flags: 0x50210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.947245:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf400/0xaf35132e63032ed4 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 46 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.947246:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499e00/0xaf35132e63033866 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 45 type: IBT flags: 0x40210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.947267:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15242 00010000:00010000:3.0:1780298644.947284:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf400/0xaf35132e63032ed4 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 46 type: IBT flags: 0x50210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.947302:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243846a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2399:x1866777609645184:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298644.947306:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf400/0xaf35132e63032ed4 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 45 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.947307:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf341499e00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298644.947319:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf2445bf400 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.947319:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499e00/0xaf35132e63033866 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 46 type: IBT flags: 0x40210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.947325:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298644.947326:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf400/0xaf35132e63032ed4 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 46 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.947355:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e630357a7 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [5169152->38723583] (req 5169152->5177343) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d46da expref: 14 pid: 8359 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.947357:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499e00/0xaf35132e63033866 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 46 type: IBT flags: 0x40218400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.947379:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf400/0xaf35132e63032ed4 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 46 type: IBT flags: 0x40218400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.947400:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.947412:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1780298644.947413:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e630357a7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [5169152->38723583] (req 5169152->5177343) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d46da expref: 14 pid: 8359 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.947418:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499e00/0xaf35132e63033866 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 45 type: IBT flags: 0x44a19400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.947420:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298644.947437:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298644.947443:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf400/0xaf35132e63032ed4 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 46 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.947447:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499e00/0xaf35132e63033866 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 46 type: IBT flags: 0x44a19400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.947449:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf400/0xaf35132e63032ed4 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 46 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.947474:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000000190405/0xaf35132e630357a7 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [5169152->38723583] (req 5169152->5177343) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d46da expref: 14 pid: 8359 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.947488:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37800/0xaf35132e6303305c lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 43 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.947494:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37800/0xaf35132e6303305c lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 43 type: IBT flags: 0x50210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.947496:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(35 169) 00010000:00010000:3.0:1780298644.947504:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37800/0xaf35132e6303305c lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 43 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.947505:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf800/0xaf35132e630338dd lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 44 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.947506:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(35 169) 00010000:00010000:2.0:1780298644.947529:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000aa41763f/0xaf35132e630357d1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d46fd expref: 14 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.947530:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf800/0xaf35132e630338dd lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 43 type: IBT flags: 0x50210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.947550:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf800/0xaf35132e630338dd lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 43 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.947551:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf244b37800 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:2.0:1780298644.947557:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000aa41763f/0xaf35132e630357d1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d46fd expref: 14 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.947558:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37800/0xaf35132e6303305c lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 44 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.947586:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf242baf800 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298644.947600:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37800/0xaf35132e6303305c lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 44 type: IBT flags: 0x40218400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.947600:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf800/0xaf35132e630338dd lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 43 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.947624:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298644.947626:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298644.947628:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37800/0xaf35132e6303305c lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 43 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.947630:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf800/0xaf35132e630338dd lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 44 type: IBT flags: 0x40218400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.947645:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000aa41763f/0xaf35132e630357d1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d46fd expref: 14 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.947673:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37800/0xaf35132e6303305c lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 44 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.947714:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2200/0xaf35132e6303309b lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 42 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.947718:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2200/0xaf35132e6303309b lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 42 type: IBT flags: 0x50210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.947723:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2200/0xaf35132e6303309b lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 42 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.947723:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00000100:00100000:2.0:1780298644.947733:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372f480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777605059456:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298644.947735:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298644.947746:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf344ff2200 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.947748:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf800/0xaf35132e630338dd lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 42 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.947751:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2200/0xaf35132e6303309b lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 43 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.947772:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf800/0xaf35132e630338dd lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 43 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.947774:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2200/0xaf35132e6303309b lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 43 type: IBT flags: 0x40218400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.947799:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298644.947801:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298644.947806:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2200/0xaf35132e6303309b lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 41 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.947807:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae800/0xaf35132e63033915 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 42 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.947817:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2200/0xaf35132e6303309b lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 42 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.947826:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00010000:00010000:1.0:1780298644.947835:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae800/0xaf35132e63033915 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 42 type: IBT flags: 0x50210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.947849:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157e00/0xaf35132e630330d3 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 40 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.947850:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae800/0xaf35132e63033915 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 41 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.947883:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157e00/0xaf35132e630330d3 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 41 type: IBT flags: 0x50210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.947889:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157e00/0xaf35132e630330d3 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 40 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.947890:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf242bae800 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.947898:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae800/0xaf35132e63033915 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 41 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.947901:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf243157e00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298644.947909:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157e00/0xaf35132e630330d3 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 41 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.947914:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae800/0xaf35132e63033915 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 41 type: IBT flags: 0x40218400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.947926:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.947928:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.947930:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae800/0xaf35132e63033915 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 41 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.947934:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae800/0xaf35132e63033915 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 41 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.947946:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499c00/0xaf35132e63033a73 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 40 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.947950:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499c00/0xaf35132e63033a73 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 40 type: IBT flags: 0x50210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.947954:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499c00/0xaf35132e63033a73 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 40 type: IBT flags: 0x40210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.947958:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf341499c00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.947960:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499c00/0xaf35132e63033a73 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 40 type: IBT flags: 0x40210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.947963:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499c00/0xaf35132e63033a73 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 40 type: IBT flags: 0x40218400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.947969:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157e00/0xaf35132e630330d3 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 39 type: IBT flags: 0x40218400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.947970:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.947980:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.947986:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499c00/0xaf35132e63033a73 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 39 type: IBT flags: 0x44a19400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.947988:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298644.947996:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298644.948002:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157e00/0xaf35132e630330d3 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 40 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.948007:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499c00/0xaf35132e63033a73 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 40 type: IBT flags: 0x44a19400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.948009:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157e00/0xaf35132e630330d3 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 40 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.948050:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92c00/0xaf35132e63033e1d lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 37 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.948058:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92c00/0xaf35132e63033e1d lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 37 type: IBT flags: 0x50210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.948066:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92c00/0xaf35132e63033e1d lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 37 type: IBT flags: 0x40210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.948067:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37200/0xaf35132e63033683 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 38 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.948089:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37200/0xaf35132e63033683 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 37 type: IBT flags: 0x50210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.948095:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37200/0xaf35132e63033683 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 37 type: IBT flags: 0x40210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.948095:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf375e92c00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.948102:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92c00/0xaf35132e63033e1d lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 38 type: IBT flags: 0x40210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.948117:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf244b37200 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.948119:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92c00/0xaf35132e63033e1d lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 37 type: IBT flags: 0x40218400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.948121:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37200/0xaf35132e63033683 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 38 type: IBT flags: 0x40210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.948166:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.948174:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.948178:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92c00/0xaf35132e63033e1d lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 38 type: IBT flags: 0x44a19400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.948185:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92c00/0xaf35132e63033e1d lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 38 type: IBT flags: 0x44a19400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.948199:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3a00/0xaf35132e63033eef lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 37 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.948206:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3a00/0xaf35132e63033eef lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 37 type: IBT flags: 0x50210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.948213:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3a00/0xaf35132e63033eef lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 37 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.948219:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf244eb3a00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.948224:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3a00/0xaf35132e63033eef lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 37 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.948244:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3a00/0xaf35132e63033eef lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 36 type: IBT flags: 0x40218400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.948246:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37200/0xaf35132e63033683 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 37 type: IBT flags: 0x40218400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.948293:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298644.948296:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298644.948300:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37200/0xaf35132e63033683 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 36 type: IBT flags: 0x44a19400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.948305:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.948308:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298644.948309:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244b37200/0xaf35132e63033683 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 36 type: IBT flags: 0x44a19400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.948312:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3a00/0xaf35132e63033eef lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 37 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.948339:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3a00/0xaf35132e63033eef lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 36 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.948350:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3200/0xaf35132e63033d7c lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 34 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.948356:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3200/0xaf35132e63033d7c lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 34 type: IBT flags: 0x50210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.948361:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3200/0xaf35132e63033d7c lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 34 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.948362:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93c00/0xaf35132e630340cb lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 35 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.948376:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93c00/0xaf35132e630340cb lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 34 type: IBT flags: 0x50210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.948383:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf244eb3200 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.948384:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93c00/0xaf35132e630340cb lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 34 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.948390:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3200/0xaf35132e63033d7c lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 35 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.948427:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf375e93c00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298644.948428:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3200/0xaf35132e63033d7c lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 34 type: IBT flags: 0x40218400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.948434:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93c00/0xaf35132e630340cb lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 35 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.948449:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298644.948452:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298644.948455:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3200/0xaf35132e63033d7c lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 34 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.948456:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93c00/0xaf35132e630340cb lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 35 type: IBT flags: 0x40218400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.948501:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3200/0xaf35132e63033d7c lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 34 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.948504:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.948512:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.948517:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93c00/0xaf35132e630340cb lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 34 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.948524:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93400/0xaf35132e63033dc9 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 33 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.948524:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93c00/0xaf35132e630340cb lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 34 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.948609:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93400/0xaf35132e63033dc9 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 32 type: IBT flags: 0x50210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.948617:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93400/0xaf35132e63033dc9 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 32 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.948619:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92a00/0xaf35132e63034491 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 33 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.948632:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92a00/0xaf35132e63034491 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 32 type: IBT flags: 0x50210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.948639:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf375e93400 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.948640:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92a00/0xaf35132e63034491 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 32 type: IBT flags: 0x40210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.948647:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93400/0xaf35132e63033dc9 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 33 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.948650:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf375e92a00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.948654:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92a00/0xaf35132e63034491 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 33 type: IBT flags: 0x40210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.948671:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92a00/0xaf35132e63034491 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 32 type: IBT flags: 0x40218400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.948673:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93400/0xaf35132e63033dc9 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 33 type: IBT flags: 0x40218400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.948678:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.948686:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.948692:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92a00/0xaf35132e63034491 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 32 type: IBT flags: 0x44a19400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.948696:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92a00/0xaf35132e63034491 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 32 type: IBT flags: 0x44a19400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.948737:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372f480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777605059456:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298644.948755:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298644.948755:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf000/0xaf35132e6303456a lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 31 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.948762:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf000/0xaf35132e6303456a lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 31 type: IBT flags: 0x50210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.948767:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf000/0xaf35132e6303456a lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 31 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.948768:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298644.948781:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.948790:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf2445bf000 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298644.948791:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93400/0xaf35132e63033dc9 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 31 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.948793:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243846a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2399:x1866777609645184:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1496us (1713us total) trans 0 rc 0/0 00010000:00010000:1.0:1780298644.948798:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf000/0xaf35132e6303456a lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 32 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.948832:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15242 00010000:00010000:3.0:1780298644.948835:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93400/0xaf35132e63033dc9 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 31 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.948835:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf000/0xaf35132e6303456a lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 32 type: IBT flags: 0x40218400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.948870:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:1.0:1780298644.948891:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.948894:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.948897:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf000/0xaf35132e6303456a lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 30 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.948901:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bf000/0xaf35132e6303456a lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 30 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.948916:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfa00/0xaf35132e6303458d lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 29 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.948922:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfa00/0xaf35132e6303458d lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 29 type: IBT flags: 0x50210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.948929:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfa00/0xaf35132e6303458d lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 29 type: IBT flags: 0x40210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.948936:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf2445bfa00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.948939:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfa00/0xaf35132e6303458d lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 29 type: IBT flags: 0x40210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.948945:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfa00/0xaf35132e6303458d lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 29 type: IBT flags: 0x40218400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.948951:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609645312 00010000:00010000:3.0:1780298644.948953:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2a00/0xaf35132e6303440c lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 30 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.948963:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2a00/0xaf35132e6303440c lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 29 type: IBT flags: 0x50210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.948963:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609645312 00010000:00010000:3.0:1780298644.948972:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2a00/0xaf35132e6303440c lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 29 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.948980:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf244eb2a00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.948980:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.948985:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298644.948989:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2a00/0xaf35132e6303440c lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 29 type: IBT flags: 0x40210400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.948990:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfa00/0xaf35132e6303458d lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 30 type: IBT flags: 0x44a19400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.949014:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf2445bfa00/0xaf35132e6303458d lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 30 type: IBT flags: 0x44a19400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.949024:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10133 00010000:00010000:1.0:1780298644.949034:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278e00/0xaf35132e63034777 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 29 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.949045:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243845180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:29360:x1866777609645312:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.949046:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278e00/0xaf35132e63034777 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 29 type: IBT flags: 0x50210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.949066:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278e00/0xaf35132e63034777 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 29 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.949071:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf243278e00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.949074:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278e00/0xaf35132e63034777 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 29 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.949078:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.949090:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278e00/0xaf35132e63034777 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 29 type: IBT flags: 0x40218400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.949093:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2a00/0xaf35132e6303440c lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 29 type: IBT flags: 0x40218400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.949096:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000024bbcf41/0xaf35132e630357d8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4704 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.949166:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.949171:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.949175:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278e00/0xaf35132e63034777 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 28 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.949178:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e63035722 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [69632->2764799] (req 69632->73727) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4655 expref: 15 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.949179:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.949189:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278e00/0xaf35132e63034777 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 28 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.949190:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298644.949197:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2a00/0xaf35132e6303440c lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 29 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.949213:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000075e99a4b/0xaf35132e630356ff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2764800->36319231] (req 2764800->2777087) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4632 expref: 15 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.949228:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2a00/0xaf35132e6303440c lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 29 type: IBT flags: 0x44a19400000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.949257:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e630348ea lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 26 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.949260:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 2 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(34 122) 00010000:00010000:1.0:1780298644.949266:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e630348ea lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 26 type: IBT flags: 0x50210400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.949270:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(36 122) 00010000:00010000:2.0:1780298644.949290:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000024bbcf41/0xaf35132e630357d8 lrc: 5/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4704 expref: 16 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.949322:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e630348ea lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 27 type: IBT flags: 0x40210400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.949323:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2200/0xaf35132e630344de lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 27 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.949325:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000075e99a4b/0xaf35132e630356ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2764800->36319231] (req 2764800->2777087) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4632 expref: 15 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.949357:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2200/0xaf35132e630344de lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 26 type: IBT flags: 0x50210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.949365:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf243278800 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298644.949366:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2200/0xaf35132e630344de lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 26 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.949366:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000075e99a4b/0xaf35132e630356ff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2764800->36319231] (req 2764800->2777087) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4632 expref: 15 pid: 8359 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.949371:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e630348ea lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 27 type: IBT flags: 0x40210400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.949405:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf244eb2200 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.949413:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e630348ea lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 27 type: IBT flags: 0x40218400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.949414:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2200/0xaf35132e630344de lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 27 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.949424:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605059584:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298644.949455:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.949459:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298644.949463:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2200/0xaf35132e630344de lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 26 type: IBT flags: 0x40218400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.949464:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e630348ea lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 27 type: IBT flags: 0x44a19400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.949494:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e630348ea lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 27 type: IBT flags: 0x44a19400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.949500:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298644.949503:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298644.949509:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2200/0xaf35132e630344de lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 25 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.949514:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2200/0xaf35132e630344de lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 25 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.949515:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3821e6c00/0xaf35132e63034bec lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 26 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.949533:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e63035722 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [69632->2764799] (req 69632->73727) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4655 expref: 15 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.949533:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3821e6c00/0xaf35132e63034bec lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 24 type: IBT flags: 0x50210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.949546:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08400/0xaf35132e63034873 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 25 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.949547:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3821e6c00/0xaf35132e63034bec lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 25 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.949560:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e63035722 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [69632->2764799] (req 69632->73727) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4655 expref: 15 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.949585:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08400/0xaf35132e63034873 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 25 type: IBT flags: 0x50210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.949593:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08400/0xaf35132e63034873 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 24 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.949595:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605059712:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298644.949595:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf3821e6c00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298644.949604:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf244d08400 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.949606:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3821e6c00/0xaf35132e63034bec lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 24 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.949608:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08400/0xaf35132e63034873 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 25 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.949626:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3821e6c00/0xaf35132e63034bec lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 25 type: IBT flags: 0x40218400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.949637:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00010000:00010000:1.0:1780298644.949642:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.949646:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.949650:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3821e6c00/0xaf35132e63034bec lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 25 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.949659:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3821e6c00/0xaf35132e63034bec lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 25 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.949669:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08400/0xaf35132e63034873 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 24 type: IBT flags: 0x40218400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.949682:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278200/0xaf35132e63034c0f lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 23 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.949688:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278200/0xaf35132e63034c0f lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 23 type: IBT flags: 0x50210400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.949694:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278200/0xaf35132e63034c0f lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 23 type: IBT flags: 0x40210400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.949695:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298644.949711:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.949715:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf243278200 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.949718:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278200/0xaf35132e63034c0f lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 23 type: IBT flags: 0x40210400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.949722:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278200/0xaf35132e63034c0f lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 23 type: IBT flags: 0x40218400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.949730:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.949731:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.949733:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278200/0xaf35132e63034c0f lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 23 type: IBT flags: 0x44a19400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.949736:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278200/0xaf35132e63034c0f lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 23 type: IBT flags: 0x44a19400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.949742:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08200/0xaf35132e63034c24 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 22 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.949745:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08200/0xaf35132e63034c24 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 22 type: IBT flags: 0x50210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.949749:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08200/0xaf35132e63034c24 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 22 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.949752:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf244d08200 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298644.949753:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08400/0xaf35132e63034873 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 22 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.949755:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08200/0xaf35132e63034c24 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 23 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.949791:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08200/0xaf35132e63034c24 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 23 type: IBT flags: 0x40218400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.949800:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08400/0xaf35132e63034873 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 23 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.949805:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.949810:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.949813:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08200/0xaf35132e63034c24 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 21 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.949817:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244d08200/0xaf35132e63034c24 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 21 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.949819:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279e00/0xaf35132e63034a2c lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 22 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.949851:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279e00/0xaf35132e63034a2c lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 20 type: IBT flags: 0x50210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.949860:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279e00/0xaf35132e63034a2c lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 20 type: IBT flags: 0x40210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.949860:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c400/0xaf35132e63034d20 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 21 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.949867:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c400/0xaf35132e63034d20 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 21 type: IBT flags: 0x50210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.949871:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c400/0xaf35132e63034d20 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 21 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.949874:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf34830c400 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.949875:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c400/0xaf35132e63034d20 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 21 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.949878:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c400/0xaf35132e63034d20 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 21 type: IBT flags: 0x40218400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.949882:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.949883:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.949885:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c400/0xaf35132e63034d20 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 21 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.949888:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c400/0xaf35132e63034d20 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 21 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.949896:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156000/0xaf35132e63034e07 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 20 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.949899:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156000/0xaf35132e63034e07 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 20 type: IBT flags: 0x50210400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.949902:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156000/0xaf35132e63034e07 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 20 type: IBT flags: 0x40210400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.949906:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf243156000 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.949908:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156000/0xaf35132e63034e07 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 20 type: IBT flags: 0x40210400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.949912:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156000/0xaf35132e63034e07 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 20 type: IBT flags: 0x40218400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.949917:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.949918:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.949919:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156000/0xaf35132e63034e07 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 20 type: IBT flags: 0x44a19400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.949923:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243156000/0xaf35132e63034e07 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 20 type: IBT flags: 0x44a19400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.949930:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3800/0xaf35132e63035316 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 19 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.949934:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3800/0xaf35132e63035316 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 19 type: IBT flags: 0x50210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.949937:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3800/0xaf35132e63035316 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 19 type: IBT flags: 0x40210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.949940:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf244eb3800 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.949945:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3800/0xaf35132e63035316 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 19 type: IBT flags: 0x40210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.949973:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf243279e00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.949973:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3800/0xaf35132e63035316 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 18 type: IBT flags: 0x40218400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.949980:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279e00/0xaf35132e63034a2c lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 19 type: IBT flags: 0x40210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.950008:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279e00/0xaf35132e63034a2c lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 19 type: IBT flags: 0x40218400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.950013:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.950018:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.950021:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3800/0xaf35132e63035316 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 18 type: IBT flags: 0x44a19400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.950027:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.950028:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3800/0xaf35132e63035316 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 18 type: IBT flags: 0x44a19400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.950036:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298644.950042:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279e00/0xaf35132e63034a2c lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 19 type: IBT flags: 0x44a19400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.950049:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279800/0xaf35132e63035593 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 18 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.950051:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279e00/0xaf35132e63034a2c lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 18 type: IBT flags: 0x44a19400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.950058:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279800/0xaf35132e63035593 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 18 type: IBT flags: 0x50210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.950079:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279800/0xaf35132e63035593 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 18 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.950085:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf243279800 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.950088:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279800/0xaf35132e63035593 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 16 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.950092:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279800/0xaf35132e63035593 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 16 type: IBT flags: 0x40218400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.950095:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157400/0xaf35132e63034a5d lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 17 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.950103:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157400/0xaf35132e63034a5d lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 16 type: IBT flags: 0x50210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.950110:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157400/0xaf35132e63034a5d lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 16 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.950113:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.950120:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.950141:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279800/0xaf35132e63035593 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 17 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.950150:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf243157400 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.950150:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279800/0xaf35132e63035593 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 16 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.950159:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157400/0xaf35132e63034a5d lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 17 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.950174:0:3609:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3000/0xaf35132e630356b2 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 16 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.950179:0:3609:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3000/0xaf35132e630356b2 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 16 type: IBT flags: 0x50210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.950183:0:3609:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3000/0xaf35132e630356b2 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 16 type: IBT flags: 0x40210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.950188:0:3609:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf244eb3000 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:1.0:1780298644.950192:0:3609:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3000/0xaf35132e630356b2 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 16 type: IBT flags: 0x40210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.950198:0:3609:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3000/0xaf35132e630356b2 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 16 type: IBT flags: 0x40218400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.950207:0:3609:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.950210:0:3609:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1780298644.950213:0:3609:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3000/0xaf35132e630356b2 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 16 type: IBT flags: 0x44a19400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.950219:0:3609:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3000/0xaf35132e630356b2 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 16 type: IBT flags: 0x44a19400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.950239:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157400/0xaf35132e63034a5d lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 14 type: IBT flags: 0x40218400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.950251:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298644.950254:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298644.950258:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157400/0xaf35132e63034a5d lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 14 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.950264:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243157400/0xaf35132e63034a5d lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 14 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.950293:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3582cb400/0xaf35132e63034a79 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.950300:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3582cb400/0xaf35132e63034a79 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 13 type: IBT flags: 0x50210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.950307:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3582cb400/0xaf35132e63034a79 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 13 type: IBT flags: 0x40210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.950313:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf3582cb400 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298644.950317:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3582cb400/0xaf35132e63034a79 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 13 type: IBT flags: 0x40210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.950327:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3582cb400/0xaf35132e63034a79 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 13 type: IBT flags: 0x40218400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.950336:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298644.950338:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298644.950341:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3582cb400/0xaf35132e63034a79 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 13 type: IBT flags: 0x44a19400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.950346:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf3582cb400/0xaf35132e63034a79 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 13 type: IBT flags: 0x44a19400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.950350:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605059584:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298644.950359:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830ca00/0xaf35132e63034d5f lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 12 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.950366:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830ca00/0xaf35132e63034d5f lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 12 type: IBT flags: 0x50210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.950372:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605059712:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298644.950373:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830ca00/0xaf35132e63034d5f lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 12 type: IBT flags: 0x40210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.950383:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf34830ca00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298644.950385:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830ca00/0xaf35132e63034d5f lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 12 type: IBT flags: 0x40210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.950391:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830ca00/0xaf35132e63034d5f lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 12 type: IBT flags: 0x40218400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.950399:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298644.950403:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1780298644.950404:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000024bbcf41/0xaf35132e630357d8 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4704 expref: 16 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.950406:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830ca00/0xaf35132e63034d5f lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 12 type: IBT flags: 0x44a19400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.950428:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830ca00/0xaf35132e63034d5f lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 12 type: IBT flags: 0x44a19400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.950430:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2c00, rc 0) 00010000:00010000:3.0:1780298644.950443:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c800/0xaf35132e63034de4 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 11 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.950450:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c800/0xaf35132e63034de4 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 11 type: IBT flags: 0x50210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.950456:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c800/0xaf35132e63034de4 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 11 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.950461:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf34830c800 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298644.950464:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c800/0xaf35132e63034de4 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 11 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.950469:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243845180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:29360:x1866777609645312:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1433us (1614us total) trans 0 rc 0/0 00010000:00010000:3.0:1780298644.950477:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c800/0xaf35132e63034de4 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 11 type: IBT flags: 0x40218400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.950483:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10133 00010000:00010000:3.0:1780298644.950493:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298644.950495:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298644.950498:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c800/0xaf35132e63034de4 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 11 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.950681:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf34830c800/0xaf35132e63034de4 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 11 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.950700:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3400/0xaf35132e63034e9a lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 10 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.950700:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:3.0:1780298644.950710:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3400/0xaf35132e63034e9a lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 10 type: IBT flags: 0x50210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.950717:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3400/0xaf35132e63034e9a lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 10 type: IBT flags: 0x40210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.950723:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf244eb3400 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298644.950727:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3400/0xaf35132e63034e9a lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 10 type: IBT flags: 0x40210400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.950734:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3400/0xaf35132e63034e9a lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 10 type: IBT flags: 0x40218400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.950744:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298644.950747:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1780298644.950747:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609645568 00010000:00010000:3.0:1780298644.950750:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3400/0xaf35132e63034e9a lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 10 type: IBT flags: 0x44a19400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.950753:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609645568 00010000:00010000:3.0:1780298644.950760:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3400/0xaf35132e63034e9a lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 10 type: IBT flags: 0x44a19400000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.950775:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:2.0:1780298644.950782:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e63035722 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [69632->2764799] (req 69632->73727) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4655 expref: 16 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.950816:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15243 00010000:00010000:3.0:1780298644.950821:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278a00/0xaf35132e63034fff lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.950829:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609645568:12345-192.168.203.25@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1780298644.950831:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609645440 00010000:00010000:3.0:1780298644.950835:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278a00/0xaf35132e63034fff lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x50210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.950838:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609645440 00010000:00010000:3.0:1780298644.950846:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278a00/0xaf35132e63034fff lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.950848:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298644.950855:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf243278a00 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298644.950858:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278a00/0xaf35132e63034fff lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x40210400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.950869:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278a00/0xaf35132e63034fff lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x40218400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.950870:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e63035722 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [69632->2764799] (req 69632->73727) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4655 expref: 17 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00100000:0.0:1780298644.950875:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (2764800->2777087) req@ffff9cf2433df480 x1866777609645440/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 12776/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00010000:00010000:3.0:1780298644.950894:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298644.950901:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1780298644.950902:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e63035722 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [69632->2764799] (req 69632->73727) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4655 expref: 17 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00100000:0.0:1780298644.950903:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2433df480 with x1866777609645440 ext(2764800->2777087) 00010000:00010000:3.0:1780298644.950906:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278a00/0xaf35132e63034fff lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.950914:0:19950:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 0000000075e99a4b/0xaf35132e630356ff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [2764800->36319231] (req 2764800->2777087) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4632 expref: 17 pid: 8359 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.950927:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278a00/0xaf35132e63034fff lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x44a19400000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.950940:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e63035722 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [69632->2764799] (req 69632->73727) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4655 expref: 17 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.950944:0:19950:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 0000000075e99a4b/0xaf35132e630356ff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [2764800->36319231] (req 2764800->2777087) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4632 expref: 17 pid: 8359 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.950949:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2800/0xaf35132e630352f3 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.950967:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1780298644.950971:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2800/0xaf35132e630352f3 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1780298644.950971:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9cf2433df480 00010000:00010000:3.0:1780298644.950982:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2800/0xaf35132e630352f3 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.950988:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf244eb2800 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298644.950991:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2800/0xaf35132e630352f3 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x40210400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.950998:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2800/0xaf35132e630352f3 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x40218400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.951006:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609645568:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 179us (308us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.951007:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 212 00010000:00010000:3.0:1780298644.951009:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298644.951017:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298644.951021:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2800/0xaf35132e630352f3 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.951023:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2399:x1866777609645440:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:2.0:1780298644.951040:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15243 00010000:00010000:3.0:1780298644.951043:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2800/0xaf35132e630352f3 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x44a19400000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.951063:0:3610:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278000/0xaf35132e630355b6 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.951069:0:3610:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278000/0xaf35132e630355b6 lrc: 2/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.951075:0:3610:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278000/0xaf35132e630355b6 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.951081:0:3610:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf243278000 already unused, calling callback (ffffffffc169a4f0) 00010000:00010000:3.0:1780298644.951086:0:3610:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278000/0xaf35132e630355b6 lrc: 3/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.951093:0:3610:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278000/0xaf35132e630355b6 lrc: 4/0,0 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.951101:0:3610:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298644.951103:0:3610:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1780298644.951106:0:3610:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278000/0xaf35132e630355b6 lrc: 2/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.951110:0:3610:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278000/0xaf35132e630355b6 lrc: 0/0,0 mode: --/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.951595:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2399:x1866777609645440:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 576us (827us total) trans 4294969296 rc 0/0 00000100:00100000:0.0:1780298644.951603:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 212 00002000:00100000:0.0:1780298644.951611:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (2764800->2777087) req@ffff9cf2433df480 x1866777609645440/t4294969296(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 12776/448 e 0 to 0 dl 1780298655 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298644.951619:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2433df480 with x1866777609645440 ext(2764800->2777087) 00010000:00010000:0.0:1780298644.951625:0:19950:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 0000000075e99a4b/0xaf35132e630356ff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2764800->36319231] (req 2764800->2777087) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4632 expref: 17 pid: 8359 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.951638:0:19950:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 0000000075e99a4b/0xaf35132e630356ff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2764800->36319231] (req 2764800->2777087) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4632 expref: 17 pid: 8359 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1780298644.951646:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9cf2433df480 00000100:00100000:1.0:1780298644.952487:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.952538:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609645696 00000100:00100000:0.0:1780298644.952545:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609645696 00010000:00010000:0.0:1780298644.952566:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000075e99a4b/0xaf35132e630356ff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2764800->36319231] (req 2764800->2777087) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4632 expref: 17 pid: 8359 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.952598:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15244 00000100:00100000:0.0:1780298644.952606:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348af6050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2398:x1866777609645696:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298644.952617:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:0.0:1780298644.952625:0:6463:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 104 -> 128 00010000:00010000:0.0:1780298644.952639:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000075e99a4b/0xaf35132e630356ff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2764800->36319231] (req 2764800->2777087) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4632 expref: 17 pid: 8359 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.952654:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000075e99a4b/0xaf35132e630356ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2764800->36319231] (req 2764800->2777087) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4632 expref: 17 pid: 8359 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.952670:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000075e99a4b/0xaf35132e630356ff lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2764800->36319231] (req 2764800->2777087) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4632 expref: 17 pid: 8359 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.952683:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(36 122) 00010000:00010000:0.0:1780298644.952687:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(36 122) 00010000:00010000:0.0:1780298644.952694:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000024bbcf41/0xaf35132e630357d8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4704 expref: 15 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.952704:0:6463:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000024bbcf41/0xaf35132e630357d8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4704 expref: 15 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.952733:0:6463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000024bbcf41/0xaf35132e630357d8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4704 expref: 15 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.952744:0:6463:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd4e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777605059840:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298644.952799:0:6463:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12240 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.953673:0:6463:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd4e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777605059840:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298644.953688:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298644.953722:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348af6050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2398:x1866777609645696:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1118us (1238us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.953730:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15244 00000100:00100000:3.0:1780298644.954044:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.954083:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609645824 00000100:00100000:3.0:1780298644.954084:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.954091:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609645824 00002000:00010000:1.0:1780298644.954118:0:8362:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (2707888->18446744073709551615) 00002000:00100000:1.0:1780298644.954140:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2556ab100 with x1866777609645824 ext(2707888->18446744073709551615) 00000100:00100000:0.0:1780298644.954142:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609645952 00000100:00100000:0.0:1780298644.954147:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609645952 00002000:00010000:1.0:1780298644.954151:0:8362:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2556ab100. 00002000:00010000:0.0:1780298644.954161:0:19951:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00002000:00100000:0.0:1780298644.954165:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2556a8a80 with x1866777609645952 ext(0->18446744073709551615) 00002000:00010000:0.0:1780298644.954171:0:19951:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2556a8a80. 00000100:00100000:1.0:1780298644.954183:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6383 00000100:00100000:1.0:1780298644.954194:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556ab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2397:x1866777609645824:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:0.0:1780298644.954198:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6384 00000100:00100000:0.0:1780298644.954207:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2398:x1866777609645952:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:1.0:1780298644.954648:0:8362:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 5375863 -> 2707888 00002000:00010000:1.0:1780298644.954653:0:8362:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298644.954656:0:8362:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298644.954658:0:8362:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298644.954658:0:19951:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4194304 -> 0 00002000:00010000:1.0:1780298644.954663:0:8362:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 504 -> 408 00002000:00010000:0.0:1780298644.954665:0:19951:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298644.954669:0:19951:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298644.954671:0:19951:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298644.954673:0:19951:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 128 -> 0 00000100:00100000:1.0:1780298644.954701:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556ab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609645824:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 510us (657us total) trans 4294969751 rc 0/0 00000100:00100000:1.0:1780298644.954709:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6383 00002000:00010000:1.0:1780298644.954714:0:8362:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (2707888->18446744073709551615) 00000100:00100000:0.0:1780298644.954714:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2398:x1866777609645952:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 504us (628us total) trans 4294969297 rc 0/0 00002000:00100000:1.0:1780298644.954721:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2556ab100 with x1866777609645824 ext(2707888->18446744073709551615) 00002000:00010000:1.0:1780298644.954728:0:8362:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2556ab100. 00000100:00100000:0.0:1780298644.954730:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6384 00002000:00010000:0.0:1780298644.954735:0:19951:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00002000:00100000:0.0:1780298644.954739:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2556a8a80 with x1866777609645952 ext(0->18446744073709551615) 00002000:00010000:0.0:1780298644.954744:0:19951:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2556a8a80. 00000100:00100000:1.0:1780298644.955689:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.955737:0:6476:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609646080 00000100:00100000:2.0:1780298644.955742:0:6476:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609646080 00000100:00100000:2.0:1780298644.955772:0:6476:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62051 00000100:00100000:2.0:1780298644.955777:0:6476:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+25:29360:x1866777609646080:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.955787:0:6476:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.955802:0:6476:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf350de3200/0xaf35132e630357df lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d470b expref: -99 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.955896:0:6476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf350de2a00/0xaf35132e630357e6 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT flags: 0x40000000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.955910:0:6476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf350de2a00/0xaf35132e630357e6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.955918:0:6476:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf350de2a00/0xaf35132e630357e6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298644.955938:0:6476:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 2707888 00000004:00200000:2.0:1780298644.955941:0:6476:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298644.955954:0:6476:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf350de2a00/0xaf35132e630357e6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.955972:0:6476:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf350de3200/0xaf35132e630357df lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d470b expref: 27 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.955983:0:6476:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf350de2a00/0xaf35132e630357e6 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d470b expref: 26 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.955992:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298644.955993:0:6476:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298644.955996:0:6476:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2a00, rc 0) 00000100:00100000:2.0:1780298644.956023:0:6476:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433dc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+26:29360:x1866777609646080:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 245us (336us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.956029:0:6476:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62051 00000100:00100000:2.0:1780298644.957081:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.957118:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609646208 00000100:00100000:2.0:1780298644.957149:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.957156:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609646208 00000100:00100000:0.0:1780298644.957177:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609646336 00000100:00100000:0.0:1780298644.957183:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609646336 00000100:00100000:0.0:1780298644.957213:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10134 00000100:00100000:0.0:1780298644.957220:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2399:x1866777609646336:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.957228:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.957239:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b35ce980/0xaf35132e630357ed lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4719 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.957242:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10135 00000100:00100000:3.0:1780298644.957257:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e4f2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2399:x1866777609646208:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298644.957267:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298644.957290:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e630357f4 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4712 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.957301:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605059968:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298644.957330:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2556ab800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605060096:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.957364:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.957366:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.958067:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2556ab800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605060096:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.958095:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e630357f4 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4712 expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.958097:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605059968:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298644.958115:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b35ce980/0xaf35132e630357ed lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4719 expref: 17 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.958120:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006d4295ea/0xaf35132e630357f4 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4712 expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.958149:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b35ce980/0xaf35132e630357ed lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4719 expref: 17 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.958159:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bafa00, rc 0) 00000100:00100000:0.0:1780298644.958187:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2399:x1866777609646336:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 968us (1039us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298644.958193:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10134 00010000:00010000:1.0:1780298644.958268:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157400, rc 0) 00000100:00100000:1.0:1780298644.958313:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e4f2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2399:x1866777609646208:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1060us (1233us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.958320:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10135 00000100:00100000:3.0:1780298644.959336:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.959472:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.959502:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609646464 00000100:00100000:0.0:1780298644.959508:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609646464 00000100:00100000:0.0:1780298644.959536:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609646592 00000100:00100000:0.0:1780298644.959539:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609646592 00000100:00100000:0.0:1780298644.959569:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10136 00000100:00100000:0.0:1780298644.959577:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609646464:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298644.959580:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10137 00010000:00010000:0.0:1780298644.959592:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298644.959596:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556aa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2399:x1866777609646592:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.959605:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c2d588e1/0xaf35132e630357fb lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4720 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.959606:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.959618:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017dab243/0xaf35132e63035802 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4727 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.959645:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605060224:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.959655:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433de680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605060352:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.959692:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12000 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.959693:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.960510:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605060224:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.960530:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433de680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605060352:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.960535:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c2d588e1/0xaf35132e630357fb lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4720 expref: 14 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.960555:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000017dab243/0xaf35132e63035802 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4727 expref: 17 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.960556:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c2d588e1/0xaf35132e630357fb lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4720 expref: 14 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.960584:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf800, rc 0) 00010000:00010000:2.0:1780298644.960588:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017dab243/0xaf35132e63035802 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4727 expref: 17 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.960599:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279800, rc 0) 00000100:00100000:1.0:1780298644.960622:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609646464:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1045us (1286us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.960628:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556aa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2399:x1866777609646592:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1035us (1154us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.960635:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10136 00000100:00100000:2.0:1780298644.960647:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10137 00000100:00100000:3.0:1780298644.961658:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.961703:0:17578:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609646720 00000100:00100000:1.0:1780298644.961709:0:17578:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609646720 00000100:00100000:1.0:1780298644.961743:0:17578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62052 00000100:00100000:1.0:1780298644.961749:0:17578:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:29360:x1866777609646720:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.961763:0:17578:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.961774:0:17578:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278200/0xaf35132e63035809 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d472e expref: -99 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.961851:0:17578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e63035810 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 9 type: IBT flags: 0x40000000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.961866:0:17578:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e63035810 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.961877:0:17578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e63035810 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1780298644.961897:0:17578:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 2707888 00000004:00200000:1.0:1780298644.961900:0:17578:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1780298644.961913:0:17578:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e63035810 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.961929:0:17578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278200/0xaf35132e63035809 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d472e expref: 25 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.961939:0:17578:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e63035810 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d472e expref: 24 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.961945:0:17578:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298644.961947:0:17578:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1780298644.961949:0:17578:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278800, rc 0) 00000100:00100000:1.0:1780298644.961973:0:17578:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+24:29360:x1866777609646720:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 225us (319us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.961977:0:17578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62052 00000100:00100000:3.0:1780298644.963174:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.963206:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609646848 00000100:00100000:1.0:1780298644.963211:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609646848 00000100:00100000:1.0:1780298644.963257:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6385 00000100:00100000:1.0:1780298644.963265:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556aad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2399:x1866777609646848:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298644.963451:0:8363:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 2707888 -> 2707888 00002000:00010000:1.0:1780298644.963455:0:8363:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298644.963457:0:8363:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298644.963459:0:8363:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298644.963462:0:8363:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 408 -> 424 00000100:00100000:1.0:1780298644.963493:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556aad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2399:x1866777609646848:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 229us (321us total) trans 4294969752 rc 0/0 00000100:00100000:1.0:1780298644.963501:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6385 00000100:00100000:0.0:1780298644.964701:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.964736:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609646976 00000100:00100000:2.0:1780298644.964740:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609646976 00000100:00100000:2.0:1780298644.964771:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6386 00000100:00100000:2.0:1780298644.964776:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2397:x1866777609646976:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298644.964877:0:8364:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 2707888 -> 2707888 00002000:00010000:2.0:1780298644.964880:0:8364:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298644.964881:0:8364:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298644.964883:0:8364:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298644.964884:0:8364:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 424 -> 432 00000100:00100000:2.0:1780298644.964913:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2397:x1866777609646976:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 134us (216us total) trans 4294969753 rc 0/0 00000100:00100000:2.0:1780298644.964917:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6386 00000100:00100000:3.0:1780298644.966905:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.966961:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609647104 00000100:00100000:0.0:1780298644.966969:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609647104 00000100:00100000:0.0:1780298644.967011:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6387 00000100:00100000:0.0:1780298644.967021:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556aa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2398:x1866777609647104:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298644.967197:0:19950:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 0 -> 1389382 00002000:00010000:0.0:1780298644.967201:0:19950:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298644.967204:0:19950:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298644.967206:0:19950:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298644.967208:0:19950:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 0 -> 16 00000100:00100000:0.0:1780298644.967253:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556aa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2398:x1866777609647104:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 234us (353us total) trans 4294969298 rc 0/0 00000100:00100000:0.0:1780298644.967261:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6387 00000100:00100000:3.0:1780298644.969481:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.969538:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609647360 00000100:00100000:2.0:1780298644.969547:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609647360 00000100:00100000:2.0:1780298644.969600:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62053 00000100:00100000:2.0:1780298644.969611:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+26:29360:x1866777609647360:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:2.0:1780298644.969661:0:6477:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298644.969690:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf350de2a00/0xaf35132e630357e6 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d470b expref: 26 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.969711:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298644.969716:0:6477:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298644.969871:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf350de3000/0xaf35132e63035817 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.969887:0:6477:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e63035810 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d472e expref: 23 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.969896:0:6477:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf350de3000/0xaf35132e63035817 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.969930:0:6477:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e63035810 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d472e expref: 23 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.969950:0:6477:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e63035810 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d472e expref: 23 pid: 17578 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.969965:0:6477:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372f480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6477:1866777605060480:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298644.970050:0:6477:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.970824:0:6477:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372f480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6477:1866777605060480:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298644.970853:0:6477:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf350de3000/0xaf35132e63035817 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.970965:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.971010:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609647488 00000100:00100000:0.0:1780298644.971015:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609647488 00010000:00010000:0.0:1780298644.971037:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e63035810 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d472e expref: 24 pid: 17578 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.971066:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15245 00000100:00100000:0.0:1780298644.971073:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556ab480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+24:2400:x1866777609647488:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298644.971083:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298644.971088:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e63035810 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d472e expref: 24 pid: 17578 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.971095:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e63035810 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d472e expref: 24 pid: 17578 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.971105:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e63035810 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d472e expref: 24 pid: 17578 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.971113:0:20253:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298644.971116:0:20253:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf350de3000/0xaf35132e63035817 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.971122:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf350de3000/0xaf35132e63035817 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x40290000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.971155:0:20253:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf350de3000/0xaf35132e63035817 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x40290000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.971171:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298644.971187:0:6477:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf350de3000/0xaf35132e63035817 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.971196:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf350de3000/0xaf35132e63035817 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298644.971202:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556ab480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2400:x1866777609647488:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 130us (237us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.971209:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15245 00000001:00080000:2.0:1780298644.971254:0:6477:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991102 00000004:00010000:2.0:1780298644.971292:0:6477:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 0000000032e1cffa reply state 00000000573f75f6 transno 4294991103 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf350de3000/0xaf35132e63035817 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.971307:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf350de3000/0xaf35132e63035817 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.971314:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298644.971317:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1780298644.971320:0:6477:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1780298644.971382:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+28:29360:x1866777609647360:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1773us (1907us total) trans 4294991103 rc 0/0 00000100:00100000:2.0:1780298644.971389:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62053 00000020:00080000:2.0:1780298644.971393:0:6477:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1780298644.972096:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.972160:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609647616 00000100:00100000:2.0:1780298644.972165:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609647616 00000100:00100000:2.0:1780298644.972201:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10138 00000100:00100000:2.0:1780298644.972209:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:29360:x1866777609647616:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.972221:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.972234:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e6303581e lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4735 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.972258:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000aa41763f/0xaf35132e630357d1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d46fd expref: 17 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.972266:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(35 169) 00010000:00010000:2.0:1780298644.972271:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(36 169) 00010000:00010000:2.0:1780298644.972291:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e6303581e lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4735 expref: 14 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.972316:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000aa41763f/0xaf35132e630357d1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d46fd expref: 17 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.972336:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000aa41763f/0xaf35132e630357d1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d46fd expref: 17 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.972349:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605060608:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298644.972399:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.976281:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.976330:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605060608:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298644.976331:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609647744 00000100:00100000:3.0:1780298644.976345:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609647744 00010000:00010000:1.0:1780298644.976382:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e6303581e lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4735 expref: 14 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.976385:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000aa41763f/0xaf35132e630357d1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d46fd expref: 18 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.976431:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2400, rc 0) 00000100:00100000:3.0:1780298644.976453:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15246 00000100:00100000:3.0:1780298644.976465:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2399:x1866777609647744:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298644.976480:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298644.976501:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000aa41763f/0xaf35132e630357d1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d46fd expref: 18 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.976516:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:29360:x1866777609647616:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 4303us (4417us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.976538:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10138 00010000:00010000:3.0:1780298644.976544:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000aa41763f/0xaf35132e630357d1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d46fd expref: 18 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.976574:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000aa41763f/0xaf35132e630357d1 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d46fd expref: 18 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.976592:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(36 169) 00010000:00010000:3.0:1780298644.976597:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(36 169) 00010000:00010000:3.0:1780298644.976605:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e6303581e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4735 expref: 13 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.976617:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e6303581e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4735 expref: 13 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298644.976650:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e6303581e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4735 expref: 13 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298644.976707:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2556abb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777605060736:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298644.976774:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298644.977655:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2556abb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777605060736:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298644.977672:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298644.977701:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2399:x1866777609647744:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1240us (1443us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298644.977713:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15246 00000100:00100000:0.0:1780298644.977971:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.978034:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609647872 00000100:00100000:1.0:1780298644.978043:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609647872 00000100:00100000:1.0:1780298644.978117:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10139 00000100:00100000:1.0:1780298644.978164:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:29360:x1866777609647872:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.978182:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.978200:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b25cdf87/0xaf35132e63035825 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d473c expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.978228:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000024bbcf41/0xaf35132e630357d8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4704 expref: 17 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.978237:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(36 122) 00010000:00010000:1.0:1780298644.978242:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(37 122) 00010000:00010000:1.0:1780298644.978247:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b25cdf87/0xaf35132e63035825 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d473c expref: 17 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.978288:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000024bbcf41/0xaf35132e630357d8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4704 expref: 17 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.978304:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000024bbcf41/0xaf35132e630357d8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4704 expref: 17 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.978317:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433dce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605060864:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298644.978387:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a540 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.979284:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433dce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605060864:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298644.979307:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b25cdf87/0xaf35132e63035825 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d473c expref: 17 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.979322:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278200, rc 0) 00000100:00100000:1.0:1780298644.979358:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:29360:x1866777609647872:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1231us (1387us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298644.979368:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10139 00000100:00100000:3.0:1780298644.979457:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.979520:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609648000 00000100:00100000:2.0:1780298644.979527:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609648000 00010000:00010000:2.0:1780298644.979553:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000024bbcf41/0xaf35132e630357d8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4704 expref: 18 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.979591:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15247 00000100:00100000:2.0:1780298644.979602:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556abb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2399:x1866777609648000:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298644.979614:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298644.979632:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000024bbcf41/0xaf35132e630357d8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4704 expref: 18 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.979644:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000024bbcf41/0xaf35132e630357d8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4704 expref: 18 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.979663:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000024bbcf41/0xaf35132e630357d8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4704 expref: 18 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.979678:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(37 122) 00010000:00010000:2.0:1780298644.979682:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(37 122) 00010000:00010000:2.0:1780298644.979689:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b25cdf87/0xaf35132e63035825 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d473c expref: 16 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.979698:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b25cdf87/0xaf35132e63035825 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d473c expref: 16 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.979723:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000b25cdf87/0xaf35132e63035825 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d473c expref: 16 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.979740:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372f100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777605060992:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298644.979793:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb10c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.980820:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.980839:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372f100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777605060992:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298644.980858:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298644.980869:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.980889:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556abb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2399:x1866777609648000:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1290us (1433us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.980898:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15247 00000100:00100000:3.0:1780298644.980917:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609648128 00000100:00100000:3.0:1780298644.980926:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609648128 00002000:00010000:3.0:1780298644.980963:0:19949:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00000100:00100000:1.0:1780298644.980968:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609648256 00002000:00100000:3.0:1780298644.980970:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf347cd4380 with x1866777609648128 ext(4194304->18446744073709551615) 00000100:00100000:1.0:1780298644.980974:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609648256 00002000:00010000:1.0:1780298644.980993:0:8362:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4118169->18446744073709551615) 00002000:00100000:1.0:1780298644.980996:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf347cd5c00 with x1866777609648256 ext(4118169->18446744073709551615) 00002000:00010000:1.0:1780298644.981011:0:8362:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf347cd5c00. 00002000:00010000:3.0:1780298644.981017:0:19949:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf347cd4380. 00000100:00100000:1.0:1780298644.981039:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6388 00000100:00100000:1.0:1780298644.981048:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2399:x1866777609648256:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:0.0:1780298644.981066:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6389 00000100:00100000:0.0:1780298644.981082:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609648128:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:0.0:1780298644.981318:0:19951:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 2707888 -> 4194304 00002000:00010000:1.0:1780298644.981319:0:8362:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 1389382 -> 4118169 00002000:00010000:0.0:1780298644.981323:0:19951:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298644.981326:0:8362:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298644.981327:0:19951:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298644.981332:0:8362:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298644.981332:0:19951:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298644.981338:0:8362:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298644.981338:0:19951:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 432 -> 416 00002000:00010000:1.0:1780298644.981344:0:8362:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 16 -> 16 00000100:00100000:0.0:1780298644.981385:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2400:x1866777609648128:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 308us (566us total) trans 4294969754 rc 0/0 00000100:00100000:1.0:1780298644.981386:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2399:x1866777609648256:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 337us (517us total) trans 4294969299 rc 0/0 00000100:00100000:1.0:1780298644.981402:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6388 00002000:00010000:1.0:1780298644.981406:0:8362:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4118169->18446744073709551615) 00002000:00100000:1.0:1780298644.981412:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf347cd5c00 with x1866777609648256 ext(4118169->18446744073709551615) 00000100:00100000:0.0:1780298644.981412:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6389 00002000:00010000:0.0:1780298644.981418:0:19951:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00010000:1.0:1780298644.981420:0:8362:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf347cd5c00. 00002000:00100000:0.0:1780298644.981424:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf347cd4380 with x1866777609648128 ext(4194304->18446744073709551615) 00002000:00010000:0.0:1780298644.981431:0:19951:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf347cd4380. 00000100:00100000:0.0:1780298644.983003:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.983051:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609648384 00000100:00100000:1.0:1780298644.983057:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609648384 00002000:00100000:1.0:1780298644.983085:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x12:0x0] (4108288->4112383) req@ffff9cf3481fdc00 x1866777609648384/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.983095:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fdc00 with x1866777609648384 ext(4108288->4112383) 00002000:00010000:1.0:1780298644.983100:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf3481fdc00 00000100:00100000:1.0:1780298644.983149:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6390 00000100:00100000:1.0:1780298644.983161:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2398:x1866777609648384:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298644.983351:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2398:x1866777609648384:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 193us (351us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298644.983358:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6390 00002000:00100000:1.0:1780298644.983365:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x12:0x0] (4108288->4112383) req@ffff9cf3481fdc00 x1866777609648384/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/4536 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298644.983373:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fdc00 with x1866777609648384 ext(4108288->4112383) 00002000:00010000:1.0:1780298644.983376:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf3481fdc00 00000100:00100000:3.0:1780298644.984700:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.984740:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609648512 00000100:00100000:2.0:1780298644.984745:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609648512 00002000:00100000:2.0:1780298644.984770:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (4112384->4116479) req@ffff9cf2556a9880 x1866777609648512/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.984781:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2556a9880 with x1866777609648512 ext(4112384->4116479) 00002000:00010000:2.0:1780298644.984784:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2556a9880 00000100:00100000:2.0:1780298644.984808:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6391 00000100:00100000:2.0:1780298644.984817:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2398:x1866777609648512:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298644.984915:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2398:x1866777609648512:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 99us (214us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298644.984922:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6391 00002000:00100000:2.0:1780298644.984928:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (4112384->4116479) req@ffff9cf2556a9880 x1866777609648512/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 488/4536 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298644.984937:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2556a9880 with x1866777609648512 ext(4112384->4116479) 00002000:00010000:2.0:1780298644.984941:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2556a9880 00000100:00100000:2.0:1780298644.986613:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.986663:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609648640 00000100:00100000:0.0:1780298644.986669:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609648640 00002000:00100000:0.0:1780298644.986695:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (4087808->4120575) req@ffff9cf34372f100 x1866777609648640/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 504/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298644.986705:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf34372f100 with x1866777609648640 ext(4087808->4120575) 00002000:00010000:0.0:1780298644.986709:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf34372f100 00000100:00100000:0.0:1780298644.986729:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6392 00000100:00100000:0.0:1780298644.986739:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2398:x1866777609648640:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298644.987797:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2398:x1866777609648640:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 1057us (1182us total) trans 0 rc 24576/24576 00000100:00100000:0.0:1780298644.987805:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6392 00002000:00100000:0.0:1780298644.987811:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (4087808->4120575) req@ffff9cf34372f100 x1866777609648640/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 504/440 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 24576/24576 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298644.987818:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf34372f100 with x1866777609648640 ext(4087808->4120575) 00002000:00010000:0.0:1780298644.987821:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf34372f100 00000100:00100000:0.0:1780298644.989238:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.989290:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609648768 00000100:00100000:3.0:1780298644.989297:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609648768 00002000:00100000:3.0:1780298644.989328:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x12:0x0] (4108288->4118168) req@ffff9cf3481fed80 x1866777609648768/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 10376/0 e 0 to 0 dl 1780298655 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298644.989339:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fed80 with x1866777609648768 ext(4108288->4118168) 00002000:00010000:3.0:1780298644.989342:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf3481fed80 00000100:00100000:3.0:1780298644.989362:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6393 00000100:00100000:3.0:1780298644.989370:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2399:x1866777609648768:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:3.0:1780298644.989923:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2399:x1866777609648768:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 554us (688us total) trans 4294969300 rc 0/0 00000100:00100000:3.0:1780298644.989932:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6393 00002000:00100000:3.0:1780298644.989939:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x12:0x0] (4108288->4118168) req@ffff9cf3481fed80 x1866777609648768/t4294969300(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:350/0 lens 10376/448 e 0 to 0 dl 1780298655 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298644.989949:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fed80 with x1866777609648768 ext(4108288->4118168) 00002000:00010000:3.0:1780298644.989953:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf3481fed80 00000100:00100000:3.0:1780298644.990786:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.990831:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609648896 00000100:00100000:2.0:1780298644.990837:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609648896 00000100:00100000:2.0:1780298644.990879:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10140 00000100:00100000:2.0:1780298644.990887:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556ab800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2399:x1866777609648896:12345-192.168.203.25@tcp:16:fsx.0 00000001:00080000:1.0:1780298644.991787:0:8351:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0000: transno 4294969300 is committed 00000001:00080000:1.0:1780298644.991804:0:8351:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0000: transno 4294969298 is committed 00000100:00100000:2.0:1780298644.991938:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556ab800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609648896:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 1047us (1148us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298644.991947:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10140 00000100:00100000:2.0:1780298644.993099:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.993152:0:7411:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609649024 00000100:00100000:0.0:1780298644.993163:0:7411:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609649024 00000100:00100000:0.0:1780298644.993206:0:7411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62054 00000100:00100000:0.0:1780298644.993216:0:7411:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:29360:x1866777609649024:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298644.993233:0:7411:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.993256:0:7411:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baec00/0xaf35132e6303582c lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4743 expref: -99 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.993359:0:7411:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae600/0xaf35132e63035833 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 9 type: IBT flags: 0x40000000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.993372:0:7411:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae600/0xaf35132e63035833 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.993381:0:7411:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae600/0xaf35132e63035833 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1780298644.993403:0:7411:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 8312473 00000004:00200000:0.0:1780298644.993406:0:7411:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1780298644.993422:0:7411:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae600/0xaf35132e63035833 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.993442:0:7411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baec00/0xaf35132e6303582c lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4743 expref: 25 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.993457:0:7411:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae600/0xaf35132e63035833 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4743 expref: 24 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.993463:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298644.993465:0:7411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1780298644.993468:0:7411:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae600, rc 0) 00000100:00100000:0.0:1780298644.993494:0:7411:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+24:29360:x1866777609649024:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 282us (399us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298644.993498:0:7411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62054 00000100:00100000:0.0:1780298644.994508:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298644.994542:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298644.994570:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609649152 00000100:00100000:2.0:1780298644.994576:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609649152 00000100:00100000:1.0:1780298644.994613:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609649280 00000100:00100000:1.0:1780298644.994619:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609649280 00000100:00100000:2.0:1780298644.994626:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10141 00000100:00100000:2.0:1780298644.994637:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2398:x1866777609649152:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.994644:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298644.994652:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10142 00010000:00010000:2.0:1780298644.994654:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000024bbcf41/0xaf35132e6303583a lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d474a expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.994663:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609649280:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.994674:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.994684:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004fdbd682/0xaf35132e63035841 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4751 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.994715:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605061120:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.994720:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433dce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605061248:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.994788:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb16c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.994789:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a540 going to sleep for 11 seconds 00002000:00010000:1.0:1780298644.995561:0:8358:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks: 16 -> 40 00000100:00100000:2.0:1780298644.995574:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605061120:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.995574:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433dce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605061248:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.995606:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000024bbcf41/0xaf35132e6303583a lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d474a expref: 18 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.995606:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004fdbd682/0xaf35132e63035841 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4751 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.995643:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000024bbcf41/0xaf35132e6303583a lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d474a expref: 18 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.995643:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004fdbd682/0xaf35132e63035841 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4751 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.995658:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278e00, rc 0) 00010000:00010000:2.0:1780298644.995673:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2c00, rc 0) 00000100:00100000:1.0:1780298644.995686:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609649280:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1027us (1144us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.995697:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10142 00000100:00100000:2.0:1780298644.995700:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2398:x1866777609649152:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1065us (1192us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.995710:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10141 00000100:00100000:3.0:1780298644.996719:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298644.996742:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.996761:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609649408 00000100:00100000:1.0:1780298644.996767:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609649408 00000100:00100000:0.0:1780298644.996778:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609649536 00000100:00100000:0.0:1780298644.996783:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609649536 00000100:00100000:1.0:1780298644.996806:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10143 00000100:00100000:1.0:1780298644.996815:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2398:x1866777609649408:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298644.996820:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10144 00010000:00010000:1.0:1780298644.996824:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298644.996831:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255090a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609649536:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.996834:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f4922155/0xaf35132e63035848 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4758 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298644.996841:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298644.996852:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e6303584f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d475f expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.996862:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433dce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605061376:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.996891:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605061504:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298644.996910:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a540 going to sleep for 11 seconds 00000100:00100000:0.0:1780298644.996920:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.997654:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433dce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605061376:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.997674:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f4922155/0xaf35132e63035848 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4758 expref: 18 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.997685:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605061504:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.997687:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f4922155/0xaf35132e63035848 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4758 expref: 18 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.997704:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278600, rc 0) 00010000:00010000:2.0:1780298644.997712:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e6303584f lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d475f expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.997727:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e6303584f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d475f expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.997731:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2398:x1866777609649408:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 918us (1014us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.997737:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10143 00010000:00010000:2.0:1780298644.997750:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baea00, rc 0) 00000100:00100000:2.0:1780298644.997775:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255090a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609649536:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 947us (1030us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.997782:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10144 00000100:00100000:3.0:1780298644.998764:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.998808:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609649664 00000100:00100000:1.0:1780298644.998814:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609649664 00000100:00100000:1.0:1780298644.998851:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10145 00000100:00100000:1.0:1780298644.998861:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255093b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2399:x1866777609649664:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298644.998871:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298644.998880:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63035856 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4766 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298644.998916:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433dce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605061632:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298644.998937:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298644.998967:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a540 going to sleep for 11 seconds 00000100:00100000:2.0:1780298644.998972:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609649792 00000100:00100000:2.0:1780298644.998978:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609649792 00000100:00100000:2.0:1780298644.999013:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10146 00000100:00100000:2.0:1780298644.999022:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609649792:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298644.999031:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298644.999039:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002c458c77/0xaf35132e6303585d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d476d expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298644.999067:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605061760:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298644.999098:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb16c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298644.999682:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433dce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605061632:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298644.999702:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63035856 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4766 expref: 18 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.999719:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63035856 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4766 expref: 18 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298644.999729:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279a00, rc 0) 00000100:00100000:1.0:1780298644.999759:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255093b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2399:x1866777609649664:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 900us (994us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298644.999773:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10145 00000100:00100000:2.0:1780298644.999779:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605061760:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298644.999799:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002c458c77/0xaf35132e6303585d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d476d expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.999814:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002c458c77/0xaf35132e6303585d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d476d expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298644.999825:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2a00, rc 0) 00000100:00100000:2.0:1780298644.999860:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609649792:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 837us (921us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298644.999868:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10146 00000100:00100000:3.0:1780298645.000799:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.000831:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609649920 00000100:00100000:2.0:1780298645.000835:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609649920 00000100:00100000:2.0:1780298645.000863:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10147 00000100:00100000:2.0:1780298645.000870:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255092300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609649920:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.000879:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.000888:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e63035864 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4774 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.000906:0:19992:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000b25cdf87/0xaf35132e63035825 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d473c expref: 6 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.000915:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298644 v=(37 122) 00010000:00010000:2.0:1780298645.000918:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(38 122) 00010000:00010000:2.0:1780298645.000923:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e63035864 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3813376->3817471] (req 3813376->3817471) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4774 expref: 10 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.000940:0:19992:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000b25cdf87/0xaf35132e63035825 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d473c expref: 6 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.000950:0:19992:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000b25cdf87/0xaf35132e63035825 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d473c expref: 6 pid: 8359 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.000959:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605061888:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298645.000996:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb16c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.001568:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605061888:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298645.001583:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e63035864 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3813376->3817471] (req 3813376->3817471) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4774 expref: 10 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.001593:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3200, rc 0) 00000100:00100000:2.0:1780298645.001621:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255092300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609649920:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 747us (817us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.001628:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10147 00000100:00100000:3.0:1780298645.001891:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.001924:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609650048 00000100:00100000:2.0:1780298645.001928:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609650048 00010000:00010000:2.0:1780298645.001945:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000b25cdf87/0xaf35132e63035825 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d473c expref: 7 pid: 8359 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.001970:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15248 00000100:00100000:2.0:1780298645.001976:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255091f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609650048:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298645.001987:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298645.001997:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000b25cdf87/0xaf35132e63035825 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d473c expref: 7 pid: 8359 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.002009:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000b25cdf87/0xaf35132e63035825 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d473c expref: 7 pid: 8359 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.002026:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b25cdf87/0xaf35132e63035825 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d473c expref: 7 pid: 8359 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.002037:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(38 122) 00010000:00010000:2.0:1780298645.002040:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(38 122) 00010000:00010000:2.0:1780298645.002047:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e63035864 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3813376->37367807] (req 3813376->3817471) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4774 expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.002057:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e63035864 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3813376->37367807] (req 3813376->3817471) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4774 expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.002076:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e63035864 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3813376->37367807] (req 3813376->3817471) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4774 expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.002086:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372f480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:8374:1866777605062016:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298645.002113:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb16c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.002800:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372f480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:8374:1866777605062016:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298645.002810:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298645.002833:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255091f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609650048:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 857us (942us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.002841:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15248 00000100:00100000:0.0:1780298645.003044:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.003081:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609650176 00000100:00100000:2.0:1780298645.003086:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609650176 00002000:00100000:2.0:1780298645.003110:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (3813376->3817471) req@ffff9cf243846d80 x1866777609650176/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.003117:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243846d80 with x1866777609650176 ext(3813376->3817471) 00002000:00010000:2.0:1780298645.003120:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf243846d80 00000100:00100000:2.0:1780298645.003150:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6394 00000100:00100000:2.0:1780298645.003156:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243846d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609650176:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298645.003322:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243846d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609650176:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 166us (278us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298645.003329:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6394 00002000:00100000:2.0:1780298645.003335:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (3813376->3817471) req@ffff9cf243846d80 x1866777609650176/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/4536 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.003344:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243846d80 with x1866777609650176 ext(3813376->3817471) 00002000:00010000:2.0:1780298645.003347:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf243846d80 00000100:00100000:3.0:1780298645.004399:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.004433:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609650304 00000100:00100000:1.0:1780298645.004438:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609650304 00002000:00100000:1.0:1780298645.004455:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x12:0x0] (3817472->3821567) req@ffff9cf255090000 x1866777609650304/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.004462:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255090000 with x1866777609650304 ext(3817472->3821567) 00002000:00010000:1.0:1780298645.004465:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf255090000 00000100:00100000:1.0:1780298645.004480:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6395 00000100:00100000:1.0:1780298645.004485:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255090000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609650304:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298645.004553:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255090000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609650304:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 68us (154us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298645.004561:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6395 00002000:00100000:1.0:1780298645.004568:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x12:0x0] (3817472->3821567) req@ffff9cf255090000 x1866777609650304/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/4536 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.004583:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255090000 with x1866777609650304 ext(3817472->3821567) 00002000:00010000:1.0:1780298645.004588:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf255090000 00000100:00100000:3.0:1780298645.005599:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.005639:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609650432 00000100:00100000:2.0:1780298645.005643:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609650432 00002000:00100000:2.0:1780298645.005659:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (3821568->3825663) req@ffff9cf255093100 x1866777609650432/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.005666:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255093100 with x1866777609650432 ext(3821568->3825663) 00002000:00010000:2.0:1780298645.005668:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf255093100 00000100:00100000:2.0:1780298645.005687:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6396 00000100:00100000:2.0:1780298645.005693:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255093100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609650432:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298645.005779:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255093100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609650432:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 85us (180us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298645.005787:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6396 00002000:00100000:2.0:1780298645.005795:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (3821568->3825663) req@ffff9cf255093100 x1866777609650432/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/4536 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.005804:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255093100 with x1866777609650432 ext(3821568->3825663) 00002000:00010000:2.0:1780298645.005807:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf255093100 00000100:00100000:0.0:1780298645.006836:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.006876:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609650560 00000100:00100000:2.0:1780298645.006880:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609650560 00002000:00100000:2.0:1780298645.006897:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (3813376->3825663) req@ffff9cf243845c00 x1866777609650560/t0(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 12776/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.006904:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243845c00 with x1866777609650560 ext(3813376->3825663) 00002000:00010000:2.0:1780298645.006906:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf243845c00 00000100:00100000:2.0:1780298645.006923:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6397 00000100:00100000:2.0:1780298645.006931:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243845c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609650560:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:2.0:1780298645.007429:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243845c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609650560:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 499us (593us total) trans 4294969301 rc 0/0 00000100:00100000:2.0:1780298645.007434:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6397 00002000:00100000:2.0:1780298645.007439:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (3813376->3825663) req@ffff9cf243845c00 x1866777609650560/t4294969301(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 12776/448 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.007446:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243845c00 with x1866777609650560 ext(3813376->3825663) 00002000:00010000:2.0:1780298645.007453:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf243845c00 00000100:00100000:3.0:1780298645.008192:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.008221:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609650688 00000100:00100000:2.0:1780298645.008225:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609650688 00000100:00100000:2.0:1780298645.008248:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10148 00000100:00100000:2.0:1780298645.008253:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255092680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609650688:12345-192.168.203.25@tcp:16:fsx.0 00000001:00080000:1.0:1780298645.008917:0:8351:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0000: transno 4294969301 is committed 00000100:00100000:2.0:1780298645.008977:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255092680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609650688:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 723us (786us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.008983:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10148 00000100:00100000:3.0:1780298645.009759:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.009788:0:6476:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609650816 00000100:00100000:2.0:1780298645.009792:0:6476:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609650816 00000100:00100000:2.0:1780298645.009818:0:6476:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62055 00000100:00100000:2.0:1780298645.009822:0:6476:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255091180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+28:29360:x1866777609650816:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.009836:0:6476:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.009852:0:6476:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf350de2e00/0xaf35132e6303586b lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 9 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d477b expref: -99 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.009929:0:6476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf350de3800/0xaf35132e63035872 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 10 type: IBT flags: 0x40000000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.009946:0:6476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf350de3800/0xaf35132e63035872 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.009959:0:6476:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf350de3800/0xaf35132e63035872 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298645.009988:0:6476:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 8312473 00000004:00200000:2.0:1780298645.009993:0:6476:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298645.010014:0:6476:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf350de3800/0xaf35132e63035872 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.010042:0:6476:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf350de2e00/0xaf35132e6303586b lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 10 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d477b expref: 30 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.010064:0:6476:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf350de3800/0xaf35132e63035872 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d477b expref: 29 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.010074:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298645.010077:0:6476:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298645.010080:0:6476:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3800, rc 0) 00000100:00100000:2.0:1780298645.010105:0:6476:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255091180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+29:29360:x1866777609650816:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 283us (347us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.010110:0:6476:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62055 00000100:00100000:0.0:1780298645.011042:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.011077:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609650944 00000100:00100000:1.0:1780298645.011081:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609650944 00000100:00100000:1.0:1780298645.011107:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10149 00000100:00100000:1.0:1780298645.011112:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243845180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609650944:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.011120:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.011150:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005b92bd2b/0xaf35132e63035879 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4782 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.011194:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433dce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605062144:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.011250:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00002000:00010000:1.0:1780298645.011998:0:12101:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks: 40 -> 64 00000100:00100000:1.0:1780298645.012012:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433dce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605062144:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.012028:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005b92bd2b/0xaf35132e63035879 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4782 expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.012040:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005b92bd2b/0xaf35132e63035879 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4782 expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.012047:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279c00, rc 0) 00000100:00100000:1.0:1780298645.012071:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243845180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609650944:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 959us (1029us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.012078:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10149 00000100:00100000:3.0:1780298645.013001:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.013038:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609651072 00000100:00100000:2.0:1780298645.013042:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609651072 00000100:00100000:2.0:1780298645.013076:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10150 00000100:00100000:2.0:1780298645.013084:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255090700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609651072:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.013093:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.013101:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002c458c77/0xaf35132e63035880 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4789 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.013147:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605062272:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.013181:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb10c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.013832:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605062272:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.013847:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002c458c77/0xaf35132e63035880 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4789 expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.013857:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002c458c77/0xaf35132e63035880 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4789 expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.013865:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2a00, rc 0) 00000100:00100000:2.0:1780298645.013888:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255090700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609651072:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 807us (889us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.013893:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10150 00000100:00100000:3.0:1780298645.014864:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.014898:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609651200 00000100:00100000:2.0:1780298645.014902:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609651200 00000100:00100000:2.0:1780298645.014927:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10151 00000100:00100000:2.0:1780298645.014932:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609651200:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.014939:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.014945:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000024bbcf41/0xaf35132e63035887 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4790 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.014981:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605062400:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.015019:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb10c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.015664:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372f480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605062400:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.015676:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000024bbcf41/0xaf35132e63035887 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4790 expref: 7 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.015686:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000024bbcf41/0xaf35132e63035887 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4790 expref: 7 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.015692:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2c00, rc 0) 00000100:00100000:2.0:1780298645.015714:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609651200:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 782us (852us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.015718:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10151 00000100:00100000:0.0:1780298645.016762:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.016818:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609651328 00000100:00100000:3.0:1780298645.016823:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609651328 00000100:00080000:2.0:1780298645.016860:0:6473:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00002000:00100000:3.0:1780298645.016871:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (823296->827391) req@ffff9cf243844a80 x1866777609651328/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298645.016882:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243844a80 with x1866777609651328 ext(823296->827391) 00002000:00010000:3.0:1780298645.016887:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf243844a80 00000100:00100000:3.0:1780298645.016908:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6398 00000100:00100000:3.0:1780298645.016919:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243844a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2400:x1866777609651328:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298645.017101:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243844a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2400:x1866777609651328:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 185us (338us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1780298645.017109:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6398 00002000:00100000:3.0:1780298645.017116:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (823296->827391) req@ffff9cf243844a80 x1866777609651328/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/4536 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298645.017158:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243844a80 with x1866777609651328 ext(823296->827391) 00002000:00010000:3.0:1780298645.017162:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf243844a80 00000100:00100000:3.0:1780298645.018008:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.018048:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609651456 00000100:00100000:2.0:1780298645.018054:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609651456 00002000:00100000:2.0:1780298645.018075:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x2c0000401:0x11:0x0] (827392->831487) req@ffff9cf254918a80 x1866777609651456/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.018084:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf254918a80 with x1866777609651456 ext(827392->831487) 00002000:00010000:2.0:1780298645.018087:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf254918a80 00000100:00100000:2.0:1780298645.018107:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6399 00000100:00100000:2.0:1780298645.018115:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254918a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2398:x1866777609651456:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298645.018220:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254918a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2398:x1866777609651456:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 107us (213us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298645.018224:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6399 00002000:00100000:2.0:1780298645.018228:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x2c0000401:0x11:0x0] (827392->831487) req@ffff9cf254918a80 x1866777609651456/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/4536 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.018234:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf254918a80 with x1866777609651456 ext(827392->831487) 00002000:00010000:2.0:1780298645.018237:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf254918a80 00000100:00100000:2.0:1780298645.019342:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.019372:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609651584 00000100:00100000:1.0:1780298645.019377:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609651584 00002000:00100000:1.0:1780298645.019396:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (802816->868351) req@ffff9cf34372f480 x1866777609651584/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 504/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.019403:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf34372f480 with x1866777609651584 ext(802816->868351) 00002000:00010000:1.0:1780298645.019405:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf34372f480 00000100:00100000:1.0:1780298645.019424:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6400 00000100:00100000:1.0:1780298645.019429:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2400:x1866777609651584:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298645.020409:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2400:x1866777609651584:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 978us (1065us total) trans 0 rc 57344/57344 00000100:00100000:1.0:1780298645.020416:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6400 00002000:00100000:1.0:1780298645.020421:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (802816->868351) req@ffff9cf34372f480 x1866777609651584/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 504/440 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 57344/57344 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.020428:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf34372f480 with x1866777609651584 ext(802816->868351) 00002000:00010000:1.0:1780298645.020431:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf34372f480 00000100:00100000:2.0:1780298645.021309:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.021345:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609651712 00000100:00100000:0.0:1780298645.021350:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609651712 00000100:00100000:0.0:1780298645.021383:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10152 00000100:00100000:0.0:1780298645.021390:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609651712:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.021399:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.021413:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000075508ee2/0xaf35132e6303588e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4797 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.021461:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243846a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605062528:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.021523:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12f00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.023302:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243846a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605062528:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.023328:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000075508ee2/0xaf35132e6303588e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4797 expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.023344:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000075508ee2/0xaf35132e6303588e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4797 expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.023354:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae200, rc 0) 00000100:00100000:0.0:1780298645.023380:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609651712:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1991us (2074us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.023385:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10152 00000100:00100000:3.0:1780298645.024223:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.024268:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609651840 00000100:00100000:1.0:1780298645.024285:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609651840 00000100:00100000:1.0:1780298645.024320:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10153 00000100:00100000:1.0:1780298645.024327:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609651840:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.024337:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.024350:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005b92bd2b/0xaf35132e63035895 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d479e expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.024385:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433dce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605062656:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.024435:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.025081:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433dce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605062656:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.025099:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005b92bd2b/0xaf35132e63035895 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d479e expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.025112:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005b92bd2b/0xaf35132e63035895 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d479e expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.025121:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279c00, rc 0) 00000100:00100000:1.0:1780298645.025185:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609651840:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 859us (963us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.025192:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10153 00000100:00100000:3.0:1780298645.025957:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.025988:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.026002:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609651968 00000100:00100000:2.0:1780298645.026007:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609651968 00000100:00100000:2.0:1780298645.026036:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609652096 00000100:00100000:2.0:1780298645.026038:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609652096 00000100:00100000:0.0:1780298645.026042:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10154 00000100:00100000:3.0:1780298645.026053:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10155 00000100:00100000:0.0:1780298645.026058:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2399:x1866777609651968:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298645.026066:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609652096:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.026070:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.026074:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.026083:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006d4295ea/0xaf35132e630358a3 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d47ac expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.026083:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d518bb3a/0xaf35132e6303589c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d47a5 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.026118:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25491bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605062784:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.026152:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243846a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605062912:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.026177:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.026189:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12f00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.026800:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25491bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605062784:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.026823:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000006d4295ea/0xaf35132e630358a3 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d47ac expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.026838:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006d4295ea/0xaf35132e630358a3 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d47ac expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.026848:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157400, rc 0) 00000100:00100000:2.0:1780298645.026884:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609652096:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 820us (895us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.026896:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10155 00000100:00100000:0.0:1780298645.026903:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243846a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605062912:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.026924:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d518bb3a/0xaf35132e6303589c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d47a5 expref: 18 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.026940:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d518bb3a/0xaf35132e6303589c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d47a5 expref: 18 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.026958:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bafe00, rc 0) 00000100:00100000:0.0:1780298645.026985:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2399:x1866777609651968:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 934us (1027us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.026997:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10154 00000100:00100000:3.0:1780298645.027760:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.027791:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609652224 00000100:00100000:0.0:1780298645.027795:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609652224 00000100:00100000:0.0:1780298645.027824:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10156 00000100:00100000:0.0:1780298645.027830:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254919880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:29360:x1866777609652224:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.027838:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.027845:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e630358aa lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d47b3 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.027863:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e6303581e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4735 expref: 14 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.027873:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298644 v=(36 169) 00010000:00010000:0.0:1780298645.027877:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(37 169) 00010000:00010000:0.0:1780298645.027882:0:20252:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e630358aa lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3649536->3653631] (req 3649536->3653631) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d47b3 expref: 18 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.027904:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e6303581e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4735 expref: 14 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.027919:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e6303581e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4735 expref: 14 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.027931:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243846a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605063040:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298645.027973:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12f00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.028636:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243846a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605063040:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298645.028653:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e630358aa lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3649536->3653631] (req 3649536->3653631) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d47b3 expref: 18 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.028665:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bafa00, rc 0) 00000100:00100000:0.0:1780298645.028694:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254919880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:29360:x1866777609652224:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 864us (934us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.028701:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10156 00000100:00100000:3.0:1780298645.028877:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.028914:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609652352 00000100:00100000:2.0:1780298645.028918:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609652352 00010000:00010000:2.0:1780298645.028936:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e6303581e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4735 expref: 15 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.028961:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15249 00000100:00100000:2.0:1780298645.028967:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2400:x1866777609652352:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298645.028975:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298645.028983:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e6303581e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4735 expref: 15 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.028990:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e6303581e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4735 expref: 15 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.029002:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e6303581e lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4735 expref: 15 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.029016:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(37 169) 00010000:00010000:2.0:1780298645.029019:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(37 169) 00010000:00010000:2.0:1780298645.029027:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e630358aa lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3649536->37203967] (req 3649536->3653631) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d47b3 expref: 17 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.029036:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e630358aa lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3649536->37203967] (req 3649536->3653631) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d47b3 expref: 17 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.029060:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e630358aa lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3649536->37203967] (req 3649536->3653631) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d47b3 expref: 17 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.029071:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372d880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777605063168:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298645.029105:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.029726:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372d880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777605063168:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298645.029740:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298645.029768:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609652352:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 799us (890us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.029777:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15249 00000100:00100000:3.0:1780298645.029914:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.029958:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609652480 00000100:00100000:2.0:1780298645.029963:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609652480 00000100:00100000:2.0:1780298645.029994:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6401 00000100:00100000:2.0:1780298645.030000:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254919500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609652480:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298645.030240:0:8364:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:2.0:1780298645.030245:0:8364:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298645.030248:0:8364:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298645.030250:0:8364:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298645.030252:0:8364:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 416 -> 424 00000100:00100000:2.0:1780298645.030291:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254919500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2400:x1866777609652480:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 290us (376us total) trans 4294969755 rc 0/0 00000100:00100000:2.0:1780298645.030298:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6401 00000100:00100000:0.0:1780298645.031074:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.031120:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609652608 00000100:00100000:1.0:1780298645.031136:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609652608 00000100:00100000:1.0:1780298645.031168:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10157 00000100:00100000:1.0:1780298645.031178:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243846a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2397:x1866777609652608:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.031187:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298645.031196:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:1.0:1780298645.031198:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e630358b1 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d47ba expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.031230:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433dce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605063296:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.031237:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609652736 00000100:00100000:0.0:1780298645.031243:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609652736 00000100:00100000:1.0:1780298645.031268:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.031299:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10158 00000100:00100000:0.0:1780298645.031309:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254918700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609652736:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.031318:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.031329:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002ee949fd/0xaf35132e630358b8 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d47c1 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.031355:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f737b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605063424:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.031380:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12f00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.031889:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433dce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605063296:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.031906:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e630358b1 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d47ba expref: 20 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.031922:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e630358b1 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d47ba expref: 20 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.031935:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279a00, rc 0) 00000100:00100000:1.0:1780298645.031972:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243846a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2397:x1866777609652608:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 797us (899us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.031981:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10157 00000100:00100000:2.0:1780298645.032037:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f737b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605063424:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.032053:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002ee949fd/0xaf35132e630358b8 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d47c1 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.032067:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002ee949fd/0xaf35132e630358b8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d47c1 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.032076:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bafc00, rc 0) 00000100:00100000:2.0:1780298645.032107:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254918700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609652736:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 801us (914us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.032113:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10158 00000100:00100000:0.0:1780298645.032925:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.032958:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.032973:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609652864 00000100:00100000:1.0:1780298645.032978:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609652864 00000100:00100000:3.0:1780298645.032993:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609652992 00000100:00100000:3.0:1780298645.032999:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609652992 00000100:00100000:1.0:1780298645.033007:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10159 00000100:00100000:1.0:1780298645.033015:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f737800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2397:x1866777609652864:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.033024:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.033032:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f4922155/0xaf35132e630358bf lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d47c8 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.033037:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10160 00000100:00100000:3.0:1780298645.033051:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f734e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609652992:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.033059:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.033069:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c841b52/0xaf35132e630358c6 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d47cf expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.033070:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433dce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605063552:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.033106:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25491b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605063680:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.033115:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.033204:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.033794:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433dce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605063552:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.033816:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f4922155/0xaf35132e630358bf lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d47c8 expref: 20 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.033825:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25491b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605063680:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.033832:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f4922155/0xaf35132e630358bf lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d47c8 expref: 20 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.033840:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278600, rc 0) 00010000:00010000:2.0:1780298645.033849:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009c841b52/0xaf35132e630358c6 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d47cf expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.033864:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c841b52/0xaf35132e630358c6 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d47cf expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.033873:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f737800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2397:x1866777609652864:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 857us (947us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.033889:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10159 00010000:00010000:2.0:1780298645.033903:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157600, rc 0) 00000100:00100000:2.0:1780298645.033928:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f734e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609652992:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 881us (969us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.033935:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10160 00000100:00100000:0.0:1780298645.034755:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.034792:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609653120 00000100:00100000:2.0:1780298645.034796:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609653120 00000100:00100000:2.0:1780298645.034823:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10161 00000100:00100000:2.0:1780298645.034831:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f736300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2397:x1866777609653120:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.034838:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.034845:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000024bbcf41/0xaf35132e630358cd lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d47d6 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.034853:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.034874:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605063808:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.034881:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609653248 00000100:00100000:1.0:1780298645.034885:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609653248 00000100:00100000:1.0:1780298645.034909:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10162 00000100:00100000:1.0:1780298645.034917:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f734a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609653248:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.034924:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298645.034932:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1cc0 going to sleep for 11 seconds 00010000:00010000:1.0:1780298645.034932:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005b92bd2b/0xaf35132e630358d4 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d47dd expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.034957:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433dce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605063936:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.034984:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.035531:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605063808:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.035544:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000024bbcf41/0xaf35132e630358cd lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d47d6 expref: 20 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.035556:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000024bbcf41/0xaf35132e630358cd lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d47d6 expref: 20 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.035563:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2c00, rc 0) 00000100:00100000:2.0:1780298645.035594:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f736300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2397:x1866777609653120:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 764us (839us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.035602:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10161 00000100:00100000:1.0:1780298645.035687:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433dce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605063936:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.035700:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005b92bd2b/0xaf35132e630358d4 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d47dd expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.035714:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005b92bd2b/0xaf35132e630358d4 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d47dd expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.035724:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279c00, rc 0) 00000100:00100000:1.0:1780298645.035754:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f734a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609653248:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 838us (906us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.035759:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10162 00000100:00100000:0.0:1780298645.036561:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.036593:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609653376 00000100:00100000:2.0:1780298645.036597:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609653376 00000100:00100000:2.0:1780298645.036623:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10163 00000100:00100000:2.0:1780298645.036631:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f736a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:29360:x1866777609653376:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.036640:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.036648:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002c458c77/0xaf35132e630358db lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d47e4 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.036664:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(37 169) 00010000:00010000:2.0:1780298645.036669:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(37 169) 00010000:00010000:2.0:1780298645.036677:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002c458c77/0xaf35132e630358db lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3403776->3649535] (req 3403776->3407871) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d47e4 expref: 20 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.036691:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002c458c77/0xaf35132e630358db lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3403776->3649535] (req 3403776->3407871) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d47e4 expref: 20 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.036707:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2a00, rc 0) 00000100:00100000:2.0:1780298645.036727:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f736a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:29360:x1866777609653376:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 99us (168us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.036731:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10163 00000100:00100000:0.0:1780298645.037607:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.037652:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609653504 00000100:00100000:2.0:1780298645.037656:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609653504 00002000:00100000:2.0:1780298645.037684:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (3403776->3407871) req@ffff9cf37f737b80 x1866777609653504/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.037700:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f737b80 with x1866777609653504 ext(3403776->3407871) 00002000:00010000:2.0:1780298645.037704:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f737b80 00000100:00100000:2.0:1780298645.037725:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6402 00000100:00100000:2.0:1780298645.037734:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f737b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2400:x1866777609653504:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298645.037916:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f737b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2400:x1866777609653504:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 183us (307us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298645.037924:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6402 00002000:00100000:2.0:1780298645.037931:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (3403776->3407871) req@ffff9cf37f737b80 x1866777609653504/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/4536 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.037940:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f737b80 with x1866777609653504 ext(3403776->3407871) 00002000:00010000:2.0:1780298645.037944:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f737b80 00000100:00100000:3.0:1780298645.038926:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.038980:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609653632 00000100:00100000:1.0:1780298645.038988:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609653632 00002000:00100000:1.0:1780298645.039012:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (3407872->3411967) req@ffff9cf254919c00 x1866777609653632/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.039022:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf254919c00 with x1866777609653632 ext(3407872->3411967) 00002000:00010000:1.0:1780298645.039026:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf254919c00 00000100:00100000:1.0:1780298645.039048:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6403 00000100:00100000:1.0:1780298645.039055:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254919c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2397:x1866777609653632:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298645.039164:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254919c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2397:x1866777609653632:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 110us (244us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298645.039173:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6403 00002000:00100000:1.0:1780298645.039178:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (3407872->3411967) req@ffff9cf254919c00 x1866777609653632/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/4536 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.039185:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf254919c00 with x1866777609653632 ext(3407872->3411967) 00002000:00010000:1.0:1780298645.039187:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf254919c00 00000100:00100000:0.0:1780298645.040044:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.040088:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609653760 00000100:00100000:2.0:1780298645.040093:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609653760 00000100:00100000:2.0:1780298645.040167:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10164 00000100:00100000:2.0:1780298645.040179:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a82a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2397:x1866777609653760:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.040190:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.040203:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e630358e2 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d47eb expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.040247:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.040254:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605064064:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.040305:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609653888 00000100:00100000:0.0:1780298645.040311:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609653888 00000100:00100000:2.0:1780298645.040357:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.040360:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10165 00000100:00100000:0.0:1780298645.040372:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254919180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609653888:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.040382:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.040393:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e630358e9 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d47f2 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.040421:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241a82d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605064192:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.040453:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12f00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.040974:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605064064:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.041007:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e630358e2 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d47eb expref: 21 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.041024:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e630358e2 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d47eb expref: 21 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.041035:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2400, rc 0) 00000100:00100000:2.0:1780298645.041079:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a82a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2397:x1866777609653760:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 903us (1035us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.041088:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10164 00000100:00100000:0.0:1780298645.041164:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241a82d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605064192:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.041193:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e630358e9 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d47f2 expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.041211:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e630358e9 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d47f2 expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.041227:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baea00, rc 0) 00000100:00100000:0.0:1780298645.041262:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254919180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609653888:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 893us (1017us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.041271:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10165 00000100:00100000:3.0:1780298645.041900:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.041940:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609654016 00000100:00100000:1.0:1780298645.041945:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609654016 00000100:00100000:1.0:1780298645.041980:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10166 00000100:00100000:1.0:1780298645.041990:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2397:x1866777609654016:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.042001:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.042011:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e630358f0 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d47f9 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.042036:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.042042:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433dce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605064320:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.042089:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609654144 00000100:00100000:2.0:1780298645.042094:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609654144 00000100:00100000:1.0:1780298645.042095:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837af00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.042155:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10167 00000100:00100000:2.0:1780298645.042165:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609654144:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.042174:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.042190:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d828f4f8/0xaf35132e630358f7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4800 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.042221:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605064448:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.042268:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.042679:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433dce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605064320:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.042699:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e630358f0 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d47f9 expref: 21 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.042715:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e630358f0 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d47f9 expref: 21 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.042727:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279a00, rc 0) 00000100:00100000:1.0:1780298645.042769:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2397:x1866777609654016:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 781us (869us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.042779:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10166 00000100:00100000:2.0:1780298645.042920:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605064448:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.042937:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d828f4f8/0xaf35132e630358f7 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4800 expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.042950:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d828f4f8/0xaf35132e630358f7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4800 expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.042965:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2e00, rc 0) 00000100:00100000:2.0:1780298645.042996:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609654144:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 834us (962us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.043002:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10167 00000100:00100000:0.0:1780298645.043804:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.043834:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.043852:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609654272 00000100:00100000:1.0:1780298645.043858:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609654272 00000100:00100000:1.0:1780298645.043885:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609654400 00000100:00100000:1.0:1780298645.043888:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609654400 00000100:00100000:2.0:1780298645.043908:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10168 00000100:00100000:0.0:1780298645.043920:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10169 00000100:00100000:2.0:1780298645.043924:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a82d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2399:x1866777609654272:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.043935:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298645.043939:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a82300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609654400:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.043947:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000052b89172/0xaf35132e630358fe lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4807 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.043950:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.043961:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63035905 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d480e expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.043985:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605064576:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.043997:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241a82680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605064704:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.044045:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1600 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.044045:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12f00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.044677:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241a82680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605064704:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.044697:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605064576:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.044699:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63035905 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d480e expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.044719:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63035905 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d480e expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.044722:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000052b89172/0xaf35132e630358fe lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4807 expref: 21 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.044742:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae000, rc 0) 00010000:00010000:2.0:1780298645.044748:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000052b89172/0xaf35132e630358fe lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4807 expref: 21 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.044759:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2800, rc 0) 00000100:00100000:2.0:1780298645.044787:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a82d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2399:x1866777609654272:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 869us (983us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.044788:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a82300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609654400:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 855us (951us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.044801:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10168 00000100:00100000:0.0:1780298645.044817:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10169 00000100:00100000:3.0:1780298645.045646:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.045685:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.045697:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609654528 00000100:00100000:1.0:1780298645.045704:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609654528 00000100:00100000:2.0:1780298645.045727:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609654656 00000100:00100000:2.0:1780298645.045732:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609654656 00000100:00100000:1.0:1780298645.045749:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10170 00000100:00100000:1.0:1780298645.045760:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2397:x1866777609654528:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.045770:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298645.045777:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10171 00010000:00010000:1.0:1780298645.045781:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e6303590c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4815 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.045791:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609654656:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.045800:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.045809:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa41763f/0xaf35132e63035913 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d481c expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.045821:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433dce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605064832:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.045839:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605064960:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.045883:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837af00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.045886:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1600 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.046592:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605064960:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.046617:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000aa41763f/0xaf35132e63035913 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d481c expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.046620:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433dce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605064832:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.046641:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa41763f/0xaf35132e63035913 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d481c expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.046647:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e6303590c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4815 expref: 21 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.046659:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3600, rc 0) 00010000:00010000:1.0:1780298645.046670:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e6303590c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4815 expref: 21 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.046682:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279c00, rc 0) 00000100:00100000:2.0:1780298645.046700:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609654656:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 915us (1015us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.046710:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2397:x1866777609654528:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 952us (1064us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.046711:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10171 00000100:00100000:1.0:1780298645.046730:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10170 00000100:00100000:3.0:1780298645.048112:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.048186:0:17578:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609654912 00000100:00100000:1.0:1780298645.048193:0:17578:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609654912 00000100:00100000:1.0:1780298645.048239:0:17578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62056 00000100:00100000:1.0:1780298645.048248:0:17578:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+29:29360:x1866777609654912:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:1.0:1780298645.048308:0:17578:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298645.048327:0:17578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf350de3800/0xaf35132e63035872 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d477b expref: 29 pid: 6476 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.048344:0:17578:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298645.048347:0:17578:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298645.048513:0:17578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278600/0xaf35132e6303591a lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 9 type: IBT flags: 0x40000000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.048525:0:17578:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae600/0xaf35132e63035833 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4743 expref: 23 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.048531:0:17578:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278600/0xaf35132e6303591a lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.048549:0:17578:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae600/0xaf35132e63035833 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4743 expref: 23 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.048559:0:17578:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae600/0xaf35132e63035833 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4743 expref: 23 pid: 7411 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.048567:0:17578:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433dce00 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000_UUID:17578:1866777605065088:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298645.048619:0:17578:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.049206:0:17578:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433dce00 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000_UUID:17578:1866777605065088:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298645.049222:0:17578:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278600/0xaf35132e6303591a lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.049355:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.049392:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609655040 00000100:00100000:0.0:1780298645.049399:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609655040 00010000:00010000:0.0:1780298645.049424:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae600/0xaf35132e63035833 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4743 expref: 24 pid: 7411 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.049462:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15250 00000100:00100000:0.0:1780298645.049472:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+24:2399:x1866777609655040:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298645.049483:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298645.049489:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae600/0xaf35132e63035833 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4743 expref: 24 pid: 7411 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.049501:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae600/0xaf35132e63035833 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4743 expref: 24 pid: 7411 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.049516:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae600/0xaf35132e63035833 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4743 expref: 24 pid: 7411 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.049528:0:6463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298645.049532:0:6463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278600/0xaf35132e6303591a lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.049541:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278600/0xaf35132e6303591a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x40290000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.049555:0:6463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278600/0xaf35132e6303591a lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x40290000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.049578:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298645.049604:0:17578:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278600/0xaf35132e6303591a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.049612:0:17578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278600/0xaf35132e6303591a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.049618:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2399:x1866777609655040:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 146us (260us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.049629:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15250 00000001:00080000:1.0:1780298645.049657:0:17578:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991104 00000004:00010000:1.0:1780298645.049679:0:17578:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 000000002f7243f8 reply state 00000000a689448b transno 4294991105 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278600/0xaf35132e6303591a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.049690:0:17578:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278600/0xaf35132e6303591a lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.049696:0:17578:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298645.049698:0:17578:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1780298645.049700:0:17578:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1780298645.049753:0:17578:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+31:29360:x1866777609654912:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1507us (1644us total) trans 4294991105 rc 0/0 00000100:00100000:1.0:1780298645.049759:0:17578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62056 00000020:00080000:1.0:1780298645.049763:0:17578:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1780298645.050379:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.050421:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609655168 00000100:00100000:2.0:1780298645.050429:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609655168 00000100:00100000:2.0:1780298645.050468:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10172 00000100:00100000:2.0:1780298645.050475:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:29360:x1866777609655168:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.050490:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.050506:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d828f4f8/0xaf35132e63035921 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4823 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.050534:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e630358aa lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3649536->37203967] (req 3649536->3653631) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d47b3 expref: 19 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.050544:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000002c458c77/0xaf35132e630358db lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3403776->3649535] (req 3403776->3407871) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d47e4 expref: 19 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.050550:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 2 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(37 169) 00010000:00010000:2.0:1780298645.050553:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(39 169) 00010000:00010000:2.0:1780298645.050556:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d828f4f8/0xaf35132e63035921 lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4823 expref: 15 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.050582:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000002c458c77/0xaf35132e630358db lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3403776->3649535] (req 3403776->3407871) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d47e4 expref: 19 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.050596:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000002c458c77/0xaf35132e630358db lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3403776->3649535] (req 3403776->3407871) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d47e4 expref: 19 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.050610:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605065216:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298645.050698:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e630358aa lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3649536->37203967] (req 3649536->3653631) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d47b3 expref: 19 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.050718:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e630358aa lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3649536->37203967] (req 3649536->3653631) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d47b3 expref: 19 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.050728:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605065344:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298645.050760:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1600 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.051246:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605065216:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298645.051419:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605065344:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298645.051436:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d828f4f8/0xaf35132e63035921 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4823 expref: 15 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.051449:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2e00, rc 0) 00000100:00100000:2.0:1780298645.051482:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:29360:x1866777609655168:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1007us (1102us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.051490:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10172 00000100:00100000:3.0:1780298645.051693:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.051726:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609655296 00000100:00100000:0.0:1780298645.051731:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609655296 00010000:00010000:0.0:1780298645.051747:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e630358aa lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3649536->37203967] (req 3649536->3653631) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d47b3 expref: 20 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.051769:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15251 00000100:00100000:0.0:1780298645.051776:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3487c6850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2400:x1866777609655296:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298645.051784:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298645.051796:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e630358aa lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3649536->37203967] (req 3649536->3653631) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d47b3 expref: 20 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.051803:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e630358aa lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3649536->37203967] (req 3649536->3653631) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d47b3 expref: 20 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.051816:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e630358aa lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3649536->37203967] (req 3649536->3653631) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d47b3 expref: 20 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.051827:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298645.051859:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3487c6850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2400:x1866777609655296:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 85us (168us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.051864:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15251 00000100:00100000:3.0:1780298645.051887:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.051938:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609655424 00000100:00100000:2.0:1780298645.051946:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609655424 00010000:00010000:2.0:1780298645.051972:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000002c458c77/0xaf35132e630358db lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3403776->3649535] (req 3403776->3407871) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d47e4 expref: 19 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.052007:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15252 00000100:00100000:2.0:1780298645.052017:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2400:x1866777609655424:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298645.052028:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298645.052045:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000002c458c77/0xaf35132e630358db lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3403776->3649535] (req 3403776->3407871) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d47e4 expref: 19 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.052058:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000002c458c77/0xaf35132e630358db lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3403776->3649535] (req 3403776->3407871) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d47e4 expref: 19 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.052076:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002c458c77/0xaf35132e630358db lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3403776->3649535] (req 3403776->3407871) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d47e4 expref: 19 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.052090:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(39 169) 00010000:00010000:2.0:1780298645.052094:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(39 169) 00010000:00010000:2.0:1780298645.052101:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000d828f4f8/0xaf35132e63035921 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4823 expref: 14 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.052110:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d828f4f8/0xaf35132e63035921 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4823 expref: 14 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.052163:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000d828f4f8/0xaf35132e63035921 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4823 expref: 14 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.052175:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372c380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777605065472:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298645.052221:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1600 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.052951:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372c380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777605065472:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298645.052956:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:2.0:1780298645.052965:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298645.053005:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609655552 00000100:00100000:0.0:1780298645.053010:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609655552 00000100:00100000:2.0:1780298645.053048:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609655424:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1029us (1166us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.053055:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10173 00000100:00100000:2.0:1780298645.053067:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15252 00000100:00100000:0.0:1780298645.053069:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:29360:x1866777609655552:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.053083:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.053095:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63035928 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d482a expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.053113:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e63035864 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3813376->37367807] (req 3813376->3817471) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4774 expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.053119:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(38 122) 00010000:00010000:0.0:1780298645.053122:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(39 122) 00010000:00010000:0.0:1780298645.053143:0:20252:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63035928 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d482a expref: 7 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.053161:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e63035864 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3813376->37367807] (req 3813376->3817471) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4774 expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.053171:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e63035864 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3813376->37367807] (req 3813376->3817471) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4774 expref: 9 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.053179:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241a82680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605065600:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298645.053238:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.053736:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241a82680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605065600:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298645.053751:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63035928 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d482a expref: 7 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.053759:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae000, rc 0) 00000100:00100000:0.0:1780298645.053785:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609655552:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 723us (835us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.053793:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10173 00000100:00100000:3.0:1780298645.054026:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.054081:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609655680 00000100:00100000:2.0:1780298645.054087:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609655680 00010000:00010000:2.0:1780298645.054111:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e63035864 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3813376->37367807] (req 3813376->3817471) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4774 expref: 10 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.054165:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15253 00000100:00100000:2.0:1780298645.054182:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609655680:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298645.054195:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298645.054208:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e63035864 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3813376->37367807] (req 3813376->3817471) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4774 expref: 10 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.054220:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e63035864 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3813376->37367807] (req 3813376->3817471) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4774 expref: 10 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.054236:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e63035864 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3813376->37367807] (req 3813376->3817471) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4774 expref: 10 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.054248:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(39 122) 00010000:00010000:2.0:1780298645.054252:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(39 122) 00010000:00010000:2.0:1780298645.054257:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63035928 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d482a expref: 6 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.054265:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63035928 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d482a expref: 6 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.054301:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63035928 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d482a expref: 6 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.054312:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372c380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777605065728:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298645.054358:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1600 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.054974:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372c380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777605065728:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298645.054986:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298645.055016:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609655680:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 843us (994us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.055029:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15253 00000100:00100000:3.0:1780298645.055177:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.055202:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609655936 00000100:00100000:2.0:1780298645.055207:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609655936 00002000:00010000:2.0:1780298645.055234:0:19951:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4194304->18446744073709551615) 00002000:00100000:2.0:1780298645.055239:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24304c380 with x1866777609655936 ext(4194304->18446744073709551615) 00002000:00010000:2.0:1780298645.055248:0:19951:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24304c380. 00000100:00100000:0.0:1780298645.055265:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.055285:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6404 00000100:00100000:2.0:1780298645.055300:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609655936:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:3.0:1780298645.055315:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609655808 00000100:00100000:3.0:1780298645.055320:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609655808 00002000:00010000:3.0:1780298645.055336:0:19928:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (5850327->18446744073709551615) 00002000:00100000:3.0:1780298645.055339:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf241a82680 with x1866777609655808 ext(5850327->18446744073709551615) 00002000:00010000:3.0:1780298645.055344:0:19928:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf241a82680. 00000100:00100000:3.0:1780298645.055364:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6405 00000100:00100000:3.0:1780298645.055372:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a82680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2399:x1866777609655808:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:2.0:1780298645.055454:0:19951:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4118169 -> 4194304 00002000:00010000:2.0:1780298645.055459:0:19951:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298645.055461:0:19951:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298645.055464:0:19951:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298645.055467:0:19951:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 64 -> 64 00002000:00010000:3.0:1780298645.055490:0:19928:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 5850327 00002000:00010000:3.0:1780298645.055493:0:19928:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:3.0:1780298645.055495:0:19928:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:3.0:1780298645.055496:0:19928:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:3.0:1780298645.055498:0:19928:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 424 -> 424 00000100:00100000:2.0:1780298645.055503:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609655936:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 209us (330us total) trans 4294969302 rc 0/0 00000100:00100000:2.0:1780298645.055511:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6404 00002000:00010000:2.0:1780298645.055515:0:19951:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4194304->18446744073709551615) 00000100:00100000:3.0:1780298645.055521:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a82680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2399:x1866777609655808:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 149us (255us total) trans 4294969756 rc 0/0 00002000:00100000:2.0:1780298645.055521:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24304c380 with x1866777609655936 ext(4194304->18446744073709551615) 00002000:00010000:2.0:1780298645.055530:0:19951:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24304c380. 00000100:00100000:3.0:1780298645.055534:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6405 00002000:00010000:3.0:1780298645.055537:0:19928:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (5850327->18446744073709551615) 00002000:00100000:3.0:1780298645.055539:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf241a82680 with x1866777609655808 ext(5850327->18446744073709551615) 00002000:00010000:3.0:1780298645.055544:0:19928:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf241a82680. 00000100:00100000:0.0:1780298645.056697:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.056733:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609656064 00000100:00100000:2.0:1780298645.056738:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609656064 00002000:00100000:2.0:1780298645.056766:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (5840896->5844991) req@ffff9cf241a83b80 x1866777609656064/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.056776:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf241a83b80 with x1866777609656064 ext(5840896->5844991) 00002000:00010000:2.0:1780298645.056780:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf241a83b80 00000100:00100000:2.0:1780298645.056799:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6406 00000100:00100000:2.0:1780298645.056808:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a83b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609656064:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298645.056940:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a83b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609656064:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 133us (243us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298645.056948:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6406 00002000:00100000:2.0:1780298645.056955:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (5840896->5844991) req@ffff9cf241a83b80 x1866777609656064/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/4536 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.056964:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf241a83b80 with x1866777609656064 ext(5840896->5844991) 00002000:00010000:2.0:1780298645.056968:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf241a83b80 00000100:00100000:1.0:1780298645.057986:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.058027:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609656192 00000100:00100000:2.0:1780298645.058033:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609656192 00002000:00100000:2.0:1780298645.058054:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (5844992->5849087) req@ffff9cf2433dce00 x1866777609656192/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.058064:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2433dce00 with x1866777609656192 ext(5844992->5849087) 00002000:00010000:2.0:1780298645.058068:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2433dce00 00000100:00100000:2.0:1780298645.058086:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6407 00000100:00100000:2.0:1780298645.058095:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609656192:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298645.058209:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609656192:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 115us (223us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298645.058217:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6407 00002000:00100000:2.0:1780298645.058224:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (5844992->5849087) req@ffff9cf2433dce00 x1866777609656192/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/4536 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.058232:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2433dce00 with x1866777609656192 ext(5844992->5849087) 00002000:00010000:2.0:1780298645.058236:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2433dce00 00000100:00100000:1.0:1780298645.059302:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.059335:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609656320 00000100:00100000:3.0:1780298645.059339:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609656320 00002000:00100000:3.0:1780298645.059356:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (5820416->5853183) req@ffff9cf2433ddf80 x1866777609656320/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 504/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298645.059363:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2433ddf80 with x1866777609656320 ext(5820416->5853183) 00002000:00010000:3.0:1780298645.059365:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2433ddf80 00000100:00100000:3.0:1780298645.059380:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6408 00000100:00100000:3.0:1780298645.059385:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609656320:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298645.060052:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609656320:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 665us (749us total) trans 0 rc 24576/24576 00000100:00100000:3.0:1780298645.060057:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6408 00002000:00100000:3.0:1780298645.060062:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (5820416->5853183) req@ffff9cf2433ddf80 x1866777609656320/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 504/440 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 24576/24576 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298645.060068:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2433ddf80 with x1866777609656320 ext(5820416->5853183) 00002000:00010000:3.0:1780298645.060071:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2433ddf80 00000100:00100000:0.0:1780298645.061175:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.061210:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609656448 00000100:00100000:1.0:1780298645.061215:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609656448 00002000:00100000:1.0:1780298645.061240:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (5840896->5850326) req@ffff9cf347cd6a00 x1866777609656448/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 9920/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.061250:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf347cd6a00 with x1866777609656448 ext(5840896->5850326) 00002000:00010000:1.0:1780298645.061253:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf347cd6a00 00000100:00100000:1.0:1780298645.061283:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6409 00000100:00100000:1.0:1780298645.061294:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2397:x1866777609656448:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:1.0:1780298645.061761:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2397:x1866777609656448:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 470us (587us total) trans 4294969757 rc 0/0 00000100:00100000:1.0:1780298645.061767:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6409 00002000:00100000:1.0:1780298645.061772:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (5840896->5850326) req@ffff9cf347cd6a00 x1866777609656448/t4294969757(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 9920/448 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.061778:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf347cd6a00 with x1866777609656448 ext(5840896->5850326) 00002000:00010000:1.0:1780298645.061781:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf347cd6a00 00000100:00100000:1.0:1780298645.062661:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.062695:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609656576 00000100:00100000:0.0:1780298645.062699:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609656576 00000100:00100000:0.0:1780298645.062724:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10174 00000100:00100000:0.0:1780298645.062729:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2397:x1866777609656576:12345-192.168.203.25@tcp:16:fsx.0 00000001:00080000:1.0:1780298645.063547:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969757 is committed 00000001:00080000:1.0:1780298645.063560:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969755 is committed 00000100:00100000:0.0:1780298645.063690:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609656576:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 958us (1028us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.063700:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10174 00000100:00100000:2.0:1780298645.064634:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.064679:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609656704 00000100:00100000:0.0:1780298645.064684:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609656704 00000100:00100000:0.0:1780298645.064714:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62057 00000100:00100000:0.0:1780298645.064719:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+31:29360:x1866777609656704:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.064734:0:6477:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.064751:0:6477:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafa00/0xaf35132e6303592f lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 9 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4831 expref: -99 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.064812:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae600/0xaf35132e63035936 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 10 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.064826:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae600/0xaf35132e63035936 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.064833:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae600/0xaf35132e63035936 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1780298645.064853:0:6477:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 10044631 00000004:00200000:0.0:1780298645.064856:0:6477:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1780298645.064868:0:6477:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae600/0xaf35132e63035936 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.064884:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafa00/0xaf35132e6303592f lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 10 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4831 expref: 33 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.064896:0:6477:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae600/0xaf35132e63035936 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4831 expref: 32 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.064902:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298645.064904:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1780298645.064906:0:6477:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae600, rc 0) 00000100:00100000:0.0:1780298645.064931:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+32:29360:x1866777609656704:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 212us (299us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.064935:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62057 00000100:00100000:3.0:1780298645.066032:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.066075:0:7411:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609656832 00000100:00100000:0.0:1780298645.066080:0:7411:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609656832 00000100:00100000:0.0:1780298645.066115:0:7411:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62058 00000100:00100000:0.0:1780298645.066149:0:7411:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:29360:x1866777609656832:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.066158:0:7411:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.066167:0:7411:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf400/0xaf35132e6303593d lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 10 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4838 expref: -99 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.066201:0:7411:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf200/0xaf35132e63035944 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 11 type: IBT flags: 0x40000000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.066211:0:7411:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf200/0xaf35132e63035944 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT flags: 0x50210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.066217:0:7411:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf200/0xaf35132e63035944 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1780298645.066228:0:7411:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 10044631 00000004:00200000:0.0:1780298645.066230:0:7411:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1780298645.066237:0:7411:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf200/0xaf35132e63035944 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 7411 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.066249:0:7411:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf400/0xaf35132e6303593d lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 11 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4838 expref: 25 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.066258:0:7411:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf200/0xaf35132e63035944 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4838 expref: 24 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.066262:0:7411:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298645.066264:0:7411:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1780298645.066265:0:7411:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf200, rc 0) 00000100:00100000:0.0:1780298645.066299:0:7411:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+24:29360:x1866777609656832:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 178us (270us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.066304:0:7411:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62058 00000100:00100000:1.0:1780298645.067089:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.067120:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.067156:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609656960 00000100:00100000:3.0:1780298645.067160:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609656960 00000100:00100000:2.0:1780298645.067171:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609657088 00000100:00100000:2.0:1780298645.067176:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609657088 00000100:00100000:3.0:1780298645.067193:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10175 00000100:00100000:3.0:1780298645.067202:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2400:x1866777609656960:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.067208:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298645.067214:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10176 00010000:00010000:3.0:1780298645.067218:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e6303594b lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d483f expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.067228:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609657088:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.067239:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.067251:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63035952 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4846 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.067262:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304e680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605065856:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.067302:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2556ab480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605065984:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.067337:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.067337:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12cc0 going to sleep for 11 seconds 00002000:00010000:3.0:1780298645.068156:0:12101:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks: 424 -> 448 00000100:00100000:3.0:1780298645.068172:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304e680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605065856:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.068194:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e6303594b lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d483f expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.068210:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e6303594b lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d483f expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.068217:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157200, rc 0) 00000100:00100000:0.0:1780298645.068224:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2556ab480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605065984:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.068244:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63035952 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4846 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.068258:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63035952 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4846 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.068268:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae400, rc 0) 00000100:00100000:3.0:1780298645.068285:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2400:x1866777609656960:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1083us (1194us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.068293:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10175 00000100:00100000:0.0:1780298645.068304:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609657088:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1081us (1183us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.068311:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10176 00000100:00100000:2.0:1780298645.069356:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.069383:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.069400:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609657216 00000100:00100000:0.0:1780298645.069405:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609657216 00000100:00100000:1.0:1780298645.069421:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609657344 00000100:00100000:1.0:1780298645.069428:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609657344 00000100:00100000:0.0:1780298645.069440:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10177 00000100:00100000:0.0:1780298645.069447:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2397:x1866777609657216:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.069455:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.069462:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000058e2d0/0xaf35132e63035959 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d484d expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.069474:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10178 00000100:00100000:2.0:1780298645.069487:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244811880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609657344:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298645.069490:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2556ab480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605066112:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.069498:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.069507:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000024bbcf41/0xaf35132e63035960 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4854 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.069528:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.069533:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244810700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605066240:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.069559:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1180 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.070207:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2556ab480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605066112:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.070228:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000058e2d0/0xaf35132e63035959 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d484d expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.070238:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000058e2d0/0xaf35132e63035959 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d484d expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.070245:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baea00, rc 0) 00000100:00100000:0.0:1780298645.070270:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2397:x1866777609657216:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 825us (915us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.070284:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10177 00000100:00100000:3.0:1780298645.070331:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244810700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605066240:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.070382:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000024bbcf41/0xaf35132e63035960 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4854 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.070396:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000024bbcf41/0xaf35132e63035960 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4854 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.070407:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2c00, rc 0) 00000100:00100000:3.0:1780298645.070444:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244811880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609657344:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 961us (1060us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.070450:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10178 00000100:00100000:1.0:1780298645.071313:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.071340:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609657472 00000100:00100000:3.0:1780298645.071345:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609657472 00000100:00100000:3.0:1780298645.071382:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6410 00000100:00100000:3.0:1780298645.071390:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609657472:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:3.0:1780298645.071535:0:19928:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:3.0:1780298645.071539:0:19928:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:3.0:1780298645.071541:0:19928:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:3.0:1780298645.071543:0:19928:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:3.0:1780298645.071545:0:19928:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 64 -> 64 00000100:00100000:3.0:1780298645.071575:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609657472:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 185us (262us total) trans 4294969303 rc 0/0 00000100:00100000:3.0:1780298645.071582:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6410 00000100:00100000:2.0:1780298645.072746:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.072783:0:6476:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609657728 00000100:00100000:1.0:1780298645.072787:0:6476:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609657728 00000100:00100000:1.0:1780298645.072814:0:6476:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62059 00000100:00100000:1.0:1780298645.072820:0:6476:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244813480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+24:29360:x1866777609657728:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:1.0:1780298645.072852:0:6476:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298645.072867:0:6476:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf200/0xaf35132e63035944 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4838 expref: 24 pid: 7411 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.072879:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298645.072882:0:6476:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298645.072981:0:6476:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278000/0xaf35132e63035967 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 10 type: IBT flags: 0x40000000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.072990:0:6476:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae600/0xaf35132e63035936 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4831 expref: 31 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.073007:0:6476:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278000/0xaf35132e63035967 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 10 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.073037:0:6476:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae600/0xaf35132e63035936 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4831 expref: 31 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.073050:0:6476:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae600/0xaf35132e63035936 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4831 expref: 31 pid: 6477 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.073062:0:6476:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433dea00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6476:1866777605066368:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298645.073107:0:6476:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.073778:0:6476:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433dea00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:6476:1866777605066368:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298645.073792:0:6476:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278000/0xaf35132e63035967 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.073914:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.073947:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609657856 00000100:00100000:1.0:1780298645.073952:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609657856 00010000:00010000:1.0:1780298645.073969:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae600/0xaf35132e63035936 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4831 expref: 32 pid: 6477 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.073996:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15254 00000100:00100000:1.0:1780298645.074003:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244812680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+32:2399:x1866777609657856:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298645.074013:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298645.074018:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae600/0xaf35132e63035936 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4831 expref: 32 pid: 6477 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.074028:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae600/0xaf35132e63035936 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4831 expref: 32 pid: 6477 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.074041:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae600/0xaf35132e63035936 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4831 expref: 32 pid: 6477 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.074051:0:8374:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298645.074055:0:8374:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278000/0xaf35132e63035967 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.074062:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278000/0xaf35132e63035967 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 10 type: IBT flags: 0x40290000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.074069:0:8374:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278000/0xaf35132e63035967 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 10 type: IBT flags: 0x40290000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.074089:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298645.074116:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244812680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+31:2399:x1866777609657856:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 113us (201us total) trans 0 rc 0/0 00010000:00010000:2.0:1780298645.074121:0:6476:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278000/0xaf35132e63035967 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.074145:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15254 00010000:00010000:2.0:1780298645.074152:0:6476:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278000/0xaf35132e63035967 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1780298645.074216:0:6476:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991106 00000004:00010000:2.0:1780298645.074242:0:6476:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 000000007b1a9228 reply state 00000000568b6bb7 transno 4294991107 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278000/0xaf35132e63035967 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.074262:0:6476:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278000/0xaf35132e63035967 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT flags: 0x50210000000000 pid: 6476 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.074269:0:6476:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298645.074271:0:6476:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1780298645.074281:0:6476:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1780298645.074341:0:6476:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244813480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+26:29360:x1866777609657728:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1519us (1593us total) trans 4294991107 rc 0/0 00000100:00100000:2.0:1780298645.074349:0:6476:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62059 00000020:00080000:2.0:1780298645.074353:0:6476:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1780298645.075006:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.075036:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609657984 00000100:00100000:1.0:1780298645.075040:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609657984 00000100:00100000:1.0:1780298645.075062:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10179 00000100:00100000:1.0:1780298645.075067:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244811f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:29360:x1866777609657984:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.075075:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.075084:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e6303596e lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d485b expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.075099:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d828f4f8/0xaf35132e63035921 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4823 expref: 6 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.075105:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(39 169) 00010000:00010000:1.0:1780298645.075108:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(40 169) 00010000:00010000:1.0:1780298645.075111:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e6303596e lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d485b expref: 9 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.075122:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d828f4f8/0xaf35132e63035921 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4823 expref: 6 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.075172:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d828f4f8/0xaf35132e63035921 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4823 expref: 6 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.075183:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433dea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605066496:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298645.075224:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a600 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.075800:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433dea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605066496:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298645.075817:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e6303596e lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d485b expref: 9 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.075829:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279e00, rc 0) 00000100:00100000:3.0:1780298645.075860:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244811f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609657984:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 791us (853us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298645.075870:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10179 00000100:00100000:3.0:1780298645.076058:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.076100:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609658112 00000100:00100000:0.0:1780298645.076105:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609658112 00010000:00010000:0.0:1780298645.076154:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d828f4f8/0xaf35132e63035921 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4823 expref: 7 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.076182:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15255 00000100:00100000:0.0:1780298645.076189:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609658112:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298645.076199:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298645.076214:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d828f4f8/0xaf35132e63035921 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4823 expref: 7 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.076225:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d828f4f8/0xaf35132e63035921 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4823 expref: 7 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.076250:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d828f4f8/0xaf35132e63035921 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4823 expref: 7 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.076260:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(40 169) 00010000:00010000:0.0:1780298645.076264:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(40 169) 00010000:00010000:0.0:1780298645.076270:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e6303596e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d485b expref: 8 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.076290:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e6303596e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d485b expref: 8 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.076312:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e6303596e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d485b expref: 8 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.076323:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2556ab480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777605066624:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298645.076359:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.076975:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2556ab480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777605066624:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298645.076987:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298645.077011:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609658112:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 823us (954us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.077017:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15255 00000100:00100000:3.0:1780298645.077036:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.077062:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609658240 00000100:00100000:0.0:1780298645.077065:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609658240 00000100:00100000:0.0:1780298645.077095:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10180 00000100:00100000:0.0:1780298645.077101:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e4f3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609658240:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.077108:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.077116:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63035975 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4862 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.077151:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63035928 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d482a expref: 8 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.077158:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(39 122) 00010000:00010000:0.0:1780298645.077163:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(40 122) 00010000:00010000:0.0:1780298645.077168:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63035975 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4862 expref: 10 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.077187:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63035928 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d482a expref: 8 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.077195:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63035928 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d482a expref: 8 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.077202:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2556ab480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605066752:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298645.077229:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.077869:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2556ab480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605066752:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298645.077885:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63035975 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4862 expref: 10 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.077901:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf400, rc 0) 00000100:00100000:0.0:1780298645.077932:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e4f3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609658240:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 830us (895us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.077943:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10180 00000100:00100000:1.0:1780298645.078042:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.078086:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609658368 00000100:00100000:2.0:1780298645.078092:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609658368 00010000:00010000:2.0:1780298645.078117:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63035928 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d482a expref: 9 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.078163:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15256 00000100:00100000:2.0:1780298645.078173:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dc700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609658368:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298645.078182:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298645.078194:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63035928 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d482a expref: 9 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.078206:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63035928 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d482a expref: 9 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.078223:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63035928 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d482a expref: 9 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.078234:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(40 122) 00010000:00010000:2.0:1780298645.078238:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(40 122) 00010000:00010000:2.0:1780298645.078243:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63035975 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4862 expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.078252:0:6463:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63035975 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4862 expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.078287:0:6463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63035975 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4862 expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.078298:0:6463:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244810e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777605066880:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298645.078342:0:6463:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb10c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.079053:0:6463:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244810e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777605066880:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298645.079067:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298645.079098:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433dc700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609658368:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 927us (1057us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.079105:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15256 00000100:00100000:1.0:1780298645.079149:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.079185:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609658496 00000100:00100000:2.0:1780298645.079190:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609658496 00002000:00010000:2.0:1780298645.079210:0:8364:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:2.0:1780298645.079215:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2433dd880 with x1866777609658496 ext(4194304->18446744073709551615) 00002000:00010000:2.0:1780298645.079223:0:8364:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2433dd880. 00000100:00100000:2.0:1780298645.079247:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6411 00000100:00100000:2.0:1780298645.079255:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609658496:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:0.0:1780298645.079351:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.079375:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609658624 00000100:00100000:3.0:1780298645.079380:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609658624 00002000:00010000:3.0:1780298645.079396:0:19950:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (612600->18446744073709551615) 00002000:00100000:3.0:1780298645.079400:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2556a9f80 with x1866777609658624 ext(612600->18446744073709551615) 00002000:00010000:3.0:1780298645.079405:0:19950:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2556a9f80. 00000100:00100000:3.0:1780298645.079426:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6412 00000100:00100000:3.0:1780298645.079434:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609658624:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:2.0:1780298645.079493:0:8364:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 5850327 -> 4194304 00002000:00010000:2.0:1780298645.079496:0:8364:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298645.079498:0:8364:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298645.079499:0:8364:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298645.079501:0:8364:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 448 -> 424 00000100:00100000:2.0:1780298645.079526:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609658496:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 272us (405us total) trans 4294969758 rc 0/0 00000100:00100000:2.0:1780298645.079533:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6411 00002000:00010000:2.0:1780298645.079536:0:8364:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:2.0:1780298645.079539:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2433dd880 with x1866777609658496 ext(4194304->18446744073709551615) 00002000:00010000:2.0:1780298645.079543:0:8364:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2433dd880. 00002000:00010000:3.0:1780298645.079692:0:19950:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4194304 -> 612600 00002000:00010000:3.0:1780298645.079695:0:19950:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:3.0:1780298645.079698:0:19950:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:3.0:1780298645.079700:0:19950:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:3.0:1780298645.079702:0:19950:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 64 -> 0 00000100:00100000:3.0:1780298645.079733:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609658624:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 300us (380us total) trans 4294969304 rc 0/0 00000100:00100000:3.0:1780298645.079740:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6412 00002000:00010000:3.0:1780298645.079743:0:19950:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (612600->18446744073709551615) 00002000:00100000:3.0:1780298645.079746:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2556a9f80 with x1866777609658624 ext(612600->18446744073709551615) 00002000:00010000:3.0:1780298645.079750:0:19950:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2556a9f80. 00000100:00100000:0.0:1780298645.080479:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.080516:0:17578:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609658752 00000100:00100000:1.0:1780298645.080520:0:17578:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609658752 00000100:00100000:1.0:1780298645.080548:0:17578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62060 00000100:00100000:1.0:1780298645.080554:0:17578:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556aa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+31:29360:x1866777609658752:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.080562:0:17578:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.080572:0:17578:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e6303597c lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 10 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4869 expref: -99 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.080649:0:17578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278200/0xaf35132e63035983 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 11 type: IBT flags: 0x40000000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.080662:0:17578:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278200/0xaf35132e63035983 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.080672:0:17578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278200/0xaf35132e63035983 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1780298645.080691:0:17578:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 4806904 00000004:00200000:1.0:1780298645.080694:0:17578:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1780298645.080706:0:17578:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278200/0xaf35132e63035983 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.080722:0:17578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e6303597c lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 11 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4869 expref: 33 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.080731:0:17578:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278200/0xaf35132e63035983 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 10 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4869 expref: 32 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.080736:0:17578:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298645.080737:0:17578:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1780298645.080739:0:17578:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278200, rc 0) 00000100:00100000:1.0:1780298645.080763:0:17578:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556aa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+32:29360:x1866777609658752:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 210us (286us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298645.080767:0:17578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62060 00000100:00100000:2.0:1780298645.081592:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.081615:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.081629:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609658880 00000100:00100000:3.0:1780298645.081634:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609658880 00000100:00100000:0.0:1780298645.081646:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609659008 00000100:00100000:0.0:1780298645.081651:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609659008 00000100:00100000:3.0:1780298645.081671:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10181 00000100:00100000:3.0:1780298645.081685:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244810e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609658880:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298645.081685:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10182 00000100:00100000:2.0:1780298645.081694:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244811880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609659008:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.081696:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.081701:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.081707:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e6303598a lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4870 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.081709:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003540747e/0xaf35132e63035991 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4877 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.081756:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244811f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605067008:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.081761:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605067136:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.081800:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb16c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.081801:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.082603:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244811f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605067008:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.082621:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605067136:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.082629:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003540747e/0xaf35132e63035991 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4877 expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.082638:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e6303598a lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4870 expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.082648:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003540747e/0xaf35132e63035991 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4877 expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.082651:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008746f5b4/0xaf35132e6303598a lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4870 expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.082671:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157200, rc 0) 00010000:00010000:3.0:1780298645.082673:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3c00, rc 0) 00000100:00100000:0.0:1780298645.082697:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244810e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609658880:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1019us (1106us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.082704:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244811880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609659008:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1010us (1084us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.082704:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10181 00000100:00100000:3.0:1780298645.082716:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10182 00000100:00100000:0.0:1780298645.083586:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.083616:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.083617:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609659264 00000100:00100000:3.0:1780298645.083622:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609659264 00000100:00100000:2.0:1780298645.083652:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609659136 00000100:00100000:2.0:1780298645.083656:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609659136 00000100:00100000:3.0:1780298645.083657:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10183 00000100:00100000:3.0:1780298645.083670:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609659264:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.083679:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298645.083684:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10184 00010000:00010000:3.0:1780298645.083689:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044cfb726/0xaf35132e63035998 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4885 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.083693:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556ab800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609659136:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.083702:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.083710:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000024bbcf41/0xaf35132e6303599f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d487e expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.083739:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244810700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605067392:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.083742:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605067264:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.083774:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb16c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.083779:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.084408:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244810700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605067392:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.084421:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605067264:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.084425:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000024bbcf41/0xaf35132e6303599f lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d487e expref: 7 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.084437:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000024bbcf41/0xaf35132e6303599f lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d487e expref: 7 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.084443:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2c00, rc 0) 00010000:00010000:3.0:1780298645.084444:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000044cfb726/0xaf35132e63035998 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4885 expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.084461:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044cfb726/0xaf35132e63035998 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4885 expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.084473:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157e00, rc 0) 00000100:00100000:1.0:1780298645.084475:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556ab800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609659136:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 784us (858us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.084481:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10184 00000100:00100000:3.0:1780298645.084504:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609659264:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 837us (916us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.084512:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10183 00000100:00100000:1.0:1780298645.085492:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.085519:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609659392 00000100:00100000:0.0:1780298645.085524:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609659392 00000100:00100000:0.0:1780298645.085550:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10185 00000100:00100000:0.0:1780298645.085555:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:29360:x1866777609659392:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.085562:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.085569:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e630359a6 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d488c expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.085583:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e6303596e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d485b expref: 9 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.085589:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(40 169) 00010000:00010000:0.0:1780298645.085593:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(41 169) 00010000:00010000:0.0:1780298645.085596:0:20252:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e630359a6 lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2838528->2846719] (req 2838528->2846719) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d488c expref: 7 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.085610:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e6303596e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d485b expref: 9 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.085626:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e6303596e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d485b expref: 9 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.085634:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2556a9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605067520:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298645.085665:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.086317:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2556a9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605067520:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298645.086333:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e630359a6 lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2838528->2846719] (req 2838528->2846719) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d488c expref: 7 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.086343:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bafa00, rc 0) 00000100:00100000:2.0:1780298645.086365:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609659392:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 811us (875us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.086370:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10185 00000100:00100000:2.0:1780298645.086517:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.086553:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609659520 00000100:00100000:0.0:1780298645.086557:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609659520 00010000:00010000:0.0:1780298645.086578:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e6303596e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d485b expref: 10 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.086605:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15257 00000100:00100000:0.0:1780298645.086613:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244810700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609659520:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298645.086623:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298645.086635:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e6303596e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d485b expref: 10 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.086647:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e6303596e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d485b expref: 10 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.086663:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e6303596e lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d485b expref: 10 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.086674:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(41 169) 00010000:00010000:0.0:1780298645.086677:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(41 169) 00010000:00010000:0.0:1780298645.086685:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e630359a6 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2838528->2846719) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d488c expref: 6 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.086693:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e630359a6 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2838528->2846719) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d488c expref: 6 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.086716:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e630359a6 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2838528->2846719) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d488c expref: 6 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.086727:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2556ab480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777605067648:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298645.086765:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12000 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.087475:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2556ab480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777605067648:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298645.087486:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298645.087511:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244810700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609659520:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 900us (993us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.087517:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15257 00000100:00100000:2.0:1780298645.087928:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.087954:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609659648 00000100:00100000:3.0:1780298645.087960:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609659648 00002000:00100000:3.0:1780298645.087992:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (2838528->2846719) req@ffff9cf244811f80 x1866777609659648/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298645.088002:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244811f80 with x1866777609659648 ext(2838528->2846719) 00002000:00010000:3.0:1780298645.088007:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf244811f80 00000100:00100000:3.0:1780298645.088029:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6413 00000100:00100000:3.0:1780298645.088039:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244811f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609659648:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298645.088236:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244811f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609659648:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 199us (307us total) trans 0 rc 8192/8192 00000100:00100000:3.0:1780298645.088248:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6413 00002000:00100000:3.0:1780298645.088256:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (2838528->2846719) req@ffff9cf244811f80 x1866777609659648/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/8632 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 8192/8192 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298645.088265:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244811f80 with x1866777609659648 ext(2838528->2846719) 00002000:00010000:3.0:1780298645.088269:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf244811f80 00000100:00100000:2.0:1780298645.089294:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.089347:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609659776 00000100:00100000:3.0:1780298645.089352:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609659776 00000100:00100000:3.0:1780298645.089382:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10186 00000100:00100000:3.0:1780298645.089391:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343966d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609659776:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.089399:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.089411:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000700ba1da/0xaf35132e630359ad lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4893 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.089449:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605067776:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.089498:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856a80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.090181:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605067776:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.090200:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000700ba1da/0xaf35132e630359ad lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4893 expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.090215:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000700ba1da/0xaf35132e630359ad lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4893 expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.090225:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156000, rc 0) 00000100:00100000:2.0:1780298645.090292:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343966d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609659776:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 902us (997us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.090297:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10186 00000100:00100000:2.0:1780298645.091104:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.091166:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609659904 00000100:00100000:1.0:1780298645.091172:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609659904 00000100:00100000:1.0:1780298645.091208:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10187 00000100:00100000:1.0:1780298645.091218:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343966300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609659904:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.091228:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298645.091237:0:3624:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00010000:00010000:1.0:1780298645.091240:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e630359b4 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d489a expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.091241:0:3623:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1780298645.091253:0:3624:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343965880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a32054e7-c858-4203-a2c6-5654368a105b:3624:1866777605067904:0@lo:400:kworker.0 00000100:00100000:0.0:1780298645.091266:0:3623:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343965f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3623:1866777605068032:0@lo:400:kworker.0 00000100:00100000:1.0:1780298645.091295:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433dea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605068544:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.091321:0:3624:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1780298645.091338:0:3623:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1780298645.091355:0:6472:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777605067904 00000100:00100000:2.0:1780298645.091359:0:6472:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777605067904 00000100:00100000:1.0:1780298645.091366:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837ae40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.091371:0:3624:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3624:1866777605068672:0@lo:400:kworker.0 00000100:00100000:0.0:1780298645.091372:0:3623:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255489500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3623:1866777605068416:0@lo:400:kworker.0 00000100:00100000:2.0:1780298645.091391:0:6472:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 577 00000100:00100000:2.0:1780298645.091400:0:6472:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347c9bc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a32054e7-c858-4203-a2c6-5654368a105b+23:3624:x1866777605067904:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1780298645.091412:0:3623:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1780298645.091441:0:3623:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3623:1866777605069056:0@lo:400:kworker.0 00000100:00100000:3.0:1780298645.091448:0:3624:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1780298645.091458:0:6472:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347c9bc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:a32054e7-c858-4203-a2c6-5654368a105b+23:3624:x1866777605067904:12345-0@lo:400:kworker.0 Request processed in 60us (138us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.091465:0:6472:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 577 00000100:00100000:1.0:1780298645.091473:0:3626:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255489c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3626:1866777605068288:0@lo:400:kworker.0 00000100:00100000:0.0:1780298645.091479:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777605068032 00000100:00100000:0.0:1780298645.091484:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777605068032 00000100:00100000:2.0:1780298645.091503:0:9976:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777605068416 00000100:00100000:2.0:1780298645.091509:0:9976:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777605068416 00000100:00100000:0.0:1780298645.091519:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777605068672 00000100:00100000:0.0:1780298645.091524:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777605068672 00000100:00100000:3.0:1780298645.091538:0:3624:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3624:1866777605069184:0@lo:400:kworker.0 00000100:00100000:1.0:1780298645.091546:0:3626:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1780298645.091552:0:9976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1377 00000100:00100000:2.0:1780298645.091560:0:9976:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556ab480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3623:x1866777605068032:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1780298645.091560:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777605068288 00000100:00100000:0.0:1780298645.091565:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777605068288 00000100:00100000:1.0:1780298645.091579:0:3626:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255488380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3626:1866777605068928:0@lo:400:kworker.0 00000100:00100000:3.0:1780298645.091580:0:3624:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1780298645.091586:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1378 00000100:00100000:0.0:1780298645.091596:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556abb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3623:x1866777605068416:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1780298645.091615:0:3624:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343965880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:a32054e7-c858-4203-a2c6-5654368a105b:3624:1866777605067904:0@lo:400:kworker.0 00000100:00100000:2.0:1780298645.091622:0:9976:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556ab480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3623:x1866777605068032:12345-0@lo:400:kworker.0 Request processed in 62us (292us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298645.091631:0:3626:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1780298645.091633:0:9976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1377 00000100:00100000:0.0:1780298645.091642:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556abb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3623:x1866777605068416:12345-0@lo:400:kworker.0 Request processed in 47us (229us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.091649:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1378 00000100:00100000:2.0:1780298645.091652:0:9976:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777605069184 00000100:00100000:3.0:1780298645.091657:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777605068928 00000100:00100000:2.0:1780298645.091657:0:9976:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777605069184 00000100:00100000:3.0:1780298645.091663:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777605068928 00000100:00100000:1.0:1780298645.091663:0:3626:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3626:1866777605069440:0@lo:400:kworker.0 00000100:00100000:0.0:1780298645.091670:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1379 00000100:00100000:0.0:1780298645.091680:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3624:x1866777605068672:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1780298645.091692:0:9976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1380 00000100:00100000:2.0:1780298645.091703:0:9976:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433de680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3626:x1866777605068288:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1780298645.091703:0:3626:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1780298645.091708:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 861 00000100:00100000:0.0:1780298645.091729:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3624:x1866777605068672:12345-0@lo:400:kworker.0 Request processed in 52us (282us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.091737:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1379 00000100:00100000:3.0:1780298645.091743:0:3624:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3624:1866777605068672:0@lo:400:kworker.0 00000100:00100000:2.0:1780298645.091747:0:9976:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433de680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3626:x1866777605068288:12345-0@lo:400:kworker.0 Request processed in 47us (205us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.091755:0:9976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1380 00000100:00100000:1.0:1780298645.091759:0:3626:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255489c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3626:1866777605068288:0@lo:400:kworker.0 00000100:00100000:0.0:1780298645.091763:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1381 00000100:00100000:0.0:1780298645.091775:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3624:x1866777605069184:12345-0@lo:400:kworker.0 00000100:00080000:2.0:1780298645.091778:0:6473:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1780298645.091780:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3626:x1866777605068928:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1780298645.091780:0:3626:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [2->3] 00000100:00080000:2.0:1780298645.091782:0:6473:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1780298645.091787:0:6473:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1780298645.091792:0:3626:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343964380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3626:1866777605068160:0@lo:400:kworker.0 00000100:00100000:2.0:1780298645.091800:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777605069440 00000100:00100000:2.0:1780298645.091804:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777605069440 00000100:00080000:2.0:1780298645.091822:0:6473:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1780298645.091827:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3624:x1866777605069184:12345-0@lo:400:kworker.0 Request processed in 52us (246us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298645.091834:0:3624:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f746680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3624:1866777605069184:0@lo:400:kworker.0 00000100:00100000:1.0:1780298645.091837:0:3626:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1780298645.091841:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1381 00000100:00100000:2.0:1780298645.091853:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 862 00000100:00100000:1.0:1780298645.091870:0:3626:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3626:1866777605068800:0@lo:400:kworker.0 00000100:00100000:2.0:1780298645.091877:0:7406:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777605068160 00000100:00100000:2.0:1780298645.091881:0:7406:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777605068160 00000100:00100000:3.0:1780298645.091887:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3626:x1866777605068928:12345-0@lo:400:kworker.0 Request processed in 167us (256us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.091887:0:3623:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1780298645.091900:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 861 00000100:00100000:1.0:1780298645.091904:0:3626:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1780298645.091919:0:3623:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343965f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3623:1866777605068032:0@lo:400:kworker.0 00000100:00100000:2.0:1780298645.091928:0:7406:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 649 00000100:00100000:1.0:1780298645.091932:0:3626:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3626:1866777605069312:0@lo:400:kworker.0 00000100:00100000:2.0:1780298645.091938:0:7406:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3626:x1866777605068160:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1780298645.091946:0:3623:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255489500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3623:1866777605068416:0@lo:400:kworker.0 00000100:00100000:1.0:1780298645.091964:0:3626:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1780298645.091974:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777605068800 00000100:00100000:2.0:1780298645.091975:0:7406:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3626:x1866777605068160:12345-0@lo:400:kworker.0 Request processed in 40us (144us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.091979:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777605068800 00000100:00100000:2.0:1780298645.091983:0:7406:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 649 00000100:00100000:1.0:1780298645.092001:0:3626:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255488380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3626:1866777605068928:0@lo:400:kworker.0 00000100:00080000:0.0:1780298645.092005:0:6473:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1780298645.092006:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777605069312 00000100:00100000:2.0:1780298645.092008:0:9976:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777605069056 00000100:00100000:3.0:1780298645.092012:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777605069312 00000100:00100000:2.0:1780298645.092013:0:9976:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777605069056 00000100:00100000:1.0:1780298645.092026:0:3626:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343964380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3626:1866777605068160:0@lo:400:kworker.0 00000100:00100000:0.0:1780298645.092033:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 863 00000100:00080000:0.0:1780298645.092041:0:6473:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1780298645.092044:0:6473:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1780298645.092058:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3626:x1866777605068800:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1780298645.092064:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 864 00000100:00100000:1.0:1780298645.092068:0:17578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1382 00000100:00100000:3.0:1780298645.092077:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3626:x1866777605069312:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1780298645.092082:0:17578:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3623:x1866777605069056:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1780298645.092088:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+5:3626:x1866777605069440:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1780298645.092101:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3626:x1866777605068800:12345-0@lo:400:kworker.0 Request processed in 61us (203us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.092107:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 863 00000100:00100000:3.0:1780298645.092168:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3626:x1866777605069312:12345-0@lo:400:kworker.0 Request processed in 95us (204us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298645.092176:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 864 00000100:00100000:0.0:1780298645.092180:0:3623:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3623:1866777605069056:0@lo:400:kworker.0 00000100:00100000:1.0:1780298645.092182:0:17578:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3623:x1866777605069056:12345-0@lo:400:kworker.0 Request processed in 103us (295us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.092183:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+5:3626:x1866777605069440:12345-0@lo:400:kworker.0 Request processed in 320us (481us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298645.092200:0:17578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1382 00000100:00100000:2.0:1780298645.092201:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 862 00000100:00100000:0.0:1780298645.092222:0:3623:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3623:1866777605069568:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1780298645.092247:0:3626:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3626:1866777605068800:0@lo:400:kworker.0 00000100:00100000:0.0:1780298645.092261:0:3623:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1780298645.092266:0:3626:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f744000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3626:1866777605069312:0@lo:400:kworker.0 00000100:00100000:1.0:1780298645.092300:0:3626:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f745500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3626:1866777605069440:0@lo:400:kworker.0 00000100:00100000:1.0:1780298645.092351:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433dea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605068544:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.092370:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e630359b4 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d489a expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.092387:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e630359b4 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d489a expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.092399:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279200, rc 0) 00000100:00100000:1.0:1780298645.092429:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343966300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609659904:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1214us (1325us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.092436:0:8360:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777605069568 00000100:00100000:1.0:1780298645.092437:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10187 00000100:00100000:0.0:1780298645.092441:0:8360:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777605069568 00000100:00100000:0.0:1780298645.092474:0:8360:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 589 00000100:00100000:0.0:1780298645.092481:0:8360:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3623:x1866777605069568:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1780298645.092493:0:8360:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 81920 granted 969636672 pending 0 free 3917541376 avail 3685711872 00000100:00100000:0.0:1780298645.092536:0:8360:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3623:x1866777605069568:12345-0@lo:13:osp-pre-1-1.0 Request processed in 49us (270us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.092546:0:8360:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 589 00000100:00100000:1.0:1780298645.092570:0:3623:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3623:1866777605069568:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1780298645.093413:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.093465:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609660032 00000100:00100000:0.0:1780298645.093470:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609660032 00000100:00100000:0.0:1780298645.093504:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62061 00000100:00100000:0.0:1780298645.093513:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+26:29360:x1866777609660032:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.093526:0:6477:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.093541:0:6477:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baec00/0xaf35132e630359bb lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 11 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d48a1 expref: -99 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.093618:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafc00/0xaf35132e630359c2 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 12 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.093634:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafc00/0xaf35132e630359c2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.093646:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafc00/0xaf35132e630359c2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1780298645.093671:0:6477:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 4806904 00000004:00200000:0.0:1780298645.093675:0:6477:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1780298645.093692:0:6477:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafc00/0xaf35132e630359c2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.093715:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baec00/0xaf35132e630359bb lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 12 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48a1 expref: 28 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.093728:0:6477:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafc00/0xaf35132e630359c2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48a1 expref: 27 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.093737:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298645.093740:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1780298645.093743:0:6477:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bafc00, rc 0) 00000100:00100000:0.0:1780298645.093774:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+27:29360:x1866777609660032:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 263us (362us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.093780:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62061 00000100:00100000:3.0:1780298645.094788:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.094819:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609660160 00000100:00100000:2.0:1780298645.094823:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609660160 00000100:00100000:2.0:1780298645.094847:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10188 00000100:00100000:2.0:1780298645.094852:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609660160:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.094859:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.094868:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e630359c9 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d48a8 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.094884:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(41 169) 00010000:00010000:2.0:1780298645.094888:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(41 169) 00010000:00010000:2.0:1780298645.094893:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e630359c9 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48a8 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.094903:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e630359c9 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48a8 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.094910:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2400, rc 0) 00000100:00100000:2.0:1780298645.094935:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609660160:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 83us (149us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.094939:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10188 00000100:00100000:3.0:1780298645.095856:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.095893:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609660288 00000100:00100000:0.0:1780298645.095898:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609660288 00000100:00100000:0.0:1780298645.095927:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10189 00000100:00100000:0.0:1780298645.095935:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609660288:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.095943:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.095951:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d518bb3a/0xaf35132e630359d0 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d48af expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.095968:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e630359a6 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2838528->2846719) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d488c expref: 6 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.095977:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e630359c9 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48a8 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.095988:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(41 169) 00010000:00010000:0.0:1780298645.095991:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(41 169) 00010000:00010000:0.0:1780298645.095996:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d518bb3a/0xaf35132e630359d0 lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3371008->3383295] (req 3371008->3383295) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48af expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.096019:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e630359c9 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48a8 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.096032:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e630359c9 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48a8 expref: 11 pid: 19992 timeout: 934 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.096043:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605069696:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298645.096098:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e630359a6 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2838528->2846719) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d488c expref: 6 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.096109:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e630359a6 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2838528->2846719) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d488c expref: 6 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.096119:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605069824:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298645.096174:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12000 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.096870:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605069696:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298645.096881:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605069824:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298645.096901:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d518bb3a/0xaf35132e630359d0 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3371008->3383295] (req 3371008->3383295) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48af expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.096913:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bafe00, rc 0) 00000100:00100000:0.0:1780298645.096944:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:29360:x1866777609660288:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1009us (1087us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.096951:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10189 00000100:00100000:3.0:1780298645.097051:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.097087:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609660544 00000100:00100000:2.0:1780298645.097092:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609660544 00010000:00010000:2.0:1780298645.097112:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e630359a6 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2838528->2846719) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d488c expref: 7 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.097117:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.097171:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609660416 00000100:00100000:2.0:1780298645.097175:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609660416 00000100:00100000:0.0:1780298645.097184:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15258 00010000:00010000:2.0:1780298645.097189:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e630359c9 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48a8 expref: 11 pid: 19992 timeout: 934 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.097197:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3487c7450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609660544:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298645.097209:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1780298645.097216:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15259 00000100:00100000:2.0:1780298645.097222:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609660416:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298645.097227:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e630359a6 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2838528->2846719) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d488c expref: 7 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.097230:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298645.097239:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e630359c9 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48a8 expref: 11 pid: 19992 timeout: 934 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.097244:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e630359a6 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 2838528->2846719) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d488c expref: 7 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.097267:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e630359c9 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48a8 expref: 11 pid: 19992 timeout: 934 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.097299:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b35ce980/0xaf35132e630359a6 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2838528->2846719) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d488c expref: 7 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.097306:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e630359c9 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48a8 expref: 11 pid: 19992 timeout: 934 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.097317:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(41 169) 00010000:00010000:0.0:1780298645.097322:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(41 169) 00010000:00010000:0.0:1780298645.097330:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000d518bb3a/0xaf35132e630359d0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3371008->36925439] (req 3371008->3383295) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48af expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.097338:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d518bb3a/0xaf35132e630359d0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3371008->36925439] (req 3371008->3383295) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48af expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.097348:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1780298645.097364:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000d518bb3a/0xaf35132e630359d0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3371008->36925439] (req 3371008->3383295) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48af expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.097373:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609660416:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 152us (259us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.097379:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fea00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777605069952:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298645.097381:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15259 00000100:00100000:0.0:1780298645.097418:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12000 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.097975:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fea00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777605069952:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298645.097983:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298645.098004:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3487c7450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609660544:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 813us (957us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.098009:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15258 00000100:00100000:1.0:1780298645.098168:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.098195:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609660672 00000100:00100000:0.0:1780298645.098200:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609660672 00000100:00100000:0.0:1780298645.098236:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6414 00000100:00100000:0.0:1780298645.098244:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609660672:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298645.098413:0:8362:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:0.0:1780298645.098417:0:8362:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298645.098419:0:8362:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298645.098421:0:8362:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298645.098424:0:8362:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 424 -> 424 00000100:00100000:0.0:1780298645.098454:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609660672:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 210us (289us total) trans 4294969759 rc 0/0 00000100:00100000:0.0:1780298645.098462:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6414 00000100:00100000:3.0:1780298645.099366:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.099425:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609660800 00000100:00100000:0.0:1780298645.099429:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609660800 00000100:00100000:0.0:1780298645.099462:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10190 00000100:00100000:0.0:1780298645.099476:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609660800:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298645.099478:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:0.0:1780298645.099489:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.099499:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000075508ee2/0xaf35132e630359d7 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d48b6 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.099526:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609660928 00000100:00100000:1.0:1780298645.099531:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609660928 00000100:00100000:0.0:1780298645.099542:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605070080:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.099569:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10191 00000100:00100000:1.0:1780298645.099579:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609660928:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.099588:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298645.099594:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12000 going to sleep for 11 seconds 00010000:00010000:1.0:1780298645.099599:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e630359de lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d48bd expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.099630:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605070208:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.099672:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837ae40 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.100365:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605070080:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.100390:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000075508ee2/0xaf35132e630359d7 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48b6 expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.100411:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000075508ee2/0xaf35132e630359d7 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48b6 expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.100429:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae200, rc 0) 00000100:00100000:1.0:1780298645.100460:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605070208:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.100469:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609660800:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 998us (1102us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.100477:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10190 00010000:00010000:1.0:1780298645.100481:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e630359de lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48bd expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.100505:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e630359de lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48bd expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.100516:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279200, rc 0) 00000100:00100000:1.0:1780298645.100544:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609660928:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 968us (1072us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.100552:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10191 00000100:00100000:3.0:1780298645.101571:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.101604:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.101630:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609661056 00000100:00100000:2.0:1780298645.101638:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609661056 00000100:00100000:2.0:1780298645.101686:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609661184 00000100:00100000:2.0:1780298645.101690:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609661184 00000100:00100000:0.0:1780298645.101696:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10192 00000100:00100000:0.0:1780298645.101714:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609661056:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298645.101716:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10193 00000100:00100000:3.0:1780298645.101730:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609661184:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.101736:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.101740:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.101753:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000700ba1da/0xaf35132e630359ec lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d48cb expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.101753:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630359e5 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d48c4 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.101825:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605070336:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.101826:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605070464:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.101896:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856180 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.101897:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12000 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.102862:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605070464:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.102884:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605070336:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.102893:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630359e5 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48c4 expref: 7 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.102913:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000700ba1da/0xaf35132e630359ec lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48cb expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.102917:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4d8e3d7/0xaf35132e630359e5 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48c4 expref: 7 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.102953:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf000, rc 0) 00010000:00010000:1.0:1780298645.102971:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000700ba1da/0xaf35132e630359ec lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48cb expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.102984:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156000, rc 0) 00000100:00100000:0.0:1780298645.102999:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609661056:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1290us (1434us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.103009:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10192 00000100:00100000:1.0:1780298645.103021:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609661184:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1292us (1415us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.103033:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10193 00000100:00100000:2.0:1780298645.105065:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.105109:0:17567:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609661440 00000100:00100000:0.0:1780298645.105117:0:17567:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609661440 00000100:00100000:0.0:1780298645.105190:0:17567:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62062 00000100:00100000:0.0:1780298645.105203:0:17567:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f744700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+32:29360:x1866777609661440:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:0.0:1780298645.105253:0:17567:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298645.105285:0:17567:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278200/0xaf35132e63035983 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4869 expref: 32 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.105301:0:17567:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298645.105305:0:17567:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1780298645.105477:0:17567:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baec00/0xaf35132e630359f3 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 11 type: IBT flags: 0x40000000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.105491:0:17567:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafc00/0xaf35132e630359c2 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48a1 expref: 26 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.105500:0:17567:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baec00/0xaf35132e630359f3 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 11 type: IBT flags: 0x50210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.105524:0:17567:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafc00/0xaf35132e630359c2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48a1 expref: 26 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.105537:0:17567:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafc00/0xaf35132e630359c2 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48a1 expref: 26 pid: 6477 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.105549:0:17567:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fd500 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:17567:1866777605070592:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298645.105611:0:17567:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.106543:0:17567:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fd500 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:17567:1866777605070592:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298645.106558:0:17567:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baec00/0xaf35132e630359f3 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.106744:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.106787:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609661568 00000100:00100000:1.0:1780298645.106795:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609661568 00010000:00010000:1.0:1780298645.106829:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafc00/0xaf35132e630359c2 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48a1 expref: 27 pid: 6477 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.106863:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15260 00000100:00100000:1.0:1780298645.106872:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+27:2399:x1866777609661568:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298645.106887:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298645.106894:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafc00/0xaf35132e630359c2 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48a1 expref: 27 pid: 6477 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.106907:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafc00/0xaf35132e630359c2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48a1 expref: 27 pid: 6477 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.106925:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafc00/0xaf35132e630359c2 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48a1 expref: 27 pid: 6477 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.106943:0:8374:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298645.106955:0:8374:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baec00/0xaf35132e630359f3 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.106966:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baec00/0xaf35132e630359f3 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 11 type: IBT flags: 0x40290000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.106974:0:8374:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baec00/0xaf35132e630359f3 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 11 type: IBT flags: 0x40290000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.107006:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1780298645.107029:0:17567:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baec00/0xaf35132e630359f3 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.107037:0:17567:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baec00/0xaf35132e630359f3 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.107057:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+26:2399:x1866777609661568:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 185us (317us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298645.107070:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15260 00000001:00080000:0.0:1780298645.107082:0:17567:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991108 00000004:00010000:0.0:1780298645.107109:0:17567:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000b96b1c81 reply state 0000000093d730d3 transno 4294991109 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baec00/0xaf35132e630359f3 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.107121:0:17567:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baec00/0xaf35132e630359f3 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 10 type: IBT flags: 0x50210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.107154:0:17567:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298645.107158:0:17567:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1780298645.107162:0:17567:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1780298645.107243:0:17567:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f744700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+34:29360:x1866777609661440:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 2042us (2176us total) trans 4294991109 rc 0/0 00000100:00100000:0.0:1780298645.107253:0:17567:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62062 00000020:00080000:0.0:1780298645.107258:0:17567:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1780298645.108022:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.108074:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609661696 00000100:00100000:0.0:1780298645.108080:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609661696 00000100:00100000:0.0:1780298645.108116:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10194 00000100:00100000:0.0:1780298645.108156:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:29360:x1866777609661696:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.108167:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.108182:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e630359fa lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d48d2 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.108203:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d518bb3a/0xaf35132e630359d0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3371008->36925439] (req 3371008->3383295) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48af expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.108212:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(41 169) 00010000:00010000:0.0:1780298645.108216:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(42 169) 00010000:00010000:0.0:1780298645.108220:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e630359fa lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48d2 expref: 7 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.108243:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d518bb3a/0xaf35132e630359d0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3371008->36925439] (req 3371008->3383295) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48af expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.108256:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d518bb3a/0xaf35132e630359d0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3371008->36925439] (req 3371008->3383295) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48af expref: 10 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.108267:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605070720:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298645.108325:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12000 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.108968:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605070720:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298645.108985:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e630359fa lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48d2 expref: 7 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.108998:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf600, rc 0) 00000100:00100000:0.0:1780298645.109031:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609661696:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 909us (1012us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.109038:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10194 00000100:00100000:2.0:1780298645.109174:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.109213:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609661824 00000100:00100000:3.0:1780298645.109219:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609661824 00010000:00010000:3.0:1780298645.109244:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d518bb3a/0xaf35132e630359d0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3371008->36925439] (req 3371008->3383295) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48af expref: 11 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.109290:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15261 00000100:00100000:3.0:1780298645.109301:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f744000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609661824:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298645.109313:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298645.109329:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d518bb3a/0xaf35132e630359d0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3371008->36925439] (req 3371008->3383295) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48af expref: 11 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.109341:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d518bb3a/0xaf35132e630359d0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3371008->36925439] (req 3371008->3383295) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48af expref: 11 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.109358:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d518bb3a/0xaf35132e630359d0 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3371008->36925439] (req 3371008->3383295) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48af expref: 11 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.109372:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(42 169) 00010000:00010000:3.0:1780298645.109377:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(42 169) 00010000:00010000:3.0:1780298645.109383:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e630359fa lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48d2 expref: 6 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.109393:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e630359fa lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48d2 expref: 6 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.109423:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e630359fa lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48d2 expref: 6 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.109437:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304e300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777605070848:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298645.109492:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856180 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.110166:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304e300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777605070848:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298645.110181:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298645.110226:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f744000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609661824:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 927us (1053us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.110235:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15261 00000100:00100000:2.0:1780298645.110352:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.110397:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609661952 00000100:00100000:1.0:1780298645.110401:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609661952 00000100:00100000:1.0:1780298645.110432:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10195 00000100:00100000:1.0:1780298645.110441:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f746680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:29360:x1866777609661952:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.110453:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.110464:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63035a01 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d48d9 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.110483:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63035975 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4862 expref: 10 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.110491:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(40 122) 00010000:00010000:1.0:1780298645.110495:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(41 122) 00010000:00010000:1.0:1780298645.110499:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63035a01 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48d9 expref: 9 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.110518:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63035975 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4862 expref: 10 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.110529:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63035975 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4862 expref: 10 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.110539:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605070976:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298645.110574:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837ae40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.111171:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605070976:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298645.111188:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63035a01 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48d9 expref: 9 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.111200:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279200, rc 0) 00000100:00100000:1.0:1780298645.111226:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f746680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:29360:x1866777609661952:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 787us (874us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298645.111233:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10195 00000100:00100000:3.0:1780298645.111418:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.111456:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609662080 00000100:00100000:2.0:1780298645.111461:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609662080 00010000:00010000:2.0:1780298645.111481:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63035975 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4862 expref: 11 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.111511:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15262 00000100:00100000:2.0:1780298645.111519:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609662080:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298645.111530:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298645.111542:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63035975 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4862 expref: 11 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.111559:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63035975 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4862 expref: 11 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.111574:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000441cddf1/0xaf35132e63035975 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4862 expref: 11 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.111585:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(41 122) 00010000:00010000:2.0:1780298645.111589:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(41 122) 00010000:00010000:2.0:1780298645.111595:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63035a01 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48d9 expref: 8 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.111603:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63035a01 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48d9 expref: 8 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.111626:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63035a01 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48d9 expref: 8 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.111637:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244812680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:7400:1866777605071104:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298645.111677:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1a80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.112336:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244812680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:7400:1866777605071104:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298645.112348:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298645.112376:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609662080:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 858us (957us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298645.112381:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15262 00000100:00100000:3.0:1780298645.112466:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.112502:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609662208 00000100:00100000:2.0:1780298645.112506:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609662208 00002000:00010000:2.0:1780298645.112523:0:19928:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4618229->18446744073709551615) 00002000:00100000:2.0:1780298645.112526:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25548aa00 with x1866777609662208 ext(4618229->18446744073709551615) 00002000:00010000:2.0:1780298645.112532:0:19928:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf25548aa00. 00000100:00100000:2.0:1780298645.112552:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6415 00000100:00100000:2.0:1780298645.112558:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609662208:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:3.0:1780298645.112593:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.112638:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609662336 00000100:00100000:0.0:1780298645.112645:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609662336 00002000:00010000:0.0:1780298645.112673:0:8363:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4194304->18446744073709551615) 00002000:00100000:0.0:1780298645.112678:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25548a300 with x1866777609662336 ext(4194304->18446744073709551615) 00002000:00010000:0.0:1780298645.112686:0:8363:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf25548a300. 00002000:00010000:2.0:1780298645.112718:0:19928:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4618229 00000100:00100000:0.0:1780298645.112719:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6416 00002000:00010000:2.0:1780298645.112724:0:19928:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298645.112728:0:19928:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298645.112730:0:19928:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298645.112733:0:19928:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 424 -> 424 00000100:00100000:0.0:1780298645.112739:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609662336:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:2.0:1780298645.112763:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609662208:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 204us (294us total) trans 4294969760 rc 0/0 00000100:00100000:2.0:1780298645.112769:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6415 00002000:00010000:2.0:1780298645.112771:0:19928:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4618229->18446744073709551615) 00002000:00100000:2.0:1780298645.112774:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25548aa00 with x1866777609662208 ext(4618229->18446744073709551615) 00002000:00010000:2.0:1780298645.112777:0:19928:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf25548aa00. 00002000:00010000:0.0:1780298645.112878:0:8363:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 612600 -> 4194304 00002000:00010000:0.0:1780298645.112888:0:8363:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298645.112891:0:8363:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298645.112893:0:8363:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298645.112896:0:8363:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:0.0:1780298645.112935:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609662336:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 201us (340us total) trans 4294969305 rc 0/0 00000100:00100000:0.0:1780298645.112943:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6416 00002000:00010000:0.0:1780298645.112947:0:8363:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (4194304->18446744073709551615) 00002000:00100000:0.0:1780298645.112959:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25548a300 with x1866777609662336 ext(4194304->18446744073709551615) 00002000:00010000:0.0:1780298645.112965:0:8363:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf25548a300. 00000100:00100000:1.0:1780298645.114253:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.114303:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609662464 00000100:00100000:3.0:1780298645.114310:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609662464 00002000:00100000:3.0:1780298645.114339:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (4612096->4616191) req@ffff9cf24519ea00 x1866777609662464/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298645.114348:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24519ea00 with x1866777609662464 ext(4612096->4616191) 00002000:00010000:3.0:1780298645.114353:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24519ea00 00000100:00100000:3.0:1780298645.114377:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6417 00000100:00100000:3.0:1780298645.114385:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609662464:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298645.114532:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609662464:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 148us (279us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1780298645.114540:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6417 00002000:00100000:3.0:1780298645.114547:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (4612096->4616191) req@ffff9cf24519ea00 x1866777609662464/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/4536 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298645.114556:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24519ea00 with x1866777609662464 ext(4612096->4616191) 00002000:00010000:3.0:1780298645.114559:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24519ea00 00000100:00100000:3.0:1780298645.115864:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.115945:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609662592 00000100:00100000:1.0:1780298645.115949:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609662592 00002000:00100000:1.0:1780298645.115966:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (4616192->4620287) req@ffff9cf255488380 x1866777609662592/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.115973:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255488380 with x1866777609662592 ext(4616192->4620287) 00002000:00010000:1.0:1780298645.115975:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255488380 00000100:00100000:1.0:1780298645.115994:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6418 00000100:00100000:1.0:1780298645.115999:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255488380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609662592:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298645.116064:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255488380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609662592:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 63us (201us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298645.116071:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6418 00002000:00100000:1.0:1780298645.116078:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (4616192->4620287) req@ffff9cf255488380 x1866777609662592/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/4536 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.116087:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255488380 with x1866777609662592 ext(4616192->4620287) 00002000:00010000:1.0:1780298645.116091:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255488380 00000100:00100000:2.0:1780298645.117393:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.117437:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609662720 00000100:00100000:0.0:1780298645.117443:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609662720 00002000:00100000:0.0:1780298645.117467:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (4612096->4618228) req@ffff9cf244810700 x1866777609662720/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 6624/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298645.117475:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244810700 with x1866777609662720 ext(4612096->4618228) 00002000:00010000:0.0:1780298645.117479:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf244810700 00000100:00100000:0.0:1780298645.117501:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6419 00000100:00100000:0.0:1780298645.117507:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244810700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609662720:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:0.0:1780298645.117986:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244810700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609662720:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 479us (595us total) trans 4294969761 rc 0/0 00000100:00100000:0.0:1780298645.117994:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6419 00002000:00100000:0.0:1780298645.118003:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (4612096->4618228) req@ffff9cf244810700 x1866777609662720/t4294969761(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 6624/448 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298645.118012:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244810700 with x1866777609662720 ext(4612096->4618228) 00002000:00010000:0.0:1780298645.118016:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf244810700 00000100:00100000:3.0:1780298645.118770:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.118810:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609662848 00000100:00100000:1.0:1780298645.118815:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609662848 00000100:00100000:1.0:1780298645.118845:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10196 00000100:00100000:1.0:1780298645.118851:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255489500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609662848:12345-192.168.203.25@tcp:16:fsx.0 00000001:00080000:2.0:1780298645.119641:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969761 is committed 00000001:00080000:2.0:1780298645.119650:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969759 is committed 00000100:00100000:1.0:1780298645.119739:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255489500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609662848:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 885us (968us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298645.119747:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10196 00000100:00100000:3.0:1780298645.120549:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.120613:0:9976:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609662976 00000100:00100000:2.0:1780298645.120618:0:9976:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609662976 00000100:00100000:2.0:1780298645.120652:0:9976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62063 00000100:00100000:2.0:1780298645.120657:0:9976:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255489c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+26:29360:x1866777609662976:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.120667:0:9976:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.120682:0:9976:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf350de3200/0xaf35132e63035a08 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 11 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d48e0 expref: -99 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.120749:0:9976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf350de3600/0xaf35132e63035a0f lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 12 type: IBT flags: 0x40000000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.120765:0:9976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf350de3600/0xaf35132e63035a0f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.120773:0:9976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf350de3600/0xaf35132e63035a0f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298645.120791:0:9976:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 8812533 00000004:00200000:2.0:1780298645.120794:0:9976:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298645.120806:0:9976:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf350de3600/0xaf35132e63035a0f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.120823:0:9976:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf350de3200/0xaf35132e63035a08 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 12 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48e0 expref: 28 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.120836:0:9976:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf350de3600/0xaf35132e63035a0f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 11 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48e0 expref: 27 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.120841:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298645.120844:0:9976:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298645.120845:0:9976:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3600, rc 0) 00000100:00100000:2.0:1780298645.120871:0:9976:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255489c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+27:29360:x1866777609662976:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 215us (323us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.120876:0:9976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62063 00000100:00100000:2.0:1780298645.121880:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.121923:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609663104 00000100:00100000:1.0:1780298645.121929:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609663104 00000100:00100000:1.0:1780298645.121967:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10197 00000100:00100000:1.0:1780298645.121980:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244812680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609663104:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.121989:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.122002:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63035a16 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d48e7 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.122030:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.122050:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433dc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605071232:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.122070:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609663232 00000100:00100000:0.0:1780298645.122077:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609663232 00000100:00100000:0.0:1780298645.122119:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10198 00000100:00100000:0.0:1780298645.122157:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609663232:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298645.122163:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a540 going to sleep for 11 seconds 00010000:00010000:0.0:1780298645.122168:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.122182:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d4d8e3d7/0xaf35132e63035a1d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d48ee expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.122222:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605071360:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.122283:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12000 going to sleep for 11 seconds 00002000:00010000:1.0:1780298645.123081:0:9128:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks: 424 -> 440 00000100:00100000:0.0:1780298645.123097:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605071360:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.123100:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433dc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605071232:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.123122:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d4d8e3d7/0xaf35132e63035a1d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48ee expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.123154:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63035a16 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48e7 expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.123156:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d4d8e3d7/0xaf35132e63035a1d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48ee expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.123179:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf000, rc 0) 00010000:00010000:1.0:1780298645.123203:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63035a16 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48e7 expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.123218:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609663232:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1065us (1228us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.123238:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10198 00010000:00010000:1.0:1780298645.123239:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279a00, rc 0) 00000100:00100000:1.0:1780298645.123287:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244812680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609663104:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1311us (1406us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.123295:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10197 00000100:00100000:2.0:1780298645.124253:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.124296:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609663360 00000100:00100000:1.0:1780298645.124301:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609663360 00000100:00100000:1.0:1780298645.124337:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10199 00000100:00100000:1.0:1780298645.124347:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609663360:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.124358:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.124368:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e63035a24 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d48f5 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.124388:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.124404:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433dc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605071488:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.124415:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609663488 00000100:00100000:2.0:1780298645.124420:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609663488 00000100:00100000:1.0:1780298645.124447:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a540 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.124467:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10200 00000100:00100000:2.0:1780298645.124483:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343964380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609663488:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.124493:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.124505:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d828f4f8/0xaf35132e63035a2b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d48fc expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.124535:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b42300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605071616:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.124572:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1600 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.125217:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b42300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605071616:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.125240:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d828f4f8/0xaf35132e63035a2b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48fc expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.125257:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d828f4f8/0xaf35132e63035a2b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48fc expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.125261:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433dc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605071488:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.125288:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2e00, rc 0) 00010000:00010000:1.0:1780298645.125296:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e63035a24 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48f5 expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.125312:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e63035a24 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48f5 expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.125319:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343964380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609663488:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 841us (928us total) trans 0 rc 301/301 00010000:00010000:1.0:1780298645.125327:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279800, rc 0) 00000100:00100000:2.0:1780298645.125332:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10200 00000100:00100000:1.0:1780298645.125355:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609663360:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1010us (1102us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.125364:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10199 00000100:00100000:3.0:1780298645.126265:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.126304:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609663616 00000100:00100000:2.0:1780298645.126309:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609663616 00000100:00100000:2.0:1780298645.126355:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6420 00000100:00100000:2.0:1780298645.126363:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343965f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609663616:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298645.126612:0:8364:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:2.0:1780298645.126617:0:8364:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298645.126619:0:8364:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298645.126622:0:8364:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298645.126624:0:8364:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 0 -> 16 00000100:00100000:2.0:1780298645.126657:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343965f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609663616:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 294us (392us total) trans 4294969306 rc 0/0 00000100:00100000:2.0:1780298645.126664:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6420 00000100:00100000:3.0:1780298645.127471:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.127504:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609663744 00000100:00100000:2.0:1780298645.127508:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609663744 00000100:00100000:2.0:1780298645.127538:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10201 00000100:00100000:2.0:1780298645.127549:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343965880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609663744:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.127557:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.127571:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000045ae2ef5/0xaf35132e63035a32 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4903 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.127585:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.127605:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b42300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605071744:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.127618:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609663872 00000100:00100000:1.0:1780298645.127624:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609663872 00000100:00100000:2.0:1780298645.127651:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.127660:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10202 00000100:00100000:1.0:1780298645.127670:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609663872:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.127678:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.127687:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6416acb/0xaf35132e63035a39 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d490a expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.127713:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433dc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605071872:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.127744:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a540 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.128310:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b42300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605071744:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.128338:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000045ae2ef5/0xaf35132e63035a32 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4903 expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.128353:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000045ae2ef5/0xaf35132e63035a32 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4903 expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.128362:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3200, rc 0) 00000100:00100000:1.0:1780298645.128392:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433dc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605071872:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.128398:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343965880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609663744:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 851us (925us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.128406:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10201 00010000:00010000:1.0:1780298645.128414:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a6416acb/0xaf35132e63035a39 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d490a expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.128428:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6416acb/0xaf35132e63035a39 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d490a expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.128438:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278c00, rc 0) 00000100:00100000:1.0:1780298645.128462:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609663872:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 795us (876us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.128468:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10202 00000100:00100000:3.0:1780298645.129318:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.129349:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.129351:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609664128 00000100:00100000:0.0:1780298645.129359:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609664128 00000100:00100000:0.0:1780298645.129409:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609664000 00000100:00100000:0.0:1780298645.129411:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609664000 00000100:00100000:2.0:1780298645.129416:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10203 00000100:00100000:1.0:1780298645.129430:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10204 00000100:00100000:2.0:1780298645.129431:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7be300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609664128:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.129441:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298645.129448:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2398:x1866777609664000:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.129452:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000075e99a4b/0xaf35132e63035a40 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4918 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.129458:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.129469:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63035a47 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4911 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.129487:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b42300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605072000:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.129507:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433dc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605072128:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.129528:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf242bb1600 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.129540:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a540 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.130324:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b42300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605072000:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.130344:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000075e99a4b/0xaf35132e63035a40 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4918 expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.130358:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000075e99a4b/0xaf35132e63035a40 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4918 expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.130364:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433dc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605072128:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.130367:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3800, rc 0) 00010000:00010000:1.0:1780298645.130378:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63035a47 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4911 expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.130391:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63035a47 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4911 expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.130407:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7be300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609664128:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 977us (1084us total) trans 0 rc 301/301 00010000:00010000:1.0:1780298645.130407:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278a00, rc 0) 00000100:00100000:2.0:1780298645.130414:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10203 00000100:00100000:1.0:1780298645.130425:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2398:x1866777609664000:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 985us (1081us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.130431:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10204 00000100:00100000:0.0:1780298645.131580:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.131626:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609664256 00000100:00100000:1.0:1780298645.131633:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609664256 00000100:00100000:1.0:1780298645.131676:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10205 00000100:00100000:1.0:1780298645.131684:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2397:x1866777609664256:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.131691:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.131700:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e63035a4e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d491f expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.131736:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433dc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605072256:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.131750:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.131805:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a540 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.131836:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609664384 00000100:00100000:1.0:1780298645.131841:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609664384 00000100:00100000:1.0:1780298645.131874:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10206 00000100:00100000:1.0:1780298645.131886:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609664384:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.131894:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.131903:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b25cdf87/0xaf35132e63035a55 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4926 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.131929:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433dea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605072384:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.131966:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837ae40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.132540:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433dc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605072256:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.132562:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e63035a4e lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d491f expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.132578:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e63035a4e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d491f expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.132589:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278400, rc 0) 00000100:00100000:1.0:1780298645.132628:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2397:x1866777609664256:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 945us (1052us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.132635:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10205 00000100:00100000:1.0:1780298645.132800:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433dea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605072384:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.132819:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b25cdf87/0xaf35132e63035a55 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4926 expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.132832:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b25cdf87/0xaf35132e63035a55 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4926 expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.132841:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278200, rc 0) 00000100:00100000:1.0:1780298645.132874:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609664384:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 993us (1127us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.132882:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10206 00000100:00100000:3.0:1780298645.133682:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.133729:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609664512 00000100:00100000:0.0:1780298645.133736:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609664512 00000100:00100000:0.0:1780298645.133784:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10207 00000100:00100000:0.0:1780298645.133797:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609664512:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.133814:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.133830:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007b6539a7/0xaf35132e63035a5c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d492d expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.133858:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63035a01 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48d9 expref: 10 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.133869:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(41 122) 00010000:00010000:0.0:1780298645.133874:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(42 122) 00010000:00010000:0.0:1780298645.133880:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007b6539a7/0xaf35132e63035a5c lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2031616->2035711] (req 2031616->2035711) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d492d expref: 11 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.133915:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63035a01 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48d9 expref: 10 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.133935:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63035a01 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48d9 expref: 10 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.133950:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605072512:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298645.134027:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12000 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.134667:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605072512:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298645.134693:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007b6539a7/0xaf35132e63035a5c lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2031616->2035711] (req 2031616->2035711) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d492d expref: 11 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.134714:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae800, rc 0) 00000100:00100000:0.0:1780298645.134751:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:29360:x1866777609664512:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 959us (1071us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.134760:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10207 00000100:00100000:1.0:1780298645.136096:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.136229:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609664640 00000100:00100000:1.0:1780298645.136235:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609664640 00010000:00010000:1.0:1780298645.136267:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63035a01 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48d9 expref: 11 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.136316:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15263 00000100:00100000:1.0:1780298645.136327:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609664640:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298645.136341:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298645.136357:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63035a01 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48d9 expref: 11 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.136373:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63035a01 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48d9 expref: 11 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.136395:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63035a01 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48d9 expref: 11 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.136405:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(42 122) 00010000:00010000:1.0:1780298645.136408:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(42 122) 00010000:00010000:1.0:1780298645.136416:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000007b6539a7/0xaf35132e63035a5c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2031616->35586047] (req 2031616->2035711) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d492d expref: 10 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.136426:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007b6539a7/0xaf35132e63035a5c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2031616->35586047] (req 2031616->2035711) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d492d expref: 10 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.136457:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000007b6539a7/0xaf35132e63035a5c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2031616->35586047] (req 2031616->2035711) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d492d expref: 10 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.136469:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433dc700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:20253:1866777605072640:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298645.136524:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837ae40 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.137644:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433dc700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:20253:1866777605072640:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298645.137657:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298645.137689:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433dea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609664640:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1365us (1598us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.137699:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15263 00000100:00100000:1.0:1780298645.137785:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.137811:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609664768 00000100:00100000:0.0:1780298645.137815:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609664768 00000100:00100000:0.0:1780298645.137865:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6421 00000100:00100000:0.0:1780298645.137876:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609664768:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298645.138080:0:8363:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:0.0:1780298645.138085:0:8363:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298645.138087:0:8363:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298645.138090:0:8363:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298645.138093:0:8363:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 16 -> 24 00000100:00100000:0.0:1780298645.138138:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2399:x1866777609664768:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 253us (341us total) trans 4294969307 rc 0/0 00000100:00100000:0.0:1780298645.138144:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6421 00000100:00100000:0.0:1780298645.139035:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.139068:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.139080:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609664896 00000100:00100000:1.0:1780298645.139084:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609664896 00000100:00100000:0.0:1780298645.139094:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609665024 00000100:00100000:0.0:1780298645.139100:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609665024 00000100:00100000:1.0:1780298645.139116:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10208 00000100:00100000:1.0:1780298645.139139:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609664896:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.139147:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.139159:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e63035a63 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4934 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.139163:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10209 00000100:00100000:0.0:1780298645.139177:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243845180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609665024:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.139185:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.139196:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2d588e1/0xaf35132e63035a6a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d493b expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.139202:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605072768:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.139242:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243846a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605072896:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.139255:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a540 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.139291:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12000 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.140036:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243846a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605072896:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.140043:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605072768:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.140071:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c2d588e1/0xaf35132e63035a6a lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d493b expref: 13 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.140071:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e63035a63 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4934 expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.140097:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e63035a63 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4934 expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.140101:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2d588e1/0xaf35132e63035a6a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d493b expref: 13 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.140113:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278800, rc 0) 00010000:00010000:2.0:1780298645.140119:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf800, rc 0) 00000100:00100000:1.0:1780298645.140153:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609664896:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1029us (1119us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.140158:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10208 00000100:00100000:2.0:1780298645.140191:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243845180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609665024:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1018us (1123us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.140200:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10209 00000100:00100000:3.0:1780298645.141258:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.141296:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609665152 00000100:00100000:0.0:1780298645.141301:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609665152 00000100:00100000:3.0:1780298645.141310:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.141340:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609665280 00000100:00100000:3.0:1780298645.141345:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609665280 00000100:00100000:0.0:1780298645.141349:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10210 00000100:00100000:0.0:1780298645.141360:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2398:x1866777609665152:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.141369:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.141380:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000441cddf1/0xaf35132e63035a71 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4942 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.141386:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10211 00000100:00100000:3.0:1780298645.141398:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609665280:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.141407:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.141418:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044cfb726/0xaf35132e63035a78 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4949 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.141423:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243846a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605073024:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.141460:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e56df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605073152:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.141478:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.141495:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.142263:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243846a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605073024:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.142292:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e56df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605073152:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.142295:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000441cddf1/0xaf35132e63035a71 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4942 expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.142312:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000441cddf1/0xaf35132e63035a71 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4942 expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.142317:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000044cfb726/0xaf35132e63035a78 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4949 expref: 13 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.142321:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf400, rc 0) 00010000:00010000:3.0:1780298645.142339:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044cfb726/0xaf35132e63035a78 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4949 expref: 13 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.142352:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157e00, rc 0) 00000100:00100000:1.0:1780298645.142354:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2398:x1866777609665152:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 995us (1097us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.142363:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10210 00000100:00100000:3.0:1780298645.142374:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609665280:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 978us (1064us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.142379:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10211 00000100:00100000:1.0:1780298645.143497:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.143533:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609665408 00000100:00100000:3.0:1780298645.143539:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609665408 00000100:00100000:3.0:1780298645.143586:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6422 00000100:00100000:3.0:1780298645.143601:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609665408:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:3.0:1780298645.143776:0:19950:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:3.0:1780298645.143780:0:19950:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:3.0:1780298645.143783:0:19950:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:3.0:1780298645.143786:0:19950:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:3.0:1780298645.143789:0:19950:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 24 -> 24 00000100:00100000:3.0:1780298645.143824:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609665408:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 225us (326us total) trans 4294969308 rc 0/0 00000100:00100000:3.0:1780298645.143841:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6422 00000100:00100000:0.0:1780298645.144758:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.144797:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609665536 00000100:00100000:1.0:1780298645.144802:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609665536 00000100:00100000:1.0:1780298645.144840:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10212 00000100:00100000:1.0:1780298645.144849:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243846a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2397:x1866777609665536:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.144859:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298645.144862:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:1.0:1780298645.144870:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e63035a7f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4950 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.144881:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609665664 00000100:00100000:0.0:1780298645.144884:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609665664 00000100:00100000:1.0:1780298645.144909:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605073280:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.144911:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10213 00000100:00100000:0.0:1780298645.144919:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609665664:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.144925:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.144933:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002ee949fd/0xaf35132e63035a86 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4957 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.144954:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a540 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.144959:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f737800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605073408:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.144985:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12000 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.145702:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605073280:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.145719:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e63035a7f lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4950 expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.145735:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e63035a7f lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4950 expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.145744:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278400, rc 0) 00000100:00100000:1.0:1780298645.145774:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243846a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2397:x1866777609665536:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 927us (1016us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.145782:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10212 00000100:00100000:1.0:1780298645.145899:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f737800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605073408:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.145915:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002ee949fd/0xaf35132e63035a86 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4957 expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.145927:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002ee949fd/0xaf35132e63035a86 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4957 expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.145936:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bafc00, rc 0) 00000100:00100000:1.0:1780298645.145961:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609665664:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1045us (1101us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.145971:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10213 00000100:00100000:3.0:1780298645.146878:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.146908:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609665792 00000100:00100000:1.0:1780298645.146912:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609665792 00000100:00100000:1.0:1780298645.146942:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10214 00000100:00100000:1.0:1780298645.146957:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2398:x1866777609665792:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.146965:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.146974:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e63035a8d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d495e expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.147006:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605073536:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.147061:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.147094:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609665920 00000100:00100000:0.0:1780298645.147099:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609665920 00000100:00100000:0.0:1780298645.147144:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10215 00000100:00100000:0.0:1780298645.147154:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f734e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609665920:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.147163:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.147173:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c37b15a2/0xaf35132e63035a94 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4965 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.147197:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a540 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.147206:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605073664:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.147263:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12000 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.147724:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605073536:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.147748:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e63035a8d lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d495e expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.147766:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e63035a8d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d495e expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.147776:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278800, rc 0) 00000100:00100000:1.0:1780298645.147807:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2398:x1866777609665792:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 858us (930us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.147815:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10214 00000100:00100000:1.0:1780298645.147940:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605073664:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.147959:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c37b15a2/0xaf35132e63035a94 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4965 expref: 14 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.147974:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c37b15a2/0xaf35132e63035a94 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4965 expref: 14 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.147986:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baee00, rc 0) 00000100:00100000:1.0:1780298645.148019:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f734e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609665920:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 867us (960us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.148027:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10215 00000100:00100000:3.0:1780298645.148935:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.148977:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609666048 00000100:00100000:1.0:1780298645.148982:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609666048 00000100:00100000:1.0:1780298645.149019:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10216 00000100:00100000:1.0:1780298645.149028:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609666048:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.149038:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.149047:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63035a9b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d496c expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.149077:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605073792:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.149121:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a540 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.149767:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605073792:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.149789:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63035a9b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d496c expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.149804:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63035a9b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d496c expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.149816:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279200, rc 0) 00000100:00100000:1.0:1780298645.149852:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609666048:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 824us (916us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.149859:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10216 00000100:00100000:0.0:1780298645.150843:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.150882:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609666176 00000100:00100000:1.0:1780298645.150885:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609666176 00002000:00100000:1.0:1780298645.150913:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (2682880->2686975) req@ffff9cf37f734a80 x1866777609666176/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.150920:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f734a80 with x1866777609666176 ext(2682880->2686975) 00002000:00010000:1.0:1780298645.150925:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f734a80 00000100:00100000:1.0:1780298645.150948:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6423 00000100:00100000:1.0:1780298645.150957:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f734a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609666176:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298645.151114:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f734a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609666176:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 158us (272us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298645.151142:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6423 00002000:00100000:1.0:1780298645.151148:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (2682880->2686975) req@ffff9cf37f734a80 x1866777609666176/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/4536 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.151155:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f734a80 with x1866777609666176 ext(2682880->2686975) 00002000:00010000:1.0:1780298645.151157:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f734a80 00000100:00100000:3.0:1780298645.152339:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.152383:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609666304 00000100:00100000:1.0:1780298645.152388:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609666304 00002000:00100000:1.0:1780298645.152422:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (2686976->2691071) req@ffff9cf36e56dc00 x1866777609666304/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.152432:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e56dc00 with x1866777609666304 ext(2686976->2691071) 00002000:00010000:1.0:1780298645.152436:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf36e56dc00 00000100:00100000:1.0:1780298645.152466:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6424 00000100:00100000:1.0:1780298645.152475:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609666304:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298645.152581:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609666304:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 107us (243us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298645.152589:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6424 00002000:00100000:1.0:1780298645.152597:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (2686976->2691071) req@ffff9cf36e56dc00 x1866777609666304/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/4536 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.152606:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e56dc00 with x1866777609666304 ext(2686976->2691071) 00002000:00010000:1.0:1780298645.152609:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf36e56dc00 00000100:00100000:3.0:1780298645.153798:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.153840:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609666432 00000100:00100000:0.0:1780298645.153846:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609666432 00002000:00100000:0.0:1780298645.153872:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (2682880->2691071) req@ffff9cf36e56f800 x1866777609666432/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 8680/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298645.153881:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e56f800 with x1866777609666432 ext(2682880->2691071) 00002000:00010000:0.0:1780298645.153885:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf36e56f800 00000100:00100000:0.0:1780298645.153911:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6425 00000100:00100000:0.0:1780298645.153920:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609666432:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:0.0:1780298645.155289:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609666432:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 1370us (1490us total) trans 4294969762 rc 0/0 00000100:00100000:0.0:1780298645.155297:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6425 00002000:00100000:0.0:1780298645.155304:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (2682880->2691071) req@ffff9cf36e56f800 x1866777609666432/t4294969762(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 8680/448 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298645.155312:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e56f800 with x1866777609666432 ext(2682880->2691071) 00002000:00010000:0.0:1780298645.155315:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf36e56f800 00000100:00100000:0.0:1780298645.156149:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.156187:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609666560 00000100:00100000:2.0:1780298645.156193:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609666560 00000100:00100000:2.0:1780298645.156229:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10217 00000100:00100000:2.0:1780298645.156239:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f736a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609666560:12345-192.168.203.25@tcp:16:fsx.0 00000001:00080000:0.0:1780298645.157104:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969762 is committed 00000100:00100000:3.0:1780298645.157773:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f736a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609666560:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 1535us (1623us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298645.157779:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10217 00000100:00100000:0.0:1780298645.158721:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.158760:0:17578:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609666688 00000100:00100000:1.0:1780298645.158769:0:17578:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609666688 00000100:00100000:1.0:1780298645.158839:0:17578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62064 00000100:00100000:1.0:1780298645.158852:0:17578:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f736300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+34:29360:x1866777609666688:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.158873:0:17578:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.158892:0:17578:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279200/0xaf35132e63035aa2 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 12 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4973 expref: -99 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.158978:0:17578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e63035aa9 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.158992:0:17578:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e63035aa9 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.159006:0:17578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e63035aa9 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1780298645.159035:0:17578:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 8812533 00000004:00200000:1.0:1780298645.159040:0:17578:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1780298645.159062:0:17578:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e63035aa9 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.159095:0:17578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279200/0xaf35132e63035aa2 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 13 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4973 expref: 36 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.159118:0:17578:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e63035aa9 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4973 expref: 35 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.159149:0:17578:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298645.159152:0:17578:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1780298645.159155:0:17578:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278800, rc 0) 00000100:00100000:1.0:1780298645.159222:0:17578:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f736300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+35:29360:x1866777609666688:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 371us (501us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298645.159229:0:17578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62064 00000100:00100000:0.0:1780298645.160264:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.160306:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609666816 00000100:00100000:1.0:1780298645.160312:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609666816 00000100:00100000:1.0:1780298645.160359:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10218 00000100:00100000:1.0:1780298645.160369:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f737800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609666816:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.160377:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.160387:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e63035ab0 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d497a expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.160430:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605073920:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.160523:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a840 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.161475:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605073920:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.161518:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e63035ab0 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d497a expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.161544:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e63035ab0 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d497a expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.161561:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278400, rc 0) 00000100:00100000:0.0:1780298645.161606:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f737800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609666816:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1237us (1345us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.161615:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10218 00000100:00100000:0.0:1780298645.162657:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.162694:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609666944 00000100:00100000:0.0:1780298645.162701:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609666944 00000100:00100000:0.0:1780298645.162758:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10219 00000100:00100000:0.0:1780298645.162776:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a82a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609666944:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.162793:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.162811:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e63035ab7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4981 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.162865:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241a82d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605074048:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.162947:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12000 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.163812:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241a82d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605074048:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.163852:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e63035ab7 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4981 expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.163874:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e63035ab7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4981 expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.163887:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baea00, rc 0) 00000100:00100000:1.0:1780298645.163933:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a82a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609666944:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1166us (1284us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.163941:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10219 00000100:00100000:0.0:1780298645.164915:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.164953:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609667072 00000100:00100000:1.0:1780298645.164958:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609667072 00000100:00100000:1.0:1780298645.164993:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10220 00000100:00100000:1.0:1780298645.165005:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a82300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609667072:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.165018:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.165030:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63035abe lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4988 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.165073:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605074176:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.165168:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837af00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.165956:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605074176:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.165974:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63035abe lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4988 expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.165987:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63035abe lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4988 expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.165996:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279200, rc 0) 00000100:00100000:1.0:1780298645.166024:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a82300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609667072:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1025us (1112us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.166029:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10220 00000100:00100000:1.0:1780298645.166980:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.167016:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609667200 00000100:00100000:3.0:1780298645.167023:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609667200 00000100:00100000:3.0:1780298645.167058:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10221 00000100:00100000:3.0:1780298645.167070:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609667200:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.167083:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.167099:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c278703/0xaf35132e63035ac5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d498f expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.167175:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255491f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605074304:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.167226:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.167952:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255491f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605074304:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.167977:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009c278703/0xaf35132e63035ac5 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d498f expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.167993:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c278703/0xaf35132e63035ac5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d498f expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.168003:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156600, rc 0) 00000100:00100000:3.0:1780298645.168035:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609667200:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 971us (1055us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.168043:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10221 00000100:00100000:0.0:1780298645.169061:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.169112:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609667328 00000100:00100000:1.0:1780298645.169120:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609667328 00000100:00100000:1.0:1780298645.169591:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10222 00000100:00100000:1.0:1780298645.169620:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a82d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:29360:x1866777609667328:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.169634:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.169650:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e63035acc lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4996 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.169677:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e630359fa lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48d2 expref: 6 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.169689:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(42 169) 00010000:00010000:1.0:1780298645.169693:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(43 169) 00010000:00010000:1.0:1780298645.169699:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e63035acc lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [217088->225279] (req 217088->225279) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4996 expref: 9 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.169730:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e630359fa lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48d2 expref: 6 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.169747:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e630359fa lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48d2 expref: 6 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.169757:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605074432:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298645.169840:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837af00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.170546:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605074432:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298645.170563:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e63035acc lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [217088->225279] (req 217088->225279) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4996 expref: 9 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.170573:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279e00, rc 0) 00000100:00100000:0.0:1780298645.170609:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a82d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609667328:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1009us (1549us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.170617:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10222 00000100:00100000:0.0:1780298645.170845:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.170886:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609667456 00000100:00100000:1.0:1780298645.170892:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609667456 00010000:00010000:1.0:1780298645.170908:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e630359fa lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48d2 expref: 7 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.170933:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15264 00000100:00100000:1.0:1780298645.170940:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609667456:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298645.170952:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:1.0:1780298645.170966:0:6463:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 440 -> 456 00010000:00010000:1.0:1780298645.170973:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e630359fa lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48d2 expref: 7 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.170990:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e630359fa lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48d2 expref: 7 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.171010:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000127fe2a2/0xaf35132e630359fa lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48d2 expref: 7 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.171022:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(43 169) 00010000:00010000:1.0:1780298645.171026:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(43 169) 00010000:00010000:1.0:1780298645.171033:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e63035acc lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [217088->33771519] (req 217088->225279) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4996 expref: 8 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.171041:0:6463:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e63035acc lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [217088->33771519] (req 217088->225279) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4996 expref: 8 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.171063:0:6463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e63035acc lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [217088->33771519] (req 217088->225279) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4996 expref: 8 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.171074:0:6463:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f9c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6463:1866777605074560:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298645.171113:0:6463:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837af00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.173630:0:6463:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f9c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6463:1866777605074560:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298645.173647:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298645.173686:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609667456:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 2747us (2842us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.173692:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15264 00000100:00100000:0.0:1780298645.173973:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.174030:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609667584 00000100:00100000:3.0:1780298645.174039:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609667584 00002000:00100000:3.0:1780298645.174089:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (217088->221183) req@ffff9cf34372fb80 x1866777609667584/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298645.174098:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf34372fb80 with x1866777609667584 ext(217088->221183) 00002000:00010000:3.0:1780298645.174103:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf34372fb80 00000100:00100000:3.0:1780298645.174168:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6426 00000100:00100000:3.0:1780298645.174183:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609667584:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298645.174366:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609667584:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 188us (394us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1780298645.174375:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6426 00002000:00100000:3.0:1780298645.174384:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (217088->221183) req@ffff9cf34372fb80 x1866777609667584/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/4536 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298645.174391:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf34372fb80 with x1866777609667584 ext(217088->221183) 00002000:00010000:3.0:1780298645.174394:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf34372fb80 00000100:00100000:3.0:1780298645.175374:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.175427:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609667712 00000100:00100000:3.0:1780298645.175432:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609667712 00002000:00100000:3.0:1780298645.175464:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (221184->225279) req@ffff9cf255491f80 x1866777609667712/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298645.175475:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255491f80 with x1866777609667712 ext(221184->225279) 00002000:00010000:3.0:1780298645.175480:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255491f80 00000100:00100000:3.0:1780298645.175531:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6427 00000100:00100000:3.0:1780298645.175545:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255491f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609667712:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298645.175661:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255491f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609667712:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 119us (291us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1780298645.175669:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6427 00002000:00100000:3.0:1780298645.175674:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (221184->225279) req@ffff9cf255491f80 x1866777609667712/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/4536 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298645.175681:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255491f80 with x1866777609667712 ext(221184->225279) 00002000:00010000:3.0:1780298645.175685:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255491f80 00000100:00100000:3.0:1780298645.176735:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.176773:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.176818:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609667968 00000100:00100000:0.0:1780298645.176818:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609667840 00000100:00100000:3.0:1780298645.176823:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609667968 00000100:00100000:0.0:1780298645.176828:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609667840 00000100:00100000:0.0:1780298645.176894:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10223 00000100:00100000:1.0:1780298645.176904:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10224 00000100:00100000:0.0:1780298645.176908:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e4f3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609667968:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298645.176921:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255490380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609667840:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.176924:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.176931:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.176941:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2d588e1/0xaf35132e63035ad3 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d49a4 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.176943:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63035ada lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d499d expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.177011:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fa300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605074816:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.177012:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605074688:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.177089:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf34837a840 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.177089:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12000 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.178066:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fa300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605074816:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.178086:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605074688:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.178108:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63035ada lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d499d expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.178120:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c2d588e1/0xaf35132e63035ad3 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d49a4 expref: 14 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.178175:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63035ada lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d499d expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.178182:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2d588e1/0xaf35132e63035ad3 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d49a4 expref: 14 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.178194:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279200, rc 0) 00010000:00010000:3.0:1780298645.178202:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf800, rc 0) 00000100:00100000:2.0:1780298645.178242:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255490380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609667840:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1325us (1505us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.178249:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e4f3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609667968:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1341us (1475us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.178281:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10224 00000100:00100000:3.0:1780298645.178297:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10223 00000100:00100000:3.0:1780298645.179364:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.179412:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609668096 00000100:00100000:2.0:1780298645.179418:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609668096 00000100:00100000:3.0:1780298645.179425:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.179469:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609668224 00000100:00100000:3.0:1780298645.179475:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10225 00000100:00100000:2.0:1780298645.179477:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609668224 00000100:00100000:3.0:1780298645.179497:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255490380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609668096:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.179511:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.179530:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000044cfb726/0xaf35132e63035ae1 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d49ab expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.179544:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10226 00000100:00100000:2.0:1780298645.179557:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255490e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609668224:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.179565:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298645.179577:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255492a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605074944:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.179578:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e63035ae8 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d49b2 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.179617:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b42300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605075072:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.179692:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856180 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.179694:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.180607:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b42300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605075072:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.180641:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e63035ae8 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d49b2 expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.180663:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045ae2ef5/0xaf35132e63035ae8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d49b2 expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.180711:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255492a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605074944:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.180722:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3200, rc 0) 00010000:00010000:2.0:1780298645.180749:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000044cfb726/0xaf35132e63035ae1 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d49ab expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.180768:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255490e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609668224:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1211us (1346us total) trans 0 rc 301/301 00010000:00010000:2.0:1780298645.180769:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000044cfb726/0xaf35132e63035ae1 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d49ab expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.180787:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10226 00010000:00010000:2.0:1780298645.180792:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157e00, rc 0) 00000100:00100000:2.0:1780298645.180826:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255490380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609668096:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1337us (1463us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.180837:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10225 00000100:00100000:2.0:1780298645.181990:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.182040:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609668352 00000100:00100000:3.0:1780298645.182047:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609668352 00000100:00100000:3.0:1780298645.182178:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10227 00000100:00100000:3.0:1780298645.182200:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:29360:x1866777609668352:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.182250:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.182280:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d828f4f8/0xaf35132e63035aef lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d49b9 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.182307:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e63035acc lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [217088->33771519] (req 217088->225279) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4996 expref: 8 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.182321:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(43 169) 00010000:00010000:2.0:1780298645.182326:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(44 169) 00010000:00010000:2.0:1780298645.182332:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d828f4f8/0xaf35132e63035aef lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2387968->2400255] (req 2387968->2400255) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d49b9 expref: 7 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.182367:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e63035acc lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [217088->33771519] (req 217088->225279) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4996 expref: 8 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.182391:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e63035acc lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [217088->33771519] (req 217088->225279) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4996 expref: 8 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.182407:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605075200:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298645.182521:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.183177:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605075200:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298645.183220:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d828f4f8/0xaf35132e63035aef lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2387968->2400255] (req 2387968->2400255) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d49b9 expref: 7 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.183235:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2e00, rc 0) 00000100:00100000:2.0:1780298645.183308:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609668352:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1119us (1324us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.183315:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10227 00000100:00100000:3.0:1780298645.183421:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.183464:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609668480 00000100:00100000:1.0:1780298645.183475:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609668480 00002000:00100000:1.0:1780298645.183520:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (217088->225279) req@ffff9cf255492a00 x1866777609668480/t0(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 8680/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.183529:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255492a00 with x1866777609668480 ext(217088->225279) 00010000:00010000:1.0:1780298645.183537:0:19949:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e63035acc lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [217088->33771519] (req 217088->225279) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4996 expref: 9 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.183554:0:19949:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e63035acc lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [217088->33771519] (req 217088->225279) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4996 expref: 9 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1780298645.183563:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf255492a00 00000100:00100000:1.0:1780298645.183596:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 213 00000100:00100000:1.0:1780298645.183607:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255492a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609668480:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:2.0:1780298645.184810:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255492a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609668480:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 1199us (1385us total) trans 4294969763 rc 0/0 00000100:00100000:2.0:1780298645.184823:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 213 00002000:00100000:2.0:1780298645.184836:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (217088->225279) req@ffff9cf255492a00 x1866777609668480/t4294969763(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 8680/448 e 0 to 0 dl 1780298656 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.184845:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255492a00 with x1866777609668480 ext(217088->225279) 00010000:00010000:2.0:1780298645.184852:0:19949:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e63035acc lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [217088->33771519] (req 217088->225279) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4996 expref: 11 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.184885:0:19949:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e63035acc lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [217088->33771519] (req 217088->225279) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4996 expref: 11 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1780298645.184892:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf255492a00 00000100:00100000:2.0:1780298645.185796:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.185854:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609668608 00000100:00100000:3.0:1780298645.185860:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609668608 00010000:00010000:3.0:1780298645.185890:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e63035acc lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [217088->33771519] (req 217088->225279) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4996 expref: 11 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.185948:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15265 00000100:00100000:3.0:1780298645.185966:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3487c7850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609668608:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298645.185981:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:3.0:1780298645.185995:0:6461:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 456 -> 472 00010000:00010000:3.0:1780298645.186003:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e63035acc lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [217088->33771519] (req 217088->225279) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4996 expref: 11 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.186018:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e63035acc lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [217088->33771519] (req 217088->225279) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4996 expref: 11 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.186039:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e63035acc lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [217088->33771519] (req 217088->225279) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4996 expref: 11 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.186058:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(44 169) 00010000:00010000:3.0:1780298645.186062:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(44 169) 00010000:00010000:3.0:1780298645.186072:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000d828f4f8/0xaf35132e63035aef lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2387968->35942399] (req 2387968->2400255) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d49b9 expref: 6 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.186083:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d828f4f8/0xaf35132e63035aef lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2387968->35942399] (req 2387968->2400255) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d49b9 expref: 6 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.186117:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000d828f4f8/0xaf35132e63035aef lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2387968->35942399] (req 2387968->2400255) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d49b9 expref: 6 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.186163:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2556ab800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777605075328:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298645.186226:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856180 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.187197:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2556ab800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777605075328:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298645.187234:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:2.0:1780298645.187243:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298645.187430:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3487c7850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609668608:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1467us (1633us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.187447:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15265 00000100:00100000:1.0:1780298645.187674:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609668736 00000100:00100000:1.0:1780298645.187683:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609668736 00000100:00100000:1.0:1780298645.187742:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6428 00000100:00100000:1.0:1780298645.187752:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609668736:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298645.187935:0:19928:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4618229 -> 4618229 00002000:00010000:1.0:1780298645.187939:0:19928:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298645.187942:0:19928:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298645.187946:0:19928:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298645.187947:0:19928:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 472 -> 496 00000100:00100000:1.0:1780298645.187995:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609668736:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 247us (769us total) trans 4294969764 rc 0/0 00000100:00100000:1.0:1780298645.188001:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6428 00000100:00100000:3.0:1780298645.188987:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.189038:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609668864 00000100:00100000:2.0:1780298645.189047:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609668864 00000100:00100000:2.0:1780298645.189211:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10228 00000100:00100000:2.0:1780298645.189231:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556ab800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609668864:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.189247:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.189264:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000052b89172/0xaf35132e63035af6 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d49c0 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.189298:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.189343:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605075456:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.189345:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609668992 00000100:00100000:0.0:1780298645.189355:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609668992 00000100:00100000:0.0:1780298645.189416:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10229 00000100:00100000:0.0:1780298645.189436:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609668992:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.189452:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.189473:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000075508ee2/0xaf35132e63035afd lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d49c7 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.189488:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.189531:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605075584:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.189587:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12d80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.190121:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605075456:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.190204:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000052b89172/0xaf35132e63035af6 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d49c0 expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.190239:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000052b89172/0xaf35132e63035af6 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d49c0 expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.190261:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2800, rc 0) 00000100:00100000:2.0:1780298645.190319:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556ab800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609668864:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1094us (1331us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.190331:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10228 00000100:00100000:3.0:1780298645.190365:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605075584:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.190428:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000075508ee2/0xaf35132e63035afd lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d49c7 expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.190447:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000075508ee2/0xaf35132e63035afd lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d49c7 expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.190466:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae200, rc 0) 00000100:00100000:3.0:1780298645.190498:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609668992:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1071us (1203us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.190509:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10229 00000100:00100000:3.0:1780298645.191663:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.191705:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609669120 00000100:00100000:2.0:1780298645.191711:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609669120 00000100:00100000:3.0:1780298645.191752:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.191774:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609669248 00000100:00100000:2.0:1780298645.191781:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609669248 00000100:00100000:2.0:1780298645.191835:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10230 00000100:00100000:2.0:1780298645.191851:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609669120:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298645.191857:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10231 00010000:00010000:2.0:1780298645.191862:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.191878:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000024bbcf41/0xaf35132e63035b04 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d49ce expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.191882:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609669248:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.191899:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.191913:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63035b0b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d49d5 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.191927:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605075712:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.191973:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548bbb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605075840:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.192018:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.192019:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.193265:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605075712:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.193288:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548bbb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605075840:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.193326:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63035b0b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d49d5 expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.193330:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000024bbcf41/0xaf35132e63035b04 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d49ce expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.193372:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63035b0b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d49d5 expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.193379:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000024bbcf41/0xaf35132e63035b04 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d49ce expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.193388:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157c00, rc 0) 00010000:00010000:2.0:1780298645.193399:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2c00, rc 0) 00000100:00100000:3.0:1780298645.193436:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609669248:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1565us (1682us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.193445:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609669120:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1597us (1785us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.193452:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10231 00000100:00100000:2.0:1780298645.193470:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10230 00000100:00100000:2.0:1780298645.194688:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.194720:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609669376 00000100:00100000:3.0:1780298645.194725:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609669376 00000100:00100000:2.0:1780298645.194761:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.194768:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10232 00000100:00100000:3.0:1780298645.194778:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609669376:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.194787:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298645.194798:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609669504 00000100:00100000:1.0:1780298645.194805:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609669504 00010000:00010000:3.0:1780298645.194842:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009c278703/0xaf35132e63035b12 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d49dc expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.194853:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10233 00000100:00100000:1.0:1780298645.194871:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609669504:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.194884:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298645.194898:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548bbb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605075968:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.194908:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63035b19 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d49e3 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.194965:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605076096:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.194991:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.195029:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4d80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.195864:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548bbb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605075968:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.195900:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009c278703/0xaf35132e63035b12 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d49dc expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.195917:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009c278703/0xaf35132e63035b12 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d49dc expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.195933:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156600, rc 0) 00000100:00100000:1.0:1780298645.195952:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605076096:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.195974:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609669376:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1193us (1285us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.195994:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10232 00010000:00010000:1.0:1780298645.195994:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63035b19 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d49e3 expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.196032:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63035b19 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d49e3 expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.196049:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278a00, rc 0) 00000100:00100000:1.0:1780298645.196114:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609669504:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1249us (1352us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.196164:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10233 00000100:00100000:3.0:1780298645.196939:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.196989:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609669632 00000100:00100000:1.0:1780298645.196996:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609669632 00000100:00100000:3.0:1780298645.197008:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.197048:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609669760 00000100:00100000:1.0:1780298645.197053:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609669760 00000100:00100000:2.0:1780298645.197083:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10234 00000100:00100000:1.0:1780298645.197098:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10235 00000100:00100000:2.0:1780298645.197106:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609669632:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.197117:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298645.197119:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609669760:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.197162:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.197172:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000075e99a4b/0xaf35132e63035b20 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d49ea expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.197177:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b25cdf87/0xaf35132e63035b27 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d49f1 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.197266:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605076224:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.197286:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605076352:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.197366:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.197367:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4d80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.198117:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605076224:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.198149:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605076352:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.198188:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000075e99a4b/0xaf35132e63035b20 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d49ea expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.198193:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b25cdf87/0xaf35132e63035b27 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d49f1 expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.198264:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000075e99a4b/0xaf35132e63035b20 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d49ea expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.198265:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b25cdf87/0xaf35132e63035b27 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d49f1 expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.198308:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3800, rc 0) 00010000:00010000:1.0:1780298645.198315:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278200, rc 0) 00000100:00100000:1.0:1780298645.198362:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609669760:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1246us (1359us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.198378:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609669632:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1277us (1432us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.198379:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10235 00000100:00100000:2.0:1780298645.198397:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10234 00000100:00100000:3.0:1780298645.199357:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.199411:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609669888 00000100:00100000:1.0:1780298645.199417:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609669888 00000100:00100000:1.0:1780298645.199468:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10236 00000100:00100000:1.0:1780298645.199481:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:29360:x1866777609669888:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.199492:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.199505:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63035b2e lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d49f8 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.199527:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(44 169) 00010000:00010000:1.0:1780298645.199532:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(44 169) 00010000:00010000:1.0:1780298645.199542:0:20252:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63035b2e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [81920->2383871] (req 81920->90111) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d49f8 expref: 9 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.199557:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63035b2e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [81920->2383871] (req 81920->90111) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d49f8 expref: 9 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.199567:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279c00, rc 0) 00000100:00100000:1.0:1780298645.199608:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:29360:x1866777609669888:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 131us (254us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298645.199616:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10236 00000100:00100000:3.0:1780298645.200245:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.200304:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609670016 00000100:00100000:2.0:1780298645.200311:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609670016 00000100:00100000:2.0:1780298645.200365:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6429 00000100:00100000:2.0:1780298645.200379:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609670016:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298645.200563:0:8364:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4618229 -> 4618229 00002000:00010000:2.0:1780298645.200568:0:8364:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298645.200571:0:8364:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298645.200573:0:8364:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298645.200577:0:8364:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 496 -> 496 00000100:00100000:2.0:1780298645.200628:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609670016:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 252us (383us total) trans 4294969765 rc 0/0 00000100:00100000:2.0:1780298645.200646:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6429 00000100:00100000:2.0:1780298645.201566:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.201621:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609670144 00000100:00100000:1.0:1780298645.201626:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609670144 00000100:00100000:1.0:1780298645.201672:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10237 00000100:00100000:1.0:1780298645.201687:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609670144:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.201699:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.201714:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004fdbd682/0xaf35132e63035b35 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d49ff expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.201777:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605076480:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.201829:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.201863:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4d80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.201909:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609670272 00000100:00100000:1.0:1780298645.201914:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609670272 00000100:00100000:1.0:1780298645.201969:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10238 00000100:00100000:1.0:1780298645.201986:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25488aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609670272:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.201996:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.202011:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e63035b3c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4a06 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.202054:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481faa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605076608:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.202100:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.202600:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605076480:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.202639:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004fdbd682/0xaf35132e63035b35 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d49ff expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.202662:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004fdbd682/0xaf35132e63035b35 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d49ff expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.202679:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278e00, rc 0) 00000100:00100000:1.0:1780298645.202722:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609670144:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1040us (1155us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.202731:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10237 00000100:00100000:1.0:1780298645.202799:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481faa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605076608:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.202836:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e63035b3c lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a06 expref: 14 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.202850:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e63035b3c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a06 expref: 14 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.202864:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279a00, rc 0) 00000100:00100000:1.0:1780298645.202905:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25488aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609670272:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 924us (1075us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.202918:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10238 00000100:00100000:3.0:1780298645.203899:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.203938:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609670400 00000100:00100000:3.0:1780298645.203939:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.203944:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609670400 00000100:00100000:2.0:1780298645.203985:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609670528 00000100:00100000:2.0:1780298645.203987:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609670528 00000100:00100000:3.0:1780298645.204006:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10239 00000100:00100000:3.0:1780298645.204022:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609670400:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.204035:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298645.204035:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10240 00010000:00010000:3.0:1780298645.204052:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e8b6afab/0xaf35132e63035b43 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4a0d expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.204053:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609670528:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.204068:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.204081:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017dab243/0xaf35132e63035b4a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4a14 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.204107:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605076736:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.204158:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481faa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605076864:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.204220:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.204220:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4a80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.204953:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481faa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605076864:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.204966:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605076736:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.204992:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000017dab243/0xaf35132e63035b4a lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a14 expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.205016:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e8b6afab/0xaf35132e63035b43 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a0d expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.205018:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017dab243/0xaf35132e63035b4a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a14 expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.205052:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279800, rc 0) 00010000:00010000:3.0:1780298645.205058:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e8b6afab/0xaf35132e63035b43 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a0d expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.205083:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157800, rc 0) 00000100:00100000:1.0:1780298645.205105:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609670528:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1052us (1165us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.205117:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10240 00000100:00100000:3.0:1780298645.205179:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609670400:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1103us (1222us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.205194:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10239 00000100:00100000:2.0:1780298645.206192:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.206228:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609670656 00000100:00100000:1.0:1780298645.206235:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609670656 00000100:00100000:2.0:1780298645.206240:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.206286:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609670784 00000100:00100000:3.0:1780298645.206291:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609670784 00000100:00100000:1.0:1780298645.206317:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10241 00000100:00100000:2.0:1780298645.206335:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10242 00000100:00100000:1.0:1780298645.206335:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254888000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609670656:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.206348:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298645.206352:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25488b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609670784:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.206363:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.206364:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e63035b51 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4a1b expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.206376:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000052b89172/0xaf35132e63035b58 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4a22 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.206414:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481faa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605076992:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.206424:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254888a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605077120:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.206496:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf344ff4a80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.206504:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.207343:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254888a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605077120:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.207373:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000052b89172/0xaf35132e63035b58 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a22 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.207394:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000052b89172/0xaf35132e63035b58 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a22 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.207411:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2800, rc 0) 00000100:00100000:0.0:1780298645.207430:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481faa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605076992:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.207463:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25488b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609670784:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1114us (1226us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.207472:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10242 00010000:00010000:0.0:1780298645.207483:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e63035b51 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a1b expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.207512:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e63035b51 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a1b expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.207538:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279e00, rc 0) 00000100:00100000:0.0:1780298645.207606:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254888000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609670656:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1279us (1414us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.207617:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10241 00000100:00100000:2.0:1780298645.209304:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.209348:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609670912 00000100:00100000:3.0:1780298645.209356:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609670912 00002000:00100000:2.0:1780298645.209426:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (1929216->1994751) req@ffff9cf254889c00 x1866777609670912/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.209437:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf254889c00 with x1866777609670912 ext(1929216->1994751) 00002000:00010000:2.0:1780298645.209444:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf254889c00 00000100:00100000:2.0:1780298645.209487:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6430 00000100:00100000:2.0:1780298645.209504:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254889c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609670912:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298645.211012:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254889c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609670912:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 1510us (1705us total) trans 0 rc 65536/65536 00000100:00100000:0.0:1780298645.211022:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6430 00002000:00100000:0.0:1780298645.211032:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (1929216->1994751) req@ffff9cf254889c00 x1866777609670912/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/440 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 65536/65536 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298645.211040:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf254889c00 with x1866777609670912 ext(1929216->1994751) 00002000:00010000:0.0:1780298645.211045:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf254889c00 00000100:00100000:3.0:1780298645.212288:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.212328:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.212360:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609671040 00000100:00100000:2.0:1780298645.212370:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609671040 00000100:00100000:0.0:1780298645.212375:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609671168 00000100:00100000:0.0:1780298645.212383:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609671168 00000100:00100000:2.0:1780298645.212432:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10243 00000100:00100000:2.0:1780298645.212450:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609671040:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298645.212454:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10244 00000100:00100000:0.0:1780298645.212472:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609671168:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.212479:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.212485:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.212500:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003540747e/0xaf35132e63035b5f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4a29 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.212501:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d4d8e3d7/0xaf35132e63035b66 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4a30 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.212598:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605077248:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.212599:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254889180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605077376:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.212686:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1900 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.212688:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.217943:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254889180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605077376:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.217983:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605077248:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.217999:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003540747e/0xaf35132e63035b5f lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a29 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.218030:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003540747e/0xaf35132e63035b5f lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a29 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.218042:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d4d8e3d7/0xaf35132e63035b66 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a30 expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.218045:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3c00, rc 0) 00010000:00010000:0.0:1780298645.218076:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d4d8e3d7/0xaf35132e63035b66 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a30 expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.218095:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf000, rc 0) 00000100:00100000:2.0:1780298645.218145:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609671040:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 5676us (5847us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.218160:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10243 00000100:00100000:0.0:1780298645.218172:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609671168:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 5700us (5839us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.218183:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10244 00000100:00100000:2.0:1780298645.223733:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.223780:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609671296 00000100:00100000:2.0:1780298645.223790:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.223794:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609671296 00000100:00100000:2.0:1780298645.223821:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609671424 00000100:00100000:2.0:1780298645.223825:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609671424 00000100:00100000:2.0:1780298645.223879:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10245 00000100:00100000:3.0:1780298645.223892:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10246 00000100:00100000:2.0:1780298645.223899:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254889180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609671296:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298645.223914:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25488ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609671424:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.223922:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.223928:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.223945:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e0b8e7b/0xaf35132e63035b74 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4a3e expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.223946:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003540747e/0xaf35132e63035b6d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4a37 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.224024:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605077632:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.224024:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254888380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605077504:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.224316:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.224328:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.225586:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605077632:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.225594:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254888380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605077504:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.225640:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009e0b8e7b/0xaf35132e63035b74 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a3e expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.225645:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003540747e/0xaf35132e63035b6d lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a37 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.225677:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e0b8e7b/0xaf35132e63035b74 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a3e expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.225681:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003540747e/0xaf35132e63035b6d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a37 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.225698:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156c00, rc 0) 00010000:00010000:2.0:1780298645.225703:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3c00, rc 0) 00000100:00100000:3.0:1780298645.225745:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25488ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609671424:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1840us (1955us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.225750:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254889180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609671296:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1862us (2026us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.225758:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10246 00000100:00100000:2.0:1780298645.225767:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10245 00000100:00100000:2.0:1780298645.227396:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.227513:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609671552 00000100:00100000:0.0:1780298645.227521:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609671552 00000100:00100000:0.0:1780298645.228429:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10247 00000100:00100000:0.0:1780298645.228449:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254888380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:29360:x1866777609671552:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.228466:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.228483:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d518bb3a/0xaf35132e63035b7b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4a45 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.228505:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(42 122) 00010000:00010000:0.0:1780298645.228510:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(42 122) 00010000:00010000:0.0:1780298645.228518:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d518bb3a/0xaf35132e63035b7b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1040384->2031615] (req 1040384->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a45 expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.228531:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d518bb3a/0xaf35132e63035b7b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1040384->2031615] (req 1040384->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a45 expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.228540:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bafe00, rc 0) 00000100:00100000:0.0:1780298645.229396:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254888380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:29360:x1866777609671552:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 951us (2003us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.229405:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10247 00000100:00100000:2.0:1780298645.229695:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.229741:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609671680 00000100:00100000:3.0:1780298645.229749:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609671680 00000100:00100000:3.0:1780298645.229806:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6431 00000100:00100000:3.0:1780298645.229820:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254888a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609671680:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:3.0:1780298645.230031:0:16972:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:3.0:1780298645.230036:0:16972:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:3.0:1780298645.230040:0:16972:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:3.0:1780298645.230043:0:16972:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:3.0:1780298645.230046:0:16972:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 24 -> 24 00000100:00100000:3.0:1780298645.230085:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254888a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609671680:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 268us (393us total) trans 4294969309 rc 0/0 00000100:00100000:3.0:1780298645.230095:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6431 00000100:00100000:2.0:1780298645.231215:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.231291:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.231340:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609671936 00000100:00100000:2.0:1780298645.231349:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609671936 00000100:00100000:2.0:1780298645.231411:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609671808 00000100:00100000:2.0:1780298645.231414:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609671808 00000100:00100000:2.0:1780298645.231457:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10248 00000100:00100000:2.0:1780298645.231472:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34831a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609671936:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.231491:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.231513:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000024bbcf41/0xaf35132e63035b82 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4a53 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.231598:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34831aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605077760:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.231745:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.231797:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10249 00000100:00100000:2.0:1780298645.231814:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348318a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609671808:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.231822:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.231837:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000052b89172/0xaf35132e63035b89 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4a4c expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.231871:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf348318700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605077888:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.231908:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c240 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.233011:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348318700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605077888:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.233041:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000052b89172/0xaf35132e63035b89 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a4c expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.233059:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000052b89172/0xaf35132e63035b89 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a4c expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.233074:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2800, rc 0) 00000100:00100000:2.0:1780298645.233108:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348318a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609671808:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1302us (1819us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.233114:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10249 00000100:00100000:2.0:1780298645.233309:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34831aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605077760:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.233347:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000024bbcf41/0xaf35132e63035b82 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a53 expref: 13 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.233360:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000024bbcf41/0xaf35132e63035b82 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a53 expref: 13 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.233379:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2c00, rc 0) 00000100:00100000:2.0:1780298645.233425:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34831a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609671936:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1959us (2221us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.233437:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10248 00000100:00100000:2.0:1780298645.234526:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.234578:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.234633:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609672064 00000100:00100000:2.0:1780298645.234639:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609672064 00000100:00100000:2.0:1780298645.234714:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609672192 00000100:00100000:2.0:1780298645.234717:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609672192 00000100:00100000:2.0:1780298645.234754:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10250 00000100:00100000:2.0:1780298645.234769:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34831aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609672064:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.234777:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.234790:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000045ae2ef5/0xaf35132e63035b90 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4a5a expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.234826:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34831bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605078016:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.234926:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.235051:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10251 00000100:00100000:2.0:1780298645.235074:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348318700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609672192:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.235084:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.235101:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000075e99a4b/0xaf35132e63035b97 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4a61 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.235176:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605078144:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.235249:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c240 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.235809:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34831bb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605078016:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.235847:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000045ae2ef5/0xaf35132e63035b90 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a5a expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.235871:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000045ae2ef5/0xaf35132e63035b90 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a5a expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.235891:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3200, rc 0) 00000100:00100000:2.0:1780298645.235936:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34831aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609672064:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1170us (1411us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.235952:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10250 00000100:00100000:2.0:1780298645.236219:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605078144:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.236260:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000075e99a4b/0xaf35132e63035b97 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a61 expref: 13 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.236289:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000075e99a4b/0xaf35132e63035b97 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a61 expref: 13 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.236305:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de3800, rc 0) 00000100:00100000:2.0:1780298645.236347:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348318700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609672192:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1284us (1766us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.236361:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10251 00000100:00100000:2.0:1780298645.237459:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.237521:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609672320 00000100:00100000:2.0:1780298645.237527:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609672320 00000100:00100000:2.0:1780298645.237605:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10252 00000100:00100000:2.0:1780298645.237631:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609672320:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.237641:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.237658:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e63035b9e lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4a68 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.237686:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d828f4f8/0xaf35132e63035aef lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2387968->35942399] (req 2387968->2400255) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d49b9 expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.237696:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(44 169) 00010000:00010000:2.0:1780298645.237702:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(45 169) 00010000:00010000:2.0:1780298645.237712:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e63035b9e lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3489792->3493887] (req 3489792->3493887) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a68 expref: 11 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.237744:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d828f4f8/0xaf35132e63035aef lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2387968->35942399] (req 2387968->2400255) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d49b9 expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.237770:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d828f4f8/0xaf35132e63035aef lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2387968->35942399] (req 2387968->2400255) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d49b9 expref: 9 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.237788:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605078272:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298645.237876:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c240 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.238660:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605078272:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298645.238691:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e63035b9e lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3489792->3493887] (req 3489792->3493887) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a68 expref: 11 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.238704:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf350de2400, rc 0) 00000100:00100000:2.0:1780298645.238746:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:29360:x1866777609672320:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1128us (1287us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.238759:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10252 00000100:00100000:2.0:1780298645.238883:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.238924:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609672448 00000100:00100000:1.0:1780298645.238929:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609672448 00010000:00010000:1.0:1780298645.238954:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d828f4f8/0xaf35132e63035aef lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2387968->35942399] (req 2387968->2400255) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d49b9 expref: 10 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.239595:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15266 00000100:00100000:1.0:1780298645.239604:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609672448:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298645.239613:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298645.239624:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d828f4f8/0xaf35132e63035aef lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2387968->35942399] (req 2387968->2400255) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d49b9 expref: 10 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.239707:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d828f4f8/0xaf35132e63035aef lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2387968->35942399] (req 2387968->2400255) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d49b9 expref: 10 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.239729:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d828f4f8/0xaf35132e63035aef lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2387968->35942399] (req 2387968->2400255) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d49b9 expref: 10 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.239742:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(45 169) 00010000:00010000:1.0:1780298645.239745:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(45 169) 00010000:00010000:1.0:1780298645.239753:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e63035b9e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3489792->37044223] (req 3489792->3493887) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a68 expref: 10 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.239760:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e63035b9e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3489792->37044223] (req 3489792->3493887) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a68 expref: 10 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.239786:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e63035b9e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3489792->37044223] (req 3489792->3493887) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a68 expref: 10 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.239794:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f8380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777605078400:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298645.239832:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e540 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.240735:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f8380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777605078400:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298645.240763:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298645.240801:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609672448:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1193us (1922us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298645.240814:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15266 00000100:00100000:2.0:1780298645.240934:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.240973:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609672576 00000100:00100000:1.0:1780298645.240977:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609672576 00000100:00100000:1.0:1780298645.241047:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6432 00000100:00100000:1.0:1780298645.241061:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609672576:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298645.241535:0:19949:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4618229 -> 4618229 00002000:00010000:1.0:1780298645.241540:0:19949:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298645.241543:0:19949:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298645.241546:0:19949:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298645.241548:0:19949:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 496 -> 496 00000100:00100000:1.0:1780298645.241596:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609672576:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 534us (661us total) trans 4294969766 rc 0/0 00000100:00100000:1.0:1780298645.241605:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6432 00000100:00100000:2.0:1780298645.242483:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.242532:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609672704 00000100:00100000:0.0:1780298645.242542:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609672704 00000100:00100000:2.0:1780298645.242555:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.242600:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609672832 00000100:00100000:0.0:1780298645.242605:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609672832 00000100:00100000:1.0:1780298645.242606:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10253 00000100:00100000:1.0:1780298645.242620:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609672704:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.242628:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.242639:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e63035ba5 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4a6f expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.242661:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10254 00000100:00100000:0.0:1780298645.242675:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609672832:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298645.242676:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605078528:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.242706:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.242724:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d4d8e3d7/0xaf35132e63035bac lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4a76 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.242746:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e540 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.242783:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605078656:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.242855:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1d80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.243902:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605078656:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.243953:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d4d8e3d7/0xaf35132e63035bac lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a76 expref: 13 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.243975:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d4d8e3d7/0xaf35132e63035bac lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a76 expref: 13 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.243994:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf000, rc 0) 00000100:00100000:0.0:1780298645.244042:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609672832:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1373us (1485us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.244052:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10254 00000100:00100000:1.0:1780298645.244793:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605078528:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.244815:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e63035ba5 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a6f expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.244830:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e63035ba5 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a6f expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.244842:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279800, rc 0) 00000100:00100000:1.0:1780298645.244879:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609672704:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 2261us (2395us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.244886:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10253 00000100:00100000:3.0:1780298645.245860:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.245899:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609672960 00000100:00100000:0.0:1780298645.245906:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609672960 00000100:00100000:0.0:1780298645.245952:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10255 00000100:00100000:0.0:1780298645.245963:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609672960:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.245974:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.245987:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000075508ee2/0xaf35132e63035bb3 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4a7d expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.246015:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.246035:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605078784:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.246063:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609673088 00000100:00100000:1.0:1780298645.246070:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609673088 00000100:00100000:0.0:1780298645.246157:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1d80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.246201:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10256 00000100:00100000:1.0:1780298645.246211:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609673088:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.246220:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.246229:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e63035bba lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4a84 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.246256:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605078912:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.246312:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e540 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.246906:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605078784:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.246912:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605078912:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.246936:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000075508ee2/0xaf35132e63035bb3 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a7d expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.246949:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e63035bba lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a84 expref: 13 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.246952:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000075508ee2/0xaf35132e63035bb3 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a7d expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.246995:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae200, rc 0) 00010000:00010000:1.0:1780298645.247007:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e63035bba lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a84 expref: 13 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.247020:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279a00, rc 0) 00000100:00100000:0.0:1780298645.247033:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609672960:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1072us (1173us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.247044:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10255 00000100:00100000:1.0:1780298645.247059:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609673088:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 844us (1039us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.247071:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10256 00000100:00100000:2.0:1780298645.248120:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.248177:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609673216 00000100:00100000:0.0:1780298645.248182:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609673216 00000100:00100000:2.0:1780298645.248201:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.248223:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609673344 00000100:00100000:0.0:1780298645.248226:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609673344 00000100:00100000:2.0:1780298645.248252:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10257 00000100:00100000:1.0:1780298645.248263:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10258 00000100:00100000:2.0:1780298645.248269:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609673216:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.248290:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298645.248291:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609673344:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.248303:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63035bc1 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4a8b expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.248303:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.248312:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004fdbd682/0xaf35132e63035bc8 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4a92 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.248339:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605079040:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.248348:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605079168:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.248381:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c240 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.248381:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e540 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.249214:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605079040:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.249238:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63035bc1 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a8b expref: 13 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.249258:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d4cf86a3/0xaf35132e63035bc1 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a8b expref: 13 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.249264:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605079168:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.249297:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2600, rc 0) 00010000:00010000:0.0:1780298645.249332:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004fdbd682/0xaf35132e63035bc8 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a92 expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.249342:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609673216:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1077us (1220us total) trans 0 rc 301/301 00010000:00010000:0.0:1780298645.249350:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004fdbd682/0xaf35132e63035bc8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a92 expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.249360:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10257 00010000:00010000:0.0:1780298645.249383:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278e00, rc 0) 00000100:00100000:0.0:1780298645.249425:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609673344:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1147us (1225us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.249431:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10258 00000100:00100000:2.0:1780298645.250435:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.250489:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609673472 00000100:00100000:2.0:1780298645.250501:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609673472 00000100:00100000:2.0:1780298645.250575:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10259 00000100:00100000:2.0:1780298645.250599:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:29360:x1866777609673472:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.250613:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.250634:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e63035bcf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4a99 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.250662:0:19992:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d518bb3a/0xaf35132e63035b7b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1040384->2031615] (req 1040384->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a45 expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.250670:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(42 122) 00010000:00010000:2.0:1780298645.250675:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(43 122) 00010000:00010000:2.0:1780298645.250683:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e63035bcf lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1998848->2011135] (req 1998848->2011135) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a99 expref: 14 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.250708:0:19992:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d518bb3a/0xaf35132e63035b7b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1040384->2031615] (req 1040384->1048575) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a45 expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.250735:0:19992:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d518bb3a/0xaf35132e63035b7b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1040384->2031615] (req 1040384->1048575) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a45 expref: 11 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.250751:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605079296:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298645.250888:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.251676:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605079296:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298645.251725:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e63035bcf lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1998848->2011135] (req 1998848->2011135) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a99 expref: 14 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.251739:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2e00, rc 0) 00000100:00100000:0.0:1780298645.251778:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:29360:x1866777609673472:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1189us (1353us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.251786:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10259 00000100:00100000:0.0:1780298645.251949:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.251982:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609673600 00000100:00100000:0.0:1780298645.251986:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609673600 00010000:00010000:0.0:1780298645.252013:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d518bb3a/0xaf35132e63035b7b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1040384->2031615] (req 1040384->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a45 expref: 12 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.252062:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15267 00000100:00100000:0.0:1780298645.252077:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609673600:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298645.252091:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298645.252104:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d518bb3a/0xaf35132e63035b7b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [1040384->2031615] (req 1040384->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a45 expref: 12 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.252151:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d518bb3a/0xaf35132e63035b7b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [1040384->2031615] (req 1040384->1048575) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a45 expref: 12 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.252177:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d518bb3a/0xaf35132e63035b7b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [1040384->2031615] (req 1040384->1048575) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a45 expref: 12 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.252194:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(43 122) 00010000:00010000:0.0:1780298645.252198:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(43 122) 00010000:00010000:0.0:1780298645.252206:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e63035bcf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1998848->2031615] (req 1998848->2011135) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a99 expref: 13 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.252214:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e63035bcf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1998848->2031615] (req 1998848->2011135) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a99 expref: 13 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.252244:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e63035bcf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1998848->2031615] (req 1998848->2011135) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a99 expref: 13 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.252256:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372ce00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777605079424:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298645.252338:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1d80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.253002:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372ce00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777605079424:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298645.253023:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298645.253072:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609673600:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1001us (1129us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298645.253080:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15267 00000100:00100000:2.0:1780298645.253097:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.253121:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609673728 00000100:00100000:1.0:1780298645.253185:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609673728 00000100:00100000:1.0:1780298645.253236:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6433 00000100:00100000:1.0:1780298645.253246:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609673728:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298645.253473:0:19928:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:1.0:1780298645.253476:0:19928:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298645.253478:0:19928:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298645.253480:0:19928:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298645.253482:0:19928:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 24 -> 48 00000100:00100000:1.0:1780298645.253511:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2400:x1866777609673728:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 269us (417us total) trans 4294969310 rc 0/0 00000100:00100000:1.0:1780298645.253517:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6433 00000100:00100000:2.0:1780298645.254597:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.254631:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609673856 00000100:00100000:1.0:1780298645.254635:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609673856 00000100:00100000:1.0:1780298645.254669:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10260 00000100:00100000:1.0:1780298645.254678:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609673856:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.254686:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.254697:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63035bd6 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4aa0 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.254709:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.254744:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609673984 00000100:00100000:1.0:1780298645.254746:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605079552:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.254752:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609673984 00000100:00100000:0.0:1780298645.254794:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10261 00000100:00100000:0.0:1780298645.254805:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2397:x1866777609673984:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.254819:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.254832:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000075508ee2/0xaf35132e63035bdd lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4aa7 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.254844:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ea80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.254868:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605079680:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.254907:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.255634:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605079552:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.255666:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63035bd6 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4aa0 expref: 13 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.255684:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63035bd6 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4aa0 expref: 13 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.255699:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279a00, rc 0) 00000100:00100000:1.0:1780298645.255736:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609673856:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1058us (1140us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.255743:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10260 00000100:00100000:0.0:1780298645.255769:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605079680:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.255800:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000075508ee2/0xaf35132e63035bdd lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4aa7 expref: 16 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.255813:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000075508ee2/0xaf35132e63035bdd lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4aa7 expref: 16 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.255820:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae200, rc 0) 00000100:00100000:0.0:1780298645.255856:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2397:x1866777609673984:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1055us (1153us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.255862:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10261 00000100:00100000:3.0:1780298645.256742:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.256782:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609674112 00000100:00100000:0.0:1780298645.256788:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609674112 00000100:00100000:3.0:1780298645.256806:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.256843:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609674240 00000100:00100000:0.0:1780298645.256846:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609674240 00000100:00100000:3.0:1780298645.256869:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10262 00000100:00100000:0.0:1780298645.256881:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10263 00000100:00100000:3.0:1780298645.256885:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609674112:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298645.256898:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2397:x1866777609674240:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.256899:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.256910:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.256916:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009e0b8e7b/0xaf35132e63035be4 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4aae expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.256921:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e63035beb lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4ab5 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.256978:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605079808:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.256987:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f744000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605079936:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.257032:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.257041:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.258007:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f744000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605079936:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.258042:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009e0b8e7b/0xaf35132e63035be4 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4aae expref: 13 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.258065:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009e0b8e7b/0xaf35132e63035be4 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4aae expref: 13 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.258078:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156c00, rc 0) 00000100:00100000:3.0:1780298645.258158:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609674112:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1242us (1381us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.258169:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10262 00000100:00100000:3.0:1780298645.258414:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605079808:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.258459:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e63035beb lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4ab5 expref: 16 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.258478:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000127fe2a2/0xaf35132e63035beb lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4ab5 expref: 16 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.258493:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf600, rc 0) 00000100:00100000:3.0:1780298645.258535:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2397:x1866777609674240:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1644us (1730us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.258543:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10263 00000100:00100000:3.0:1780298645.259832:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.259885:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609674368 00000100:00100000:3.0:1780298645.259891:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609674368 00000100:00100000:3.0:1780298645.259958:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10264 00000100:00100000:3.0:1780298645.259977:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f744000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:29360:x1866777609674368:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.259988:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.260002:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63035bf2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4abc expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.260028:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000007b6539a7/0xaf35132e63035a5c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2031616->35586047] (req 2031616->2035711) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d492d expref: 14 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.260039:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(43 122) 00010000:00010000:3.0:1780298645.260045:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(44 122) 00010000:00010000:3.0:1780298645.260052:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63035bf2 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3764224->3772415] (req 3764224->3772415) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4abc expref: 12 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.260078:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000007b6539a7/0xaf35132e63035a5c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2031616->35586047] (req 2031616->2035711) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d492d expref: 14 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.260095:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000007b6539a7/0xaf35132e63035a5c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2031616->35586047] (req 2031616->2035711) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d492d expref: 14 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.260109:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f745500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605080064:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298645.260219:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.261261:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.261312:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609674496 00000100:00100000:2.0:1780298645.261321:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609674496 00010000:00010000:2.0:1780298645.261361:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000007b6539a7/0xaf35132e63035a5c lrc: 5/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2031616->35586047] (req 2031616->2035711) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d492d expref: 15 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.261399:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f745500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605080064:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298645.261415:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15268 00000100:00100000:2.0:1780298645.261430:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f746680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2400:x1866777609674496:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298645.261437:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63035bf2 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3764224->3772415] (req 3764224->3772415) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4abc expref: 12 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.261447:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298645.261455:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157c00, rc 0) 00010000:00010000:2.0:1780298645.261464:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000007b6539a7/0xaf35132e63035a5c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [2031616->35586047] (req 2031616->2035711) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d492d expref: 15 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.261480:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000007b6539a7/0xaf35132e63035a5c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [2031616->35586047] (req 2031616->2035711) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d492d expref: 15 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.261494:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f744000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:29360:x1866777609674368:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1523us (1664us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298645.261505:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10264 00010000:00010000:2.0:1780298645.261505:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007b6539a7/0xaf35132e63035a5c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [2031616->35586047] (req 2031616->2035711) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d492d expref: 15 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.261530:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(44 122) 00010000:00010000:2.0:1780298645.261536:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(44 122) 00010000:00010000:2.0:1780298645.261545:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63035bf2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3764224->37318655] (req 3764224->3772415) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4abc expref: 11 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.261554:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63035bf2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3764224->37318655] (req 3764224->3772415) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4abc expref: 11 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.261581:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63035bf2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3764224->37318655] (req 3764224->3772415) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4abc expref: 11 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.261593:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f1880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:7400:1866777605080192:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298645.261655:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.262365:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f1880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:7400:1866777605080192:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298645.262391:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298645.262436:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f746680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609674496:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1009us (1177us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.262445:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15268 00000100:00100000:3.0:1780298645.262785:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.262818:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609674624 00000100:00100000:2.0:1780298645.262823:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609674624 00002000:00100000:2.0:1780298645.262852:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (3764224->3768319) req@ffff9cf37f745500 x1866777609674624/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.262862:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f745500 with x1866777609674624 ext(3764224->3768319) 00002000:00010000:2.0:1780298645.262866:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f745500 00000100:00100000:2.0:1780298645.262884:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6434 00000100:00100000:2.0:1780298645.262890:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609674624:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298645.263059:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609674624:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 169us (280us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298645.263067:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6434 00002000:00100000:2.0:1780298645.263072:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x280000401:0x12:0x0] (3764224->3768319) req@ffff9cf37f745500 x1866777609674624/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/4536 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.263078:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f745500 with x1866777609674624 ext(3764224->3768319) 00002000:00010000:2.0:1780298645.263081:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f745500 00000100:00100000:3.0:1780298645.263981:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.264181:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609674752 00000100:00100000:0.0:1780298645.264187:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609674752 00002000:00100000:0.0:1780298645.264216:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x12:0x0] (3768320->3772415) req@ffff9cf255489500 x1866777609674752/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298645.264224:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255489500 with x1866777609674752 ext(3768320->3772415) 00002000:00010000:0.0:1780298645.264229:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf255489500 00000100:00100000:0.0:1780298645.264249:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6435 00000100:00100000:0.0:1780298645.264255:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255489500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609674752:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298645.264401:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255489500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609674752:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 146us (424us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298645.264412:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6435 00002000:00100000:0.0:1780298645.264420:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x12:0x0] (3768320->3772415) req@ffff9cf255489500 x1866777609674752/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/4536 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298645.264427:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255489500 with x1866777609674752 ext(3768320->3772415) 00002000:00010000:0.0:1780298645.264431:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf255489500 00000100:00100000:3.0:1780298645.265644:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.265675:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609675008 00000100:00100000:0.0:1780298645.265681:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609675008 00000100:00100000:3.0:1780298645.265688:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.265731:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609674880 00000100:00100000:0.0:1780298645.265734:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609674880 00000100:00100000:3.0:1780298645.265738:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10265 00000100:00100000:3.0:1780298645.265750:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609675008:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298645.265757:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10266 00010000:00010000:3.0:1780298645.265763:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298645.265767:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244810e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609674880:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.265775:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.265780:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bed20494/0xaf35132e63035bf9 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4aca expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.265786:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d518bb3a/0xaf35132e63035c00 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4ac3 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.265832:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605080320:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.265836:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244811880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605080448:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.265891:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1900 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.265896:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.266721:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605080320:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.266741:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d518bb3a/0xaf35132e63035c00 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4ac3 expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.266757:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d518bb3a/0xaf35132e63035c00 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4ac3 expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.266770:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bafe00, rc 0) 00000100:00100000:3.0:1780298645.266806:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244810e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609674880:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1042us (1118us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.266815:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10266 00000100:00100000:3.0:1780298645.266943:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244811880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605080448:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.266972:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bed20494/0xaf35132e63035bf9 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4aca expref: 13 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.266985:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bed20494/0xaf35132e63035bf9 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4aca expref: 13 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.266996:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157000, rc 0) 00000100:00100000:3.0:1780298645.267020:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609675008:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1275us (1380us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.267027:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10265 00000100:00100000:3.0:1780298645.267927:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.268112:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.268263:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609675136 00000100:00100000:3.0:1780298645.268282:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609675136 00000100:00100000:3.0:1780298645.268338:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609675264 00000100:00100000:3.0:1780298645.268344:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609675264 00000100:00100000:3.0:1780298645.268402:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10267 00000100:00100000:3.0:1780298645.268420:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244811880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609675136:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.268431:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.268444:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004ba675d2/0xaf35132e63035c07 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4ad1 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.268482:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343966d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605080576:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.268562:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.268596:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10268 00000100:00100000:3.0:1780298645.268603:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244812680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609675264:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.268610:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.268615:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8b6afab/0xaf35132e63035c0e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4ad8 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.268641:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343966300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605080704:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.268665:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2418563c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.269479:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343966300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605080704:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.269494:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e8b6afab/0xaf35132e63035c0e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4ad8 expref: 13 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.269505:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8b6afab/0xaf35132e63035c0e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4ad8 expref: 13 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.269511:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157800, rc 0) 00000100:00100000:3.0:1780298645.269533:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244812680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609675264:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 930us (1424us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.269544:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10268 00000100:00100000:3.0:1780298645.269609:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343966d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605080576:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.269621:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004ba675d2/0xaf35132e63035c07 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4ad1 expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.269630:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004ba675d2/0xaf35132e63035c07 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4ad1 expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.269637:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156e00, rc 0) 00000100:00100000:3.0:1780298645.269690:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244811880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609675136:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1281us (1765us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.269695:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10267 00000100:00100000:3.0:1780298645.270847:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.270892:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609675392 00000100:00100000:1.0:1780298645.270901:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609675392 00002000:00100000:1.0:1780298645.270939:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (3506176->3510271) req@ffff9cf343966d80 x1866777609675392/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.270948:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf343966d80 with x1866777609675392 ext(3506176->3510271) 00002000:00010000:1.0:1780298645.270954:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf343966d80 00000100:00100000:1.0:1780298645.270990:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6436 00000100:00100000:1.0:1780298645.271002:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343966d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609675392:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298645.271195:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343966d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609675392:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 193us (344us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298645.271207:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6436 00002000:00100000:1.0:1780298645.271218:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (3506176->3510271) req@ffff9cf343966d80 x1866777609675392/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/4536 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.271224:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf343966d80 with x1866777609675392 ext(3506176->3510271) 00002000:00010000:1.0:1780298645.271228:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf343966d80 00000100:00100000:3.0:1780298645.271992:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.272034:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609675520 00000100:00100000:1.0:1780298645.272038:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609675520 00002000:00100000:1.0:1780298645.272054:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x2c0000401:0x11:0x0] (3510272->3514367) req@ffff9cf343966300 x1866777609675520/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.272061:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf343966300 with x1866777609675520 ext(3510272->3514367) 00002000:00010000:1.0:1780298645.272063:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf343966300 00000100:00100000:1.0:1780298645.272083:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6437 00000100:00100000:1.0:1780298645.272096:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343966300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609675520:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298645.272243:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343966300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609675520:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 146us (245us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298645.272262:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6437 00002000:00100000:1.0:1780298645.272284:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x2c0000401:0x11:0x0] (3510272->3514367) req@ffff9cf343966300 x1866777609675520/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/4536 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.272295:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf343966300 with x1866777609675520 ext(3510272->3514367) 00002000:00010000:1.0:1780298645.272301:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf343966300 00000100:00100000:3.0:1780298645.273352:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.273389:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609675648 00000100:00100000:0.0:1780298645.273395:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609675648 00000100:00100000:0.0:1780298645.273438:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10269 00000100:00100000:0.0:1780298645.273448:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343964380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609675648:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.273461:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.273476:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000075508ee2/0xaf35132e63035c15 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4adf expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.273520:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605080832:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.273552:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.273629:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.273670:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609675776 00000100:00100000:1.0:1780298645.273674:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609675776 00000100:00100000:1.0:1780298645.273715:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10270 00000100:00100000:1.0:1780298645.273725:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2398:x1866777609675776:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.273736:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.273747:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63035c1c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4ae6 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.273787:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605080960:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.273829:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306eb40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.274389:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605080832:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.274415:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000075508ee2/0xaf35132e63035c15 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4adf expref: 13 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.274427:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000075508ee2/0xaf35132e63035c15 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4adf expref: 13 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.274435:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae200, rc 0) 00000100:00100000:1.0:1780298645.274442:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605080960:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.274472:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63035c1c lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4ae6 expref: 15 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.274475:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343964380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609675648:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1027us (1122us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.274513:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10269 00010000:00010000:1.0:1780298645.274532:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63035c1c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4ae6 expref: 15 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.274543:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279200, rc 0) 00000100:00100000:1.0:1780298645.274573:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2398:x1866777609675776:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 851us (1027us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.274578:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10270 00000100:00100000:0.0:1780298645.275495:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.275545:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.275591:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609675904 00000100:00100000:0.0:1780298645.275596:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609675904 00000100:00100000:1.0:1780298645.275617:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609676032 00000100:00100000:1.0:1780298645.275626:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609676032 00000100:00100000:0.0:1780298645.275646:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10271 00000100:00100000:0.0:1780298645.275656:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e4f1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609675904:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.275664:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298645.275671:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10272 00010000:00010000:0.0:1780298645.275673:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e63035c23 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4aed expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.275684:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2397:x1866777609676032:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.275692:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.275700:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6416acb/0xaf35132e63035c2a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4af4 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.275706:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255090a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605081088:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.275742:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605081216:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.275764:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1d80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.275778:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306eb40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.276439:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605081216:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.276462:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a6416acb/0xaf35132e63035c2a lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4af4 expref: 15 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.276469:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255090a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605081088:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.276490:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6416acb/0xaf35132e63035c2a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4af4 expref: 15 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.276501:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278c00, rc 0) 00010000:00010000:0.0:1780298645.276507:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e63035c23 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4aed expref: 13 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.276524:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e63035c23 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4aed expref: 13 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.276536:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae400, rc 0) 00000100:00100000:1.0:1780298645.276545:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2397:x1866777609676032:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 857us (991us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.276551:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10272 00000100:00100000:0.0:1780298645.276574:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e4f1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609675904:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 918us (1086us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.276582:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10271 00000100:00100000:0.0:1780298645.277641:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.277680:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609676288 00000100:00100000:1.0:1780298645.277687:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609676288 00000100:00100000:1.0:1780298645.277726:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10273 00000100:00100000:1.0:1780298645.277734:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255090a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609676288:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.277746:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.277758:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e63035c31 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4b02 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.277795:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605081344:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.277824:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.277845:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609676160 00000100:00100000:3.0:1780298645.277850:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609676160 00000100:00100000:1.0:1780298645.277882:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306eb40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.277911:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10274 00000100:00100000:3.0:1780298645.277921:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255093b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609676160:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.277928:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.277956:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e8b6afab/0xaf35132e63035c38 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4afb expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.277980:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf343965880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605081472:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.278016:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856000 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.278541:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605081344:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.278565:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e63035c31 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b02 expref: 15 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.278589:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e63035c31 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b02 expref: 15 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.278600:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278400, rc 0) 00000100:00100000:1.0:1780298645.278639:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255090a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609676288:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 906us (997us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.278648:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10273 00000100:00100000:3.0:1780298645.278726:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf343965880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605081472:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.278749:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e8b6afab/0xaf35132e63035c38 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4afb expref: 13 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.278809:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e8b6afab/0xaf35132e63035c38 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4afb expref: 13 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.278820:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157800, rc 0) 00000100:00100000:3.0:1780298645.278859:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255093b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609676160:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 939us (1036us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.278866:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10274 00000100:00100000:0.0:1780298645.279743:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.279795:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609676416 00000100:00100000:1.0:1780298645.279802:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609676416 00000100:00100000:1.0:1780298645.279848:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6438 00000100:00100000:1.0:1780298645.279858:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255092300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609676416:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298645.280039:0:8363:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4618229 -> 4618229 00002000:00010000:1.0:1780298645.280043:0:8363:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298645.280045:0:8363:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298645.280048:0:8363:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298645.280050:0:8363:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 496 -> 512 00000100:00100000:1.0:1780298645.280087:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255092300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609676416:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 231us (348us total) trans 4294969767 rc 0/0 00000100:00100000:1.0:1780298645.280095:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6438 00000100:00100000:2.0:1780298645.280970:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.281029:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609676672 00000100:00100000:1.0:1780298645.281035:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609676672 00000100:00100000:1.0:1780298645.281067:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10275 00000100:00100000:1.0:1780298645.281075:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609676672:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.281085:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298645.281092:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:1.0:1780298645.281094:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63035c3f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4b10 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.281108:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609676544 00000100:00100000:3.0:1780298645.281113:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609676544 00000100:00100000:1.0:1780298645.281120:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605081600:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.281195:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306eb40 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.281245:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10276 00000100:00100000:0.0:1780298645.281254:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255092680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609676544:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.281260:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.281269:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63035c46 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4b09 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.281319:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255090700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605081728:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.281356:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1d80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.281916:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605081600:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.281943:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63035c3f lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b10 expref: 13 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.281955:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63035c3f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b10 expref: 13 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.281962:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279200, rc 0) 00000100:00100000:1.0:1780298645.281991:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609676672:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 919us (1024us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.281999:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10275 00000100:00100000:0.0:1780298645.282044:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255090700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605081728:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.282074:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63035c46 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b09 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.282083:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63035c46 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b09 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.282089:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baee00, rc 0) 00000100:00100000:0.0:1780298645.282113:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255092680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609676544:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 862us (1023us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.282119:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10276 00000100:00100000:2.0:1780298645.282871:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.282910:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609676800 00000100:00100000:2.0:1780298645.282911:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.282916:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609676800 00000100:00100000:1.0:1780298645.282944:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609676928 00000100:00100000:1.0:1780298645.282946:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609676928 00000100:00100000:1.0:1780298645.282966:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10277 00000100:00100000:0.0:1780298645.282975:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10278 00000100:00100000:1.0:1780298645.282976:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609676800:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.282984:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298645.282989:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609676928:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.282994:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63035c4d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4b17 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.282997:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.283006:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a3f03a/0xaf35132e63035c54 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4b1e expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.283019:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605081856:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.283032:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255090700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605081984:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.283061:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306eb40 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.283062:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1d80 going to sleep for 11 seconds 00010000:00010000:3.0:1780298645.283333:0:10277:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1780298645.283728:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605081856:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.283740:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255090700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605081984:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.283745:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63035c4d lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b17 expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.283757:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63035c4d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b17 expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.283761:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000001a3f03a/0xaf35132e63035c54 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b1e expref: 13 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.283765:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278a00, rc 0) 00010000:00010000:0.0:1780298645.283783:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a3f03a/0xaf35132e63035c54 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b1e expref: 13 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.283792:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae600, rc 0) 00000100:00100000:1.0:1780298645.283794:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609676800:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 820us (925us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.283799:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10277 00000100:00100000:0.0:1780298645.283815:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609676928:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 831us (909us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.283822:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10278 00000100:00100000:3.0:1780298645.284716:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.284766:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609677056 00000100:00100000:2.0:1780298645.284775:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609677056 00000100:00100000:2.0:1780298645.284828:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10279 00000100:00100000:3.0:1780298645.284842:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.284849:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343965880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609677056:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.284864:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298645.284874:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609677184 00000100:00100000:0.0:1780298645.284879:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609677184 00010000:00010000:2.0:1780298645.284880:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e63035c5b lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4b25 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.284914:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10280 00000100:00100000:0.0:1780298645.284923:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343964380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2398:x1866777609677184:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.284931:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298645.284937:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f3100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605082112:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.284939:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63035c62 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4b2c expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.284976:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255090700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605082240:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.285021:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1d80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.285036:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c840 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.285720:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255090700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605082240:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.285737:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63035c62 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b2c expref: 15 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.285748:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63035c62 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b2c expref: 15 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.285757:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae400, rc 0) 00000100:00100000:2.0:1780298645.285774:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f3100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605082112:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.285789:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343964380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2398:x1866777609677184:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 867us (949us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.285796:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10280 00010000:00010000:2.0:1780298645.285819:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e63035c5b lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b25 expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.285842:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e63035c5b lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b25 expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.285856:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3a00, rc 0) 00000100:00100000:2.0:1780298645.285898:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343965880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609677056:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1061us (1182us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.285906:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10279 00000100:00100000:3.0:1780298645.286711:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.286752:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609677312 00000100:00100000:2.0:1780298645.286758:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609677312 00000100:00100000:2.0:1780298645.286792:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10281 00000100:00100000:2.0:1780298645.286799:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:29360:x1866777609677312:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.286810:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.286821:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e63035c69 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4b33 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.286840:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(44 122) 00010000:00010000:2.0:1780298645.286845:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(44 122) 00010000:00010000:2.0:1780298645.286854:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e63035c69 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [524288->1998847] (req 524288->536575) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b33 expref: 15 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.286869:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e63035c69 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [524288->1998847] (req 524288->536575) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b33 expref: 15 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.286878:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3800, rc 0) 00000100:00100000:2.0:1780298645.286909:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:29360:x1866777609677312:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 112us (200us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.286916:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10281 00000100:00100000:3.0:1780298645.287571:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.287611:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609677440 00000100:00100000:2.0:1780298645.287625:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609677440 00000100:00100000:2.0:1780298645.287662:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6439 00000100:00100000:2.0:1780298645.287669:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2399:x1866777609677440:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298645.287831:0:19950:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:2.0:1780298645.287834:0:19950:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298645.287837:0:19950:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298645.287840:0:19950:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298645.287842:0:19950:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 48 -> 72 00000100:00100000:2.0:1780298645.287873:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2399:x1866777609677440:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 205us (305us total) trans 4294969311 rc 0/0 00000100:00100000:2.0:1780298645.287881:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6439 00000100:00100000:3.0:1780298645.288717:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.288755:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609677568 00000100:00100000:0.0:1780298645.288762:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609677568 00000100:00100000:0.0:1780298645.288805:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10282 00000100:00100000:3.0:1780298645.288811:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.288818:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609677568:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298645.288829:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609677696 00010000:00010000:0.0:1780298645.288830:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298645.288835:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609677696 00010000:00010000:0.0:1780298645.288843:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63035c70 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4b3a expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.288859:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10283 00000100:00100000:1.0:1780298645.288872:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609677696:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.288881:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298645.288883:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255090700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605082368:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.288890:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e63035c77 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4b41 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.288914:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605082496:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.288943:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306eb40 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.288943:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1d80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.289755:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605082496:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.289782:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e63035c77 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b41 expref: 17 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.289795:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e63035c77 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b41 expref: 17 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.289798:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255090700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605082368:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.289810:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278400, rc 0) 00010000:00010000:0.0:1780298645.289818:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63035c70 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b3a expref: 14 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.289829:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c47ea6a1/0xaf35132e63035c70 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b3a expref: 14 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.289838:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baf200, rc 0) 00000100:00100000:1.0:1780298645.289842:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609677696:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 973us (1028us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.289851:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10283 00000100:00100000:0.0:1780298645.289863:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609677568:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1048us (1146us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.289872:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10282 00000100:00100000:3.0:1780298645.290820:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.290844:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.290865:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609677824 00000100:00100000:2.0:1780298645.290873:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609677824 00000100:00100000:1.0:1780298645.290878:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609677952 00000100:00100000:1.0:1780298645.290883:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609677952 00000100:00100000:2.0:1780298645.290913:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10284 00000100:00100000:2.0:1780298645.290930:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7be300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609677824:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298645.290937:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10285 00000100:00100000:1.0:1780298645.290954:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609677952:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.290958:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.290965:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.290973:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6416acb/0xaf35132e63035c85 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4b4f expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.290975:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63035c7e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4b48 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.291022:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605082624:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.291046:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f3100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605082752:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.291064:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306eb40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.291108:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.291775:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605082624:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.291797:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a6416acb/0xaf35132e63035c85 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b4f expref: 17 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.291817:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f3100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605082752:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.291821:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6416acb/0xaf35132e63035c85 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b4f expref: 17 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.291836:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278c00, rc 0) 00010000:00010000:2.0:1780298645.291853:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63035c7e lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b48 expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.291871:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000887cd623/0xaf35132e63035c7e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b48 expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.291877:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609677952:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 920us (1026us total) trans 0 rc 301/301 00010000:00010000:2.0:1780298645.291897:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3000, rc 0) 00000100:00100000:1.0:1780298645.291897:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10285 00000100:00100000:2.0:1780298645.291936:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7be300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609677824:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1014us (1115us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.291945:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10284 00000100:00100000:3.0:1780298645.293248:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.293293:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609678080 00000100:00100000:0.0:1780298645.293299:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609678080 00002000:00100000:0.0:1780298645.293330:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (1507328->1519615) req@ffff9cf37f7bc700 x1866777609678080/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298645.293340:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f7bc700 with x1866777609678080 ext(1507328->1519615) 00002000:00010000:0.0:1780298645.293345:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f7bc700 00000100:00100000:0.0:1780298645.293367:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6440 00000100:00100000:0.0:1780298645.293377:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609678080:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298645.293575:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609678080:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 201us (327us total) trans 0 rc 12288/12288 00000100:00100000:0.0:1780298645.293583:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6440 00002000:00100000:0.0:1780298645.293594:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (1507328->1519615) req@ffff9cf37f7bc700 x1866777609678080/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/12728 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 12288/12288 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298645.293602:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f7bc700 with x1866777609678080 ext(1507328->1519615) 00002000:00010000:0.0:1780298645.293605:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f7bc700 00000100:00100000:3.0:1780298645.294609:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.294648:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609678208 00000100:00100000:0.0:1780298645.294653:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609678208 00000100:00100000:3.0:1780298645.294666:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.294691:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609678336 00000100:00100000:0.0:1780298645.294694:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609678336 00000100:00100000:0.0:1780298645.294714:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10286 00000100:00100000:1.0:1780298645.294726:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10287 00000100:00100000:0.0:1780298645.294727:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609678208:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.294737:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298645.294748:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609678336:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.294752:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e63035c8c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4b56 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.294756:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.294766:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63035c93 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4b5d expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.294793:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255090700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605082880:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.294797:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605083008:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.294844:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1540 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.294846:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306eb40 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.295586:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255090700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605082880:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.295592:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605083008:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.295606:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e63035c8c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b56 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.295611:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63035c93 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b5d expref: 13 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.295623:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000001a3f03a/0xaf35132e63035c8c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b56 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.295627:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63035c93 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b5d expref: 13 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.295640:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae600, rc 0) 00010000:00010000:1.0:1780298645.295642:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278a00, rc 0) 00000100:00100000:0.0:1780298645.295670:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609678208:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 949us (1063us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.295676:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609678336:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 937us (1009us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.295680:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10286 00000100:00100000:1.0:1780298645.295688:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10287 00000100:00100000:3.0:1780298645.296532:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.296576:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609678464 00000100:00100000:0.0:1780298645.296581:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609678464 00000100:00100000:0.0:1780298645.296615:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10288 00000100:00100000:0.0:1780298645.296624:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348007b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609678464:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.296633:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.296641:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63035c9a lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4b64 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.296657:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.296665:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255090700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605083136:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.296696:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609678592 00000100:00100000:0.0:1780298645.296703:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1540 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.296706:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609678592 00000100:00100000:2.0:1780298645.296754:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10289 00000100:00100000:2.0:1780298645.296768:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348004e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609678592:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.296780:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.296793:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63035ca1 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4b6b expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.296831:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f3100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605083264:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.296876:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c840 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.297317:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255090700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605083136:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.297335:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63035c9a lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b64 expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.297349:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63035c9a lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b64 expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.297359:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baee00, rc 0) 00000100:00100000:0.0:1780298645.297399:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348007b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609678464:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 773us (862us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.297407:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10288 00000100:00100000:0.0:1780298645.297522:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f3100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605083264:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.297544:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63035ca1 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b6b expref: 13 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.297563:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000578270eb/0xaf35132e63035ca1 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b6b expref: 13 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.297582:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2800, rc 0) 00000100:00100000:0.0:1780298645.297615:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348004e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609678592:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 853us (957us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.297623:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10289 00000100:00100000:3.0:1780298645.298454:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.298484:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609678720 00000100:00100000:1.0:1780298645.298489:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609678720 00000100:00100000:1.0:1780298645.298522:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6441 00000100:00100000:1.0:1780298645.298529:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348007100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609678720:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298645.298700:0:16972:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:1.0:1780298645.298703:0:16972:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298645.298705:0:16972:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298645.298706:0:16972:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298645.298708:0:16972:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 72 -> 72 00000100:00100000:1.0:1780298645.298732:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348007100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2397:x1866777609678720:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 204us (278us total) trans 4294969312 rc 0/0 00000100:00100000:1.0:1780298645.298736:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6441 00000100:00100000:3.0:1780298645.299518:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.299553:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.299568:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609678848 00000100:00100000:2.0:1780298645.299577:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609678848 00000100:00100000:0.0:1780298645.299601:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609678976 00000100:00100000:0.0:1780298645.299608:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609678976 00000100:00100000:2.0:1780298645.299637:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10290 00000100:00100000:0.0:1780298645.299655:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10291 00000100:00100000:2.0:1780298645.299665:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348007480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609678848:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298645.299673:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348006680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609678976:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.299685:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.299688:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.299693:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000075508ee2/0xaf35132e63035ca8 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4b79 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.299702:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e63035caf lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4b72 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.299754:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255090700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605083392:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.299770:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605083520:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.299799:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1540 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.299818:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c900 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.300506:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255090700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605083392:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.300523:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000075508ee2/0xaf35132e63035ca8 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b79 expref: 13 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.300536:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000075508ee2/0xaf35132e63035ca8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b79 expref: 13 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.300543:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae200, rc 0) 00000100:00100000:3.0:1780298645.300554:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605083520:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.300576:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e63035caf lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b72 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.300584:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348006680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609678976:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 915us (1029us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.300600:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10291 00010000:00010000:3.0:1780298645.300606:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000010b6573d/0xaf35132e63035caf lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b72 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.300625:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3a00, rc 0) 00000100:00100000:3.0:1780298645.300652:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348007480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609678848:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1006us (1133us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.300660:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10290 00000100:00100000:3.0:1780298645.301523:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.301552:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609679104 00000100:00100000:0.0:1780298645.301556:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609679104 00000100:00100000:0.0:1780298645.301580:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10292 00000100:00100000:0.0:1780298645.301586:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348004380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609679104:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.301592:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.301600:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e63035cb6 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4b80 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.301641:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255090700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605083648:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.301664:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.301692:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1540 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.301740:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609679232 00000100:00100000:3.0:1780298645.301745:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609679232 00000100:00100000:3.0:1780298645.301791:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10293 00000100:00100000:3.0:1780298645.301796:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609679232:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.301803:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.301808:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004ba675d2/0xaf35132e63035cbd lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4b87 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.301829:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf348005c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605083776:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.301857:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856000 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.302301:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255090700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605083648:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.302317:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e63035cb6 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b80 expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.302331:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a8bd3274/0xaf35132e63035cb6 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b80 expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.302342:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae400, rc 0) 00000100:00100000:0.0:1780298645.302382:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348004380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609679104:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 794us (857us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.302390:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10292 00000100:00100000:1.0:1780298645.302457:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348005c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605083776:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.302476:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004ba675d2/0xaf35132e63035cbd lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b87 expref: 13 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.302487:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004ba675d2/0xaf35132e63035cbd lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b87 expref: 13 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.302494:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156e00, rc 0) 00000100:00100000:1.0:1780298645.302521:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609679232:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 725us (860us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.302528:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10293 00000100:00100000:2.0:1780298645.303292:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.303324:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609679488 00000100:00100000:0.0:1780298645.303328:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609679488 00000100:00100000:0.0:1780298645.303357:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62065 00000100:00100000:0.0:1780298645.303364:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+27:29360:x1866777609679488:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:0.0:1780298645.303402:0:6477:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298645.303416:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf350de3600/0xaf35132e63035a0f lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d48e0 expref: 27 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.303428:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298645.303431:0:6477:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1780298645.303553:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf200/0xaf35132e63035cc4 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 12 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.303563:0:6477:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e63035aa9 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4973 expref: 34 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.303570:0:6477:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf200/0xaf35132e63035cc4 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 12 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.303585:0:6477:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e63035aa9 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4973 expref: 34 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.303594:0:6477:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e63035aa9 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4973 expref: 34 pid: 17578 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.303602:0:6477:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255090700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6477:1866777605083904:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298645.303636:0:6477:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1d80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.304306:0:6477:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255090700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6477:1866777605083904:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298645.304317:0:6477:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf200/0xaf35132e63035cc4 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 12 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.304339:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.304363:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609679616 00000100:00100000:0.0:1780298645.304368:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609679616 00010000:00010000:0.0:1780298645.304383:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e63035aa9 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4973 expref: 35 pid: 17578 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.304405:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15269 00000100:00100000:0.0:1780298645.304410:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+35:2399:x1866777609679616:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298645.304418:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298645.304422:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e63035aa9 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4973 expref: 35 pid: 17578 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.304433:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e63035aa9 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4973 expref: 35 pid: 17578 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.304447:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278800/0xaf35132e63035aa9 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4973 expref: 35 pid: 17578 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.304457:0:20253:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298645.304461:0:20253:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf200/0xaf35132e63035cc4 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 12 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.304470:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf200/0xaf35132e63035cc4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 12 type: IBT flags: 0x40290000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.304478:0:20253:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf200/0xaf35132e63035cc4 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 12 type: IBT flags: 0x40290000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.304501:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298645.304531:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+34:2399:x1866777609679616:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 121us (195us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.304539:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15269 00010000:00010000:2.0:1780298645.304554:0:6477:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf200/0xaf35132e63035cc4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.304567:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf200/0xaf35132e63035cc4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1780298645.304653:0:6477:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991110 00000004:00010000:2.0:1780298645.304699:0:6477:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 0000000070c98442 reply state 00000000302b5dc3 transno 4294991111 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf200/0xaf35132e63035cc4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 11 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.304717:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baf200/0xaf35132e63035cc4 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 11 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.304726:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298645.304728:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1780298645.304732:0:6477:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1780298645.304822:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+29:29360:x1866777609679488:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1457us (1529us total) trans 4294991111 rc 0/0 00000100:00100000:2.0:1780298645.304830:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62065 00000020:00080000:2.0:1780298645.304837:0:6477:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1780298645.305408:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.305445:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609679744 00000100:00100000:3.0:1780298645.305451:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609679744 00000100:00100000:3.0:1780298645.305477:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10294 00000100:00100000:3.0:1780298645.305483:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:29360:x1866777609679744:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.305490:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.305497:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e63035ccb lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4b8e expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.305512:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63035b2e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [81920->2383871] (req 81920->90111) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d49f8 expref: 8 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.305518:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e63035b9e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3489792->37044223] (req 3489792->3493887) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a68 expref: 14 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.305523:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 2 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(45 169) 00010000:00010000:3.0:1780298645.305526:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(47 169) 00010000:00010000:3.0:1780298645.305529:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e63035ccb lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b8e expref: 14 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.305541:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e63035b9e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3489792->37044223] (req 3489792->3493887) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a68 expref: 14 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.305548:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e63035b9e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3489792->37044223] (req 3489792->3493887) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a68 expref: 14 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.305560:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf348005c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605084032:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298645.305601:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63035b2e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [81920->2383871] (req 81920->90111) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d49f8 expref: 8 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.305608:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63035b2e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [81920->2383871] (req 81920->90111) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d49f8 expref: 8 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.305616:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b00000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605084160:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298645.305630:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2418563c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.306237:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348005c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605084032:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298645.306249:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b00000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605084160:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298645.306296:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e63035ccb lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b8e expref: 14 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.306309:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243157000, rc 0) 00000100:00100000:1.0:1780298645.306340:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:29360:x1866777609679744:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 856us (931us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298645.306346:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10294 00000100:00100000:3.0:1780298645.306617:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.306649:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609680000 00000100:00100000:2.0:1780298645.306651:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.306654:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609680000 00010000:00010000:0.0:1780298645.306672:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63035b2e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [81920->2383871] (req 81920->90111) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d49f8 expref: 9 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.306693:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609679872 00000100:00100000:1.0:1780298645.306699:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609679872 00000100:00100000:0.0:1780298645.306710:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15270 00000100:00100000:0.0:1780298645.306721:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b02d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609680000:12345-192.168.203.25@tcp:103:ldlm_bl.0 00002000:00100000:1.0:1780298645.306730:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x2c0000401:0x11:0x0] (3506176->3514367) req@ffff9cf2558f2680 x1866777609679872/t0(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 8680/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00010000:00010000:0.0:1780298645.306732:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00100000:1.0:1780298645.306743:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2558f2680 with x1866777609679872 ext(3506176->3514367) 00010000:00010000:0.0:1780298645.306750:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63035b2e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [81920->2383871] (req 81920->90111) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d49f8 expref: 9 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.306751:0:19951:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e63035b9e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [3489792->37044223] (req 3489792->3493887) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a68 expref: 14 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.306775:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63035b2e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [81920->2383871] (req 81920->90111) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d49f8 expref: 9 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.306779:0:19951:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e63035b9e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [3489792->37044223] (req 3489792->3493887) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a68 expref: 14 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1780298645.306793:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf2558f2680 00010000:00010000:0.0:1780298645.306799:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63035b2e lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [81920->2383871] (req 81920->90111) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d49f8 expref: 9 pid: 20252 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.306811:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298645.306820:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 214 00000100:00100000:1.0:1780298645.306830:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609679872:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:0.0:1780298645.306850:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b02d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609680000:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 132us (233us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.306858:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15270 00000100:00100000:1.0:1780298645.307300:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2399:x1866777609679872:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 473us (656us total) trans 4294969768 rc 0/0 00000100:00100000:1.0:1780298645.307305:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 214 00002000:00100000:1.0:1780298645.307311:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x2c0000401:0x11:0x0] (3506176->3514367) req@ffff9cf2558f2680 x1866777609679872/t4294969768(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 8680/448 e 0 to 0 dl 1780298656 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.307317:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2558f2680 with x1866777609679872 ext(3506176->3514367) 00010000:00010000:1.0:1780298645.307321:0:19951:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e63035b9e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3489792->37044223] (req 3489792->3493887) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a68 expref: 16 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.307330:0:19951:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e63035b9e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3489792->37044223] (req 3489792->3493887) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a68 expref: 16 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1780298645.307335:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf2558f2680 00000100:00100000:2.0:1780298645.307985:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.308035:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609680128 00000100:00100000:0.0:1780298645.308039:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609680128 00010000:00010000:0.0:1780298645.308051:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e63035b9e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3489792->37044223] (req 3489792->3493887) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a68 expref: 16 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.308072:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15271 00000100:00100000:0.0:1780298645.308077:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f0700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609680128:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298645.308083:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:0.0:1780298645.308090:0:6461:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 512 -> 528 00010000:00010000:0.0:1780298645.308094:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e63035b9e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3489792->37044223] (req 3489792->3493887) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a68 expref: 16 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.308101:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e63035b9e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3489792->37044223] (req 3489792->3493887) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a68 expref: 16 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.308111:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002f6fb1b9/0xaf35132e63035b9e lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3489792->37044223] (req 3489792->3493887) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a68 expref: 16 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.308119:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(47 169) 00010000:00010000:0.0:1780298645.308121:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(47 169) 00010000:00010000:0.0:1780298645.308145:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e63035ccb lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b8e expref: 15 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.308152:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e63035ccb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b8e expref: 15 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.308179:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e63035ccb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b8e expref: 15 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.308190:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255090700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777605084288:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298645.308233:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1d80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.308815:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255090700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777605084288:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298645.308823:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298645.308844:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f0700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2397:x1866777609680128:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 767us (863us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.308849:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15271 00000100:00100000:3.0:1780298645.309011:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.309058:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609680256 00000100:00100000:2.0:1780298645.309066:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609680256 00000100:00100000:2.0:1780298645.309112:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10295 00000100:00100000:2.0:1780298645.309155:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b01880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:29360:x1866777609680256:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.309170:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.309183:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e63035cd2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4b95 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.309205:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e63035c69 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [524288->1998847] (req 524288->536575) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b33 expref: 18 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.309215:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e63035bcf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [1998848->2031615] (req 1998848->2011135) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a99 expref: 18 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.309220:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63035bf2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [3764224->37318655] (req 3764224->3772415) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4abc expref: 11 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.309224:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 3 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(44 122) 00010000:00010000:2.0:1780298645.309227:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(47 122) 00010000:00010000:2.0:1780298645.309230:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e63035cd2 lrc: 6/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b95 expref: 18 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.309251:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63035bf2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [3764224->37318655] (req 3764224->3772415) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4abc expref: 11 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.309260:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63035bf2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [3764224->37318655] (req 3764224->3772415) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4abc expref: 11 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.309269:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605084416:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298645.309339:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e63035bcf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [1998848->2031615] (req 1998848->2011135) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a99 expref: 18 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.309347:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e63035bcf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [1998848->2031615] (req 1998848->2011135) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a99 expref: 18 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.309357:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605084544:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298645.309382:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e63035c69 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [524288->1998847] (req 524288->536575) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b33 expref: 18 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.309389:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e63035c69 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [524288->1998847] (req 524288->536575) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b33 expref: 18 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.309396:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f3100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605084672:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298645.309409:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c900 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.310074:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605084416:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298645.310089:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605084544:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298645.310279:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f3100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605084672:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298645.310307:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:3.0:1780298645.310315:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e63035cd2 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b95 expref: 18 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.310327:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3600, rc 0) 00000100:00100000:1.0:1780298645.310351:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609680512 00000100:00100000:1.0:1780298645.310356:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609680512 00000100:00100000:0.0:1780298645.310370:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:1.0:1780298645.310375:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e63035bcf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [1998848->2031615] (req 1998848->2011135) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a99 expref: 19 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.310403:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15272 00000100:00100000:3.0:1780298645.310417:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b01880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:29360:x1866777609680256:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1294us (1401us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298645.310420:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3487c5c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2400:x1866777609680512:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298645.310435:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1780298645.310436:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10295 00010000:00010000:1.0:1780298645.310448:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e63035bcf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [1998848->2031615] (req 1998848->2011135) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a99 expref: 19 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.310455:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e63035bcf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [1998848->2031615] (req 1998848->2011135) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a99 expref: 19 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.310467:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000909e5905/0xaf35132e63035bcf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [1998848->2031615] (req 1998848->2011135) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4a99 expref: 19 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.310475:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298645.310497:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3487c5c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609680512:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 90us (198us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298645.310503:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15272 00000100:00100000:1.0:1780298645.310557:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609680384 00000100:00100000:1.0:1780298645.310560:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609680384 00002000:00100000:1.0:1780298645.310574:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x12:0x0] (3764224->3772415) req@ffff9cf255090700 x1866777609680384/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 8680/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.310580:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255090700 with x1866777609680384 ext(3764224->3772415) 00010000:00010000:1.0:1780298645.310584:0:8363:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63035bf2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3764224->37318655] (req 3764224->3772415) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4abc expref: 12 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.310590:0:8363:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63035bf2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3764224->37318655] (req 3764224->3772415) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4abc expref: 12 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1780298645.310595:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9cf255090700 00000100:00100000:1.0:1780298645.310608:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 215 00000100:00100000:1.0:1780298645.310612:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255090700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609680384:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:0.0:1780298645.310670:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.310701:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609680640 00000100:00100000:2.0:1780298645.310706:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609680640 00010000:00010000:2.0:1780298645.310727:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e63035c69 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [524288->1998847] (req 524288->536575) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b33 expref: 17 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.310754:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15273 00000100:00100000:2.0:1780298645.310761:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255091f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2399:x1866777609680640:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298645.310770:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:2.0:1780298645.310782:0:6462:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 72 -> 88 00010000:00010000:2.0:1780298645.310787:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e63035c69 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [524288->1998847] (req 524288->536575) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b33 expref: 17 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.310798:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e63035c69 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [524288->1998847] (req 524288->536575) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b33 expref: 17 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.310812:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b93eb96b/0xaf35132e63035c69 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [524288->1998847] (req 524288->536575) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b33 expref: 17 pid: 9128 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.310824:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298645.310858:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255091f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2399:x1866777609680640:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 99us (188us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.310867:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15273 00000100:00100000:1.0:1780298645.311057:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255090700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609680384:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 442us (686us total) trans 4294969313 rc 0/0 00000100:00100000:1.0:1780298645.311064:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 215 00002000:00100000:1.0:1780298645.311070:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x12:0x0] (3764224->3772415) req@ffff9cf255090700 x1866777609680384/t4294969313(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 8680/448 e 0 to 0 dl 1780298656 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.311075:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255090700 with x1866777609680384 ext(3764224->3772415) 00010000:00010000:1.0:1780298645.311080:0:8363:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63035bf2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3764224->37318655] (req 3764224->3772415) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4abc expref: 14 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.311089:0:8363:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63035bf2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3764224->37318655] (req 3764224->3772415) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4abc expref: 14 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1780298645.311094:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9cf255090700 00000100:00100000:0.0:1780298645.311855:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.311887:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609680768 00000100:00100000:2.0:1780298645.311894:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609680768 00010000:00010000:2.0:1780298645.311909:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63035bf2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3764224->37318655] (req 3764224->3772415) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4abc expref: 14 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.311930:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15274 00000100:00100000:2.0:1780298645.311936:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255490e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609680768:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298645.311943:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:2.0:1780298645.311949:0:7400:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 88 -> 96 00010000:00010000:2.0:1780298645.311953:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63035bf2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3764224->37318655] (req 3764224->3772415) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4abc expref: 14 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.311961:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63035bf2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3764224->37318655] (req 3764224->3772415) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4abc expref: 14 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.311971:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000190405/0xaf35132e63035bf2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3764224->37318655] (req 3764224->3772415) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4abc expref: 14 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.311982:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(47 122) 00010000:00010000:2.0:1780298645.311985:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(47 122) 00010000:00010000:2.0:1780298645.311989:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e63035cd2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b95 expref: 15 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.311996:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e63035cd2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b95 expref: 15 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.312016:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e63035cd2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b95 expref: 15 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.312024:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f2d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:7400:1866777605084800:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298645.312065:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c840 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.312795:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f2d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:7400:1866777605084800:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298645.312806:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298645.312832:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255490e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609680768:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 897us (979us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.312838:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15274 00000100:00100000:2.0:1780298645.312885:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.312923:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609680896 00000100:00100000:0.0:1780298645.312928:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609680896 00002000:00010000:0.0:1780298645.312944:0:19950:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (2960272->18446744073709551615) 00002000:00100000:0.0:1780298645.312947:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2558f0e00 with x1866777609680896 ext(2960272->18446744073709551615) 00002000:00010000:0.0:1780298645.312955:0:19950:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2558f0e00. 00000100:00100000:0.0:1780298645.312980:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6442 00000100:00100000:0.0:1780298645.312994:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2400:x1866777609680896:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:1.0:1780298645.313032:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.313070:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609681024 00000100:00100000:2.0:1780298645.313077:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609681024 00002000:00010000:2.0:1780298645.313104:0:19943:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00002000:00100000:2.0:1780298645.313109:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481f9180 with x1866777609681024 ext(0->18446744073709551615) 00002000:00010000:2.0:1780298645.313116:0:19943:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf3481f9180. 00000100:00100000:3.0:1780298645.313189:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6443 00000100:00100000:3.0:1780298645.313204:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609681024:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:0.0:1780298645.313338:0:19950:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4618229 -> 2960272 00002000:00010000:0.0:1780298645.313342:0:19950:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298645.313345:0:19950:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298645.313347:0:19950:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298645.313350:0:19950:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 528 -> 472 00000100:00100000:0.0:1780298645.313383:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609680896:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 395us (500us total) trans 4294969769 rc 0/0 00000100:00100000:0.0:1780298645.313390:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6442 00002000:00010000:0.0:1780298645.313396:0:19950:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (2960272->18446744073709551615) 00002000:00100000:0.0:1780298645.313400:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2558f0e00 with x1866777609680896 ext(2960272->18446744073709551615) 00002000:00010000:0.0:1780298645.313405:0:19950:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2558f0e00. 00002000:00010000:3.0:1780298645.313539:0:19928:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 4194304 -> 0 00002000:00010000:3.0:1780298645.313542:0:19928:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:3.0:1780298645.313544:0:19928:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:3.0:1780298645.313545:0:19928:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:3.0:1780298645.313547:0:19928:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 96 -> 0 00000100:00100000:3.0:1780298645.313577:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609681024:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 378us (545us total) trans 4294969314 rc 0/0 00000100:00100000:3.0:1780298645.313585:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6443 00002000:00010000:3.0:1780298645.313589:0:19928:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00002000:00100000:3.0:1780298645.313593:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481f9180 with x1866777609681024 ext(0->18446744073709551615) 00002000:00010000:3.0:1780298645.313598:0:19928:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf3481f9180. 00000100:00100000:1.0:1780298645.314339:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.314371:0:17567:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609681152 00000100:00100000:0.0:1780298645.314377:0:17567:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609681152 00000100:00100000:0.0:1780298645.314406:0:17567:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62066 00000100:00100000:0.0:1780298645.314411:0:17567:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+34:29360:x1866777609681152:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.314420:0:17567:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.314433:0:17567:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae400/0xaf35132e63035cd9 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 12 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4b9c expref: -99 pid: 17567 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.314498:0:17567:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae200/0xaf35132e63035ce0 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.314508:0:17567:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae200/0xaf35132e63035ce0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.314515:0:17567:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae200/0xaf35132e63035ce0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1780298645.314534:0:17567:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 2960272 00000004:00200000:0.0:1780298645.314537:0:17567:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1780298645.314552:0:17567:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae200/0xaf35132e63035ce0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.314576:0:17567:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae400/0xaf35132e63035cd9 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 13 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b9c expref: 36 pid: 17567 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.314587:0:17567:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae200/0xaf35132e63035ce0 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 12 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b9c expref: 35 pid: 17567 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.314597:0:17567:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298645.314598:0:17567:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1780298645.314600:0:17567:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae200, rc 0) 00000100:00100000:0.0:1780298645.314622:0:17567:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+35:29360:x1866777609681152:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 212us (284us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.314627:0:17567:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62066 00000100:00100000:2.0:1780298645.315577:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.315614:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609681280 00000100:00100000:1.0:1780298645.315618:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609681280 00000100:00100000:2.0:1780298645.315644:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.315663:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609681408 00000100:00100000:1.0:1780298645.315669:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609681408 00000100:00100000:0.0:1780298645.315681:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10296 00000100:00100000:0.0:1780298645.315689:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609681280:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.315699:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298645.315701:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10297 00010000:00010000:0.0:1780298645.315709:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63035ce7 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4ba3 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.315711:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2399:x1866777609681408:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.315718:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.315725:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63035cee lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4baa expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.315752:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605084928:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.315752:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255490380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605085056:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.315805:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.315805:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1d80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.316655:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605084928:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.316671:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255490380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605085056:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.316675:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63035cee lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4baa expref: 14 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.316689:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63035ce7 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4ba3 expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.316691:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63035cee lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4baa expref: 14 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.316711:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278a00, rc 0) 00010000:00010000:0.0:1780298645.316713:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63035ce7 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4ba3 expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.316722:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baee00, rc 0) 00000100:00100000:1.0:1780298645.316741:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2399:x1866777609681408:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1033us (1097us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.316744:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609681280:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1054us (1169us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.316752:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10297 00000100:00100000:0.0:1780298645.316764:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10296 00000100:00100000:3.0:1780298645.317752:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.317778:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.317791:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609681536 00000100:00100000:1.0:1780298645.317796:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609681536 00000100:00100000:0.0:1780298645.317822:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609681664 00000100:00100000:0.0:1780298645.317828:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609681664 00000100:00100000:1.0:1780298645.317829:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10298 00000100:00100000:1.0:1780298645.317840:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b01f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609681536:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.317847:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.317855:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63035cf5 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4bb1 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.317866:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10299 00000100:00100000:0.0:1780298645.317873:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b03100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2399:x1866777609681664:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.317880:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298645.317883:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605085184:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.317889:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a3f03a/0xaf35132e63035cfc lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4bb8 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.317924:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.317924:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255490380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605085312:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.317956:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1d80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.318726:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255490380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605085312:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.318740:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000001a3f03a/0xaf35132e63035cfc lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4bb8 expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.318747:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f8700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605085184:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.318753:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a3f03a/0xaf35132e63035cfc lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4bb8 expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.318763:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae600, rc 0) 00010000:00010000:1.0:1780298645.318772:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63035cf5 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4bb1 expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.318787:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63035cf5 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4bb1 expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.318790:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b03100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2399:x1866777609681664:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 917us (1011us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.318805:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10299 00010000:00010000:1.0:1780298645.318818:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278c00, rc 0) 00000100:00100000:1.0:1780298645.318843:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b01f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609681536:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1005us (1092us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.318852:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10298 00000100:00100000:2.0:1780298645.319877:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.319910:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609681792 00000100:00100000:1.0:1780298645.319915:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609681792 00000100:00100000:1.0:1780298645.319948:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6444 00000100:00100000:1.0:1780298645.319955:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2399:x1866777609681792:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298645.320095:0:16972:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 2960272 -> 2960272 00002000:00010000:1.0:1780298645.320098:0:16972:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298645.320100:0:16972:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298645.320101:0:16972:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298645.320102:0:16972:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 472 -> 472 00000100:00100000:1.0:1780298645.320152:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2399:x1866777609681792:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 197us (275us total) trans 4294969770 rc 0/0 00000100:00100000:1.0:1780298645.320160:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6444 00000100:00100000:3.0:1780298645.320805:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.320839:0:9976:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609681920 00000100:00100000:2.0:1780298645.320843:0:9976:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609681920 00000100:00100000:2.0:1780298645.320871:0:9976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62067 00000100:00100000:2.0:1780298645.320877:0:9976:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b03800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+29:29360:x1866777609681920:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.320890:0:9976:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.320900:0:9976:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3200/0xaf35132e63035d03 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4bbf expref: -99 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.320958:0:9976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2a00/0xaf35132e63035d0a lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 14 type: IBT flags: 0x40000000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.320968:0:9976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2a00/0xaf35132e63035d0a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 14 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.320975:0:9976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2a00/0xaf35132e63035d0a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 14 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298645.320994:0:9976:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 2960272 00000004:00200000:2.0:1780298645.320997:0:9976:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298645.321010:0:9976:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2a00/0xaf35132e63035d0a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 14 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.321026:0:9976:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb3200/0xaf35132e63035d03 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 14 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4bbf expref: 31 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.321041:0:9976:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2a00/0xaf35132e63035d0a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4bbf expref: 30 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.321049:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298645.321053:0:9976:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298645.321055:0:9976:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2a00, rc 0) 00000100:00100000:2.0:1780298645.321085:0:9976:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b03800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+30:29360:x1866777609681920:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 207us (280us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.321096:0:9976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62067 00000100:00100000:2.0:1780298645.322484:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.322508:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.322521:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609682048 00000100:00100000:3.0:1780298645.322526:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609682048 00000100:00100000:1.0:1780298645.322538:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609682176 00000100:00100000:1.0:1780298645.322542:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609682176 00000100:00100000:3.0:1780298645.322556:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10300 00000100:00100000:3.0:1780298645.322564:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609682048:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.322572:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.322580:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000042c16ab0/0xaf35132e63035d11 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4bc6 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.322587:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10301 00000100:00100000:0.0:1780298645.322599:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2399:x1866777609682176:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.322607:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.322616:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63035d18 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4bcd expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.322619:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b01c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605085440:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.322651:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255490380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605085568:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.322676:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856180 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.322682:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1d80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.323443:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b01c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605085440:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.323458:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255490380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605085568:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.323474:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000042c16ab0/0xaf35132e63035d11 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4bc6 expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.323477:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63035d18 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4bcd expref: 14 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.323503:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000042c16ab0/0xaf35132e63035d11 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4bc6 expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.323503:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63035d18 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4bcd expref: 14 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.323524:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae400, rc 0) 00010000:00010000:1.0:1780298645.323532:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156800, rc 0) 00000100:00100000:0.0:1780298645.323553:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2399:x1866777609682176:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 957us (1043us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.323559:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609682048:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 993us (1072us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.323563:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10301 00000100:00100000:1.0:1780298645.323573:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10300 00000100:00100000:1.0:1780298645.324532:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.324555:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.324575:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609682304 00000100:00100000:2.0:1780298645.324581:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609682304 00000100:00100000:0.0:1780298645.324590:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609682432 00000100:00100000:0.0:1780298645.324594:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609682432 00000100:00100000:2.0:1780298645.324620:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10302 00000100:00100000:0.0:1780298645.324628:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10303 00000100:00100000:2.0:1780298645.324633:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609682304:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298645.324641:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2398:x1866777609682432:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.324643:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.324652:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.324655:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005c47b04a/0xaf35132e63035d1f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4bd4 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.324661:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002ee949fd/0xaf35132e63035d26 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4bdb expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.324694:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf348318a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605085696:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.324699:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255490380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605085824:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.324749:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1d80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.324750:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.325478:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348318a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605085696:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.325496:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005c47b04a/0xaf35132e63035d1f lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4bd4 expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.325500:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255490380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605085824:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.325516:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005c47b04a/0xaf35132e63035d1f lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4bd4 expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.325524:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3400, rc 0) 00010000:00010000:1.0:1780298645.325527:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002ee949fd/0xaf35132e63035d26 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4bdb expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.325539:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002ee949fd/0xaf35132e63035d26 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4bdb expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.325547:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bafc00, rc 0) 00000100:00100000:0.0:1780298645.325552:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609682304:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 924us (1018us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.325558:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10302 00000100:00100000:1.0:1780298645.325570:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2398:x1866777609682432:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 931us (1012us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.325577:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10303 00000100:00100000:3.0:1780298645.327632:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.327663:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.327683:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609682560 00000100:00100000:2.0:1780298645.327690:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609682560 00000100:00100000:0.0:1780298645.327702:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609682688 00000100:00100000:0.0:1780298645.327708:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609682688 00000100:00100000:2.0:1780298645.327738:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10304 00000100:00100000:2.0:1780298645.327752:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b01c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609682560:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298645.327753:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10305 00010000:00010000:2.0:1780298645.327766:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298645.327766:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609682688:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.327775:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.327780:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e63035d2d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4be2 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.327785:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c37b15a2/0xaf35132e63035d34 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4be9 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.327827:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34831a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605085952:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.327830:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255490380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605086080:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.327892:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.327892:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.328776:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255490380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605086080:192.168.203.25@tcp:106: 00002000:00010000:2.0:1780298645.328785:0:8356:0:(ofd_lvb.c:259:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size: 2960272 -> 4149789 00000100:00100000:2.0:1780298645.328800:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34831a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605085952:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.328800:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c37b15a2/0xaf35132e63035d34 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4be9 expref: 14 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.328825:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c37b15a2/0xaf35132e63035d34 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4be9 expref: 14 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.328826:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e63035d2d lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4be2 expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.328850:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baee00, rc 0) 00010000:00010000:2.0:1780298645.328855:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000039c5f58c/0xaf35132e63035d2d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4be2 expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.328867:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb3200, rc 0) 00000100:00100000:0.0:1780298645.328886:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609682688:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1124us (1224us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.328895:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b01c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609682560:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1147us (1264us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.328898:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10305 00000100:00100000:2.0:1780298645.328908:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10304 00000100:00100000:3.0:1780298645.329977:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.330018:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609682816 00000100:00100000:1.0:1780298645.330023:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609682816 00000100:00100000:1.0:1780298645.330054:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10306 00000100:00100000:1.0:1780298645.330062:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b00000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609682816:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.330070:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.330079:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63035d3b lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4bf0 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.330099:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.330110:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481faa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605086208:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.330149:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609682944 00000100:00100000:0.0:1780298645.330154:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609682944 00000100:00100000:1.0:1780298645.330174:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.330183:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10307 00000100:00100000:0.0:1780298645.330190:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2398:x1866777609682944:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.330197:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.330203:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63035d42 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4bf7 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.330225:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255490380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605086336:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.330249:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.330853:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481faa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605086208:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.330878:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63035d3b lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4bf0 expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.330895:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63035d3b lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4bf0 expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.330905:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278a00, rc 0) 00000100:00100000:1.0:1780298645.330933:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b00000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609682816:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 873us (957us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.330940:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10306 00000100:00100000:0.0:1780298645.330945:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255490380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605086336:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.330960:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63035d42 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4bf7 expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.330973:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000033b7ed0c/0xaf35132e63035d42 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4bf7 expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.330980:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae000, rc 0) 00000100:00100000:0.0:1780298645.330996:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2398:x1866777609682944:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 808us (898us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.331001:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10307 00000100:00100000:2.0:1780298645.332003:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.332039:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609683072 00000100:00100000:1.0:1780298645.332043:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609683072 00000100:00100000:1.0:1780298645.332085:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6445 00000100:00100000:1.0:1780298645.332093:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34831a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609683072:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298645.332378:0:19951:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4149789 -> 2960272 00002000:00010000:1.0:1780298645.332381:0:19951:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298645.332383:0:19951:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298645.332385:0:19951:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298645.332386:0:19951:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 472 -> 496 00000100:00100000:1.0:1780298645.332410:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34831a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609683072:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 319us (409us total) trans 4294969771 rc 0/0 00000100:00100000:1.0:1780298645.332415:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6445 00000100:00100000:3.0:1780298645.333582:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.333609:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.333630:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609683200 00000100:00100000:1.0:1780298645.333636:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609683200 00000100:00100000:0.0:1780298645.333644:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609683328 00000100:00100000:0.0:1780298645.333648:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609683328 00000100:00100000:1.0:1780298645.333676:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10308 00000100:00100000:0.0:1780298645.333682:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10309 00000100:00100000:1.0:1780298645.333688:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609683200:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298645.333692:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2398:x1866777609683328:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.333700:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.333700:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.333714:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63035d49 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4bfe expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.333714:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e63035d50 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4c05 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.333767:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255490380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605086464:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.333770:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481faa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605086592:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.333809:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.333811:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e3c0 going to sleep for 11 seconds 00002000:00010000:1.0:1780298645.334625:0:8359:0:(ofd_lvb.c:259:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size: 2960272 -> 4149789 00000100:00100000:0.0:1780298645.334628:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255490380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605086464:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.334636:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481faa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605086592:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.334652:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e63035d50 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c05 expref: 14 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.334656:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63035d49 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4bfe expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.334683:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63035d49 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4bfe expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.334684:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000058e2d0/0xaf35132e63035d50 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c05 expref: 14 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.334699:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279200, rc 0) 00010000:00010000:0.0:1780298645.334711:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baea00, rc 0) 00000100:00100000:1.0:1780298645.334727:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609683200:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1044us (1147us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.334732:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10308 00000100:00100000:0.0:1780298645.334739:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2398:x1866777609683328:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1049us (1127us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.334746:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10309 00000100:00100000:2.0:1780298645.335769:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.335801:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609683456 00000100:00100000:1.0:1780298645.335808:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609683456 00000100:00100000:1.0:1780298645.335837:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10310 00000100:00100000:1.0:1780298645.335842:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34831aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:29360:x1866777609683456:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.335849:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.335857:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63035d57 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4c0c expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.335871:0:19992:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e63035ccb lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b8e expref: 17 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.335878:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(47 169) 00010000:00010000:1.0:1780298645.335880:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(48 169) 00010000:00010000:1.0:1780298645.335884:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63035d57 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [24576->28671] (req 24576->28671) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c0c expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.335898:0:19992:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e63035ccb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b8e expref: 17 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.335908:0:19992:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e63035ccb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b8e expref: 17 pid: 8359 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.335916:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481faa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605086720:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298645.335950:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.336626:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481faa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605086720:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298645.336647:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63035d57 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [24576->28671] (req 24576->28671) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c0c expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.336666:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279a00, rc 0) 00000100:00100000:1.0:1780298645.336698:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34831aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:29360:x1866777609683456:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 855us (931us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298645.336706:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10310 00000100:00100000:3.0:1780298645.336950:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.336990:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609683584 00000100:00100000:1.0:1780298645.336995:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609683584 00002000:00100000:1.0:1780298645.337025:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (4149248->4149788) req@ffff9cf2425f1c00 x1866777609683584/t0(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 1032/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.337035:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2425f1c00 with x1866777609683584 ext(4149248->4149788) 00010000:00010000:1.0:1780298645.337043:0:8363:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e63035ccb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b8e expref: 18 pid: 8359 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.337055:0:8363:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e63035ccb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b8e expref: 18 pid: 8359 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1780298645.337063:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf2425f1c00 00000100:00100000:1.0:1780298645.337086:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 216 00000100:00100000:1.0:1780298645.337095:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609683584:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:1.0:1780298645.337622:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2400:x1866777609683584:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 528us (673us total) trans 4294969772 rc 0/0 00000100:00100000:1.0:1780298645.337630:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 216 00002000:00100000:1.0:1780298645.337638:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (4149248->4149788) req@ffff9cf2425f1c00 x1866777609683584/t4294969772(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 1032/448 e 0 to 0 dl 1780298656 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.337658:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2425f1c00 with x1866777609683584 ext(4149248->4149788) 00010000:00010000:1.0:1780298645.337664:0:8363:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e63035ccb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b8e expref: 20 pid: 8359 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.337678:0:8363:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e63035ccb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b8e expref: 20 pid: 8359 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1780298645.337686:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf2425f1c00 00000100:00100000:2.0:1780298645.338445:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.338487:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609683712 00000100:00100000:0.0:1780298645.338492:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609683712 00010000:00010000:0.0:1780298645.338508:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e63035ccb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b8e expref: 20 pid: 8359 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.338529:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15275 00000100:00100000:0.0:1780298645.338534:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348318700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2398:x1866777609683712:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298645.338542:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:0.0:1780298645.338552:0:20253:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 496 -> 504 00010000:00010000:0.0:1780298645.338556:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e63035ccb lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b8e expref: 20 pid: 8359 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.338564:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e63035ccb lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b8e expref: 20 pid: 8359 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.338584:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bed20494/0xaf35132e63035ccb lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b8e expref: 20 pid: 8359 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.338594:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(48 169) 00010000:00010000:0.0:1780298645.338597:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(48 169) 00010000:00010000:0.0:1780298645.338602:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63035d57 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [24576->33579007] (req 24576->28671) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c0c expref: 8 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.338610:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63035d57 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [24576->33579007] (req 24576->28671) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c0c expref: 8 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.338629:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63035d57 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [24576->33579007] (req 24576->28671) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c0c expref: 8 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.338637:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255490380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777605086848:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298645.338675:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.339357:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255490380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777605086848:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298645.339371:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298645.339399:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348318700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2398:x1866777609683712:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 863us (955us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.339409:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15275 00000100:00100000:3.0:1780298645.339616:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.339655:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609683840 00000100:00100000:2.0:1780298645.339660:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609683840 00002000:00100000:2.0:1780298645.339681:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (24576->28671) req@ffff9cf2425f0000 x1866777609683840/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.339688:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2425f0000 with x1866777609683840 ext(24576->28671) 00002000:00010000:2.0:1780298645.339690:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2425f0000 00000100:00100000:2.0:1780298645.339706:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6446 00000100:00100000:2.0:1780298645.339711:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609683840:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298645.339814:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609683840:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 102us (200us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298645.339822:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6446 00002000:00100000:2.0:1780298645.339827:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (24576->28671) req@ffff9cf2425f0000 x1866777609683840/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/4536 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.339833:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2425f0000 with x1866777609683840 ext(24576->28671) 00002000:00010000:2.0:1780298645.339836:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2425f0000 00000100:00100000:2.0:1780298645.341037:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.341085:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609683968 00000100:00100000:3.0:1780298645.341091:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609683968 00002000:00100000:3.0:1780298645.341116:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (28672->32767) req@ffff9cf34831bb80 x1866777609683968/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298645.341152:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf34831bb80 with x1866777609683968 ext(28672->32767) 00002000:00010000:3.0:1780298645.341157:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf34831bb80 00000100:00100000:3.0:1780298645.341180:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6447 00000100:00100000:3.0:1780298645.341190:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34831bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609683968:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298645.341300:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34831bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609683968:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 113us (265us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1780298645.341308:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6447 00002000:00100000:3.0:1780298645.341316:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (28672->32767) req@ffff9cf34831bb80 x1866777609683968/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/4536 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298645.341325:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf34831bb80 with x1866777609683968 ext(28672->32767) 00002000:00010000:3.0:1780298645.341328:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf34831bb80 00000100:00100000:3.0:1780298645.342328:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.342360:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609684096 00000100:00100000:1.0:1780298645.342364:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609684096 00002000:00100000:1.0:1780298645.342383:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (32768->36863) req@ffff9cf2425f0a80 x1866777609684096/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.342389:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2425f0a80 with x1866777609684096 ext(32768->36863) 00002000:00010000:1.0:1780298645.342398:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2425f0a80 00000100:00100000:1.0:1780298645.342413:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6448 00000100:00100000:1.0:1780298645.342419:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609684096:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298645.342487:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609684096:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 68us (161us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298645.342491:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6448 00002000:00100000:1.0:1780298645.342495:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (32768->36863) req@ffff9cf2425f0a80 x1866777609684096/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/4536 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.342501:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2425f0a80 with x1866777609684096 ext(32768->36863) 00002000:00010000:1.0:1780298645.342504:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2425f0a80 00000100:00100000:2.0:1780298645.343756:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.343787:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609684224 00000100:00100000:1.0:1780298645.343791:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609684224 00002000:00100000:1.0:1780298645.343813:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x2c0000401:0x11:0x0] (24576->36863) req@ffff9cf348318a80 x1866777609684224/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 12776/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.343820:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf348318a80 with x1866777609684224 ext(24576->36863) 00002000:00010000:1.0:1780298645.343823:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf348318a80 00000100:00100000:1.0:1780298645.343842:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6449 00000100:00100000:1.0:1780298645.343850:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348318a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609684224:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:1.0:1780298645.344529:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348318a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609684224:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 680us (774us total) trans 4294969773 rc 0/0 00000100:00100000:1.0:1780298645.344534:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6449 00002000:00100000:1.0:1780298645.344539:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x2c0000401:0x11:0x0] (24576->36863) req@ffff9cf348318a80 x1866777609684224/t4294969773(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 12776/448 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.344545:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf348318a80 with x1866777609684224 ext(24576->36863) 00002000:00010000:1.0:1780298645.344548:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf348318a80 00000100:00100000:2.0:1780298645.345368:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.345399:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609684352 00000100:00100000:0.0:1780298645.345404:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609684352 00000100:00100000:0.0:1780298645.345433:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10311 00000100:00100000:0.0:1780298645.345441:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609684352:12345-192.168.203.25@tcp:16:fsx.0 00000001:00080000:2.0:1780298645.346237:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969773 is committed 00000001:00080000:2.0:1780298645.346248:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969772 is committed 00000100:00100000:0.0:1780298645.346357:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609684352:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 917us (989us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.346363:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10311 00000100:00100000:3.0:1780298645.347433:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.347519:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.347596:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609684480 00000100:00100000:1.0:1780298645.347597:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609684608 00000100:00100000:1.0:1780298645.347604:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609684608 00000100:00100000:0.0:1780298645.347604:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609684480 00000100:00100000:1.0:1780298645.347653:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10312 00000100:00100000:1.0:1780298645.347664:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2397:x1866777609684608:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298645.347669:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10313 00010000:00010000:1.0:1780298645.347680:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298645.347682:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609684480:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.347691:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.347696:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63035d5e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4c1a expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.347702:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000058e2d0/0xaf35132e63035d65 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4c13 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.347752:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481faa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605086976:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.347753:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255490380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605087104:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.347831:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e480 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.347831:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.348648:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255490380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605087104:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.348659:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481faa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605086976:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.348679:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63035d5e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c1a expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.348680:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000058e2d0/0xaf35132e63035d65 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c13 expref: 8 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.348705:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63035d5e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c1a expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.348709:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000058e2d0/0xaf35132e63035d65 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c13 expref: 8 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.348717:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279200, rc 0) 00010000:00010000:0.0:1780298645.348722:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baea00, rc 0) 00000100:00100000:1.0:1780298645.348744:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2397:x1866777609684608:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1084us (1224us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.348750:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10312 00000100:00100000:0.0:1780298645.348760:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609684480:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1072us (1319us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.348768:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10313 00000100:00100000:2.0:1780298645.349803:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.349840:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609684736 00000100:00100000:2.0:1780298645.349845:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.349845:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609684736 00000100:00100000:1.0:1780298645.349883:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609684864 00000100:00100000:1.0:1780298645.349887:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609684864 00000100:00100000:0.0:1780298645.349904:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10314 00000100:00100000:0.0:1780298645.349914:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609684736:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298645.349920:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10315 00010000:00010000:0.0:1780298645.349923:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.349931:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e63035d6c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4c21 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.349933:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2398:x1866777609684864:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.349947:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.349956:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63035d73 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4c28 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.349964:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255490380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605087232:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.349992:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481faa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605087360:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.350015:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.350025:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e480 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.350674:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255490380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605087232:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.350686:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e63035d6c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c21 expref: 8 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.350696:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000033b7ed0c/0xaf35132e63035d6c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c21 expref: 8 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.350707:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae000, rc 0) 00000100:00100000:0.0:1780298645.350738:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609684736:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 825us (933us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.350746:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10314 00000100:00100000:1.0:1780298645.350844:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481faa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605087360:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.350857:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63035d73 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c28 expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.350871:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63035d73 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c28 expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.350877:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278a00, rc 0) 00000100:00100000:1.0:1780298645.350898:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2398:x1866777609684864:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 972us (1053us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.350903:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10315 00000100:00100000:3.0:1780298645.351947:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.351997:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609684992 00000100:00100000:1.0:1780298645.352001:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609684992 00000100:00100000:1.0:1780298645.352032:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6450 00000100:00100000:1.0:1780298645.352038:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609684992:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298645.352218:0:8363:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4149789 -> 4149789 00002000:00010000:1.0:1780298645.352221:0:8363:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298645.352223:0:8363:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298645.352225:0:8363:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298645.352226:0:8363:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 504 -> 504 00000100:00100000:1.0:1780298645.352249:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609684992:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 211us (303us total) trans 4294969774 rc 0/0 00000100:00100000:1.0:1780298645.352254:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6450 00000100:00100000:3.0:1780298645.353187:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.353211:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609685120 00000100:00100000:3.0:1780298645.353214:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.353216:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609685120 00000100:00100000:1.0:1780298645.353244:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609685248 00000100:00100000:0.0:1780298645.353248:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10316 00000100:00100000:1.0:1780298645.353250:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609685248 00000100:00100000:0.0:1780298645.353254:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609685120:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.353265:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.353281:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63035d7a lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4c2f expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.353288:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10317 00000100:00100000:1.0:1780298645.353299:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2398:x1866777609685248:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.353307:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298645.353312:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255490380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605087488:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.353315:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6416acb/0xaf35132e63035d81 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4c36 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.353340:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481faa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605087616:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.353347:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.353365:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e480 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.354161:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481faa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605087616:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.354175:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255490380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605087488:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.354177:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a6416acb/0xaf35132e63035d81 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c36 expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.354191:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6416acb/0xaf35132e63035d81 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c36 expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.354197:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278c00, rc 0) 00010000:00010000:0.0:1780298645.354198:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63035d7a lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c2f expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.354218:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63035d7a lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c2f expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.354224:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2398:x1866777609685248:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 926us (1010us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.354236:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10317 00010000:00010000:0.0:1780298645.354239:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baee00, rc 0) 00000100:00100000:0.0:1780298645.354261:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609685120:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1007us (1076us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.354265:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10316 00000100:00100000:3.0:1780298645.355329:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.355358:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.355380:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609685376 00000100:00100000:2.0:1780298645.355384:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609685376 00000100:00100000:0.0:1780298645.355386:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609685504 00000100:00100000:0.0:1780298645.355391:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609685504 00000100:00100000:2.0:1780298645.355412:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10318 00000100:00100000:2.0:1780298645.355420:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e424450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609685376:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298645.355426:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10319 00010000:00010000:2.0:1780298645.355430:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298645.355436:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609685504:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.355438:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e63035d88 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4c3d expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.355443:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.355454:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63035d8f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4c44 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.355464:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b2300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605087744:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.355496:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255490380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605087872:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.355505:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c240 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.355526:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.356240:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8b2300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605087744:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.356247:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255490380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605087872:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.356256:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e63035d88 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c3d expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.356263:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63035d8f lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c44 expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.356268:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e63035d88 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c3d expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.356296:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63035d8f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c44 expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.356297:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2400, rc 0) 00010000:00010000:0.0:1780298645.356312:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae400, rc 0) 00000100:00100000:2.0:1780298645.356321:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e424450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609685376:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 904us (993us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.356326:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10318 00000100:00100000:0.0:1780298645.356340:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609685504:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 907us (982us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.356347:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10319 00000100:00100000:3.0:1780298645.357404:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.357446:0:17578:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609685760 00000100:00100000:1.0:1780298645.357451:0:17578:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609685760 00000100:00100000:1.0:1780298645.357488:0:17578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62068 00000100:00100000:1.0:1780298645.357498:0:17578:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+35:29360:x1866777609685760:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:1.0:1780298645.357540:0:17578:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298645.357557:0:17578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae200/0xaf35132e63035ce0 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 14 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b9c expref: 35 pid: 17567 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.357577:0:17578:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298645.357582:0:17578:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298645.357733:0:17578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278e00/0xaf35132e63035d96 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT flags: 0x40000000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.357748:0:17578:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2a00/0xaf35132e63035d0a lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4bbf expref: 29 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.357757:0:17578:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278e00/0xaf35132e63035d96 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.357778:0:17578:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2a00/0xaf35132e63035d0a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4bbf expref: 29 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.357792:0:17578:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2a00/0xaf35132e63035d0a lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4bbf expref: 29 pid: 9976 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.357804:0:17578:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481faa00 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000_UUID:17578:1866777605088000:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298645.357841:0:17578:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.358412:0:17578:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481faa00 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000_UUID:17578:1866777605088000:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298645.358426:0:17578:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278e00/0xaf35132e63035d96 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.358642:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.358676:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609685888 00000100:00100000:0.0:1780298645.358680:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609685888 00010000:00010000:0.0:1780298645.358698:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2a00/0xaf35132e63035d0a lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4bbf expref: 30 pid: 9976 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.358727:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15276 00000100:00100000:0.0:1780298645.358734:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+30:2399:x1866777609685888:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298645.358743:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298645.358748:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2a00/0xaf35132e63035d0a lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 14 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4bbf expref: 30 pid: 9976 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.358759:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2a00/0xaf35132e63035d0a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 14 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4bbf expref: 30 pid: 9976 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.358769:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf244eb2a00/0xaf35132e63035d0a lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 14 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4bbf expref: 30 pid: 9976 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.358778:0:6463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298645.358782:0:6463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278e00/0xaf35132e63035d96 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.358791:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278e00/0xaf35132e63035d96 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 13 type: IBT flags: 0x40290000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.358796:0:6463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278e00/0xaf35132e63035d96 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 13 type: IBT flags: 0x40290000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.358809:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298645.358826:0:17578:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278e00/0xaf35132e63035d96 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 12 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.358831:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+29:2399:x1866777609685888:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 96us (187us total) trans 0 rc 0/0 00010000:00010000:1.0:1780298645.358834:0:17578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278e00/0xaf35132e63035d96 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 12 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.358844:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15276 00000001:00080000:1.0:1780298645.358874:0:17578:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991112 00000004:00010000:1.0:1780298645.358908:0:17578:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000c3b0375c reply state 00000000e64a6782 transno 4294991113 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278e00/0xaf35132e63035d96 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 12 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.358924:0:17578:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278e00/0xaf35132e63035d96 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 12 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.358930:0:17578:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298645.358933:0:17578:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1780298645.358936:0:17578:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1780298645.358996:0:17578:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+37:29360:x1866777609685760:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1500us (1592us total) trans 4294991113 rc 0/0 00000100:00100000:1.0:1780298645.359003:0:17578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62068 00000020:00080000:1.0:1780298645.359008:0:17578:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1780298645.359739:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.359776:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609686016 00000100:00100000:2.0:1780298645.359780:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609686016 00000100:00100000:2.0:1780298645.359803:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10320 00000100:00100000:2.0:1780298645.359807:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:29360:x1866777609686016:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.359813:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.359821:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63035d9d lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4c4b expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.359833:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63035d57 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [24576->33579007] (req 24576->28671) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c0c expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.359839:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(48 169) 00010000:00010000:2.0:1780298645.359842:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(49 169) 00010000:00010000:2.0:1780298645.359845:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63035d9d lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c4b expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.359856:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63035d57 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [24576->33579007] (req 24576->28671) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c0c expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.359863:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63035d57 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [24576->33579007] (req 24576->28671) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c0c expref: 9 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.359870:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605088128:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298645.359906:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c240 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.360479:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8b1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605088128:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298645.360494:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63035d9d lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c4b expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.360506:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2000, rc 0) 00000100:00100000:2.0:1780298645.360529:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:29360:x1866777609686016:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 721us (792us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.360535:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10320 00000100:00100000:3.0:1780298645.360724:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.360759:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609686144 00000100:00100000:0.0:1780298645.360763:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609686144 00010000:00010000:0.0:1780298645.360776:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63035d57 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [24576->33579007] (req 24576->28671) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c0c expref: 9 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.360809:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15277 00000100:00100000:0.0:1780298645.360817:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609686144:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298645.360824:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298645.360834:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63035d57 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [24576->33579007] (req 24576->28671) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c0c expref: 9 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.360842:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63035d57 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [24576->33579007] (req 24576->28671) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c0c expref: 9 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.360852:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63035d57 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [24576->33579007] (req 24576->28671) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c0c expref: 9 pid: 19992 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.360863:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(49 169) 00010000:00010000:0.0:1780298645.360865:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(49 169) 00010000:00010000:0.0:1780298645.360869:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63035d9d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c4b expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.360874:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63035d9d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c4b expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.360896:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63035d9d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c4b expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.360906:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255490380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777605088256:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298645.360941:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.361634:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255490380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777605088256:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298645.361646:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298645.361680:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609686144:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 864us (956us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.361688:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15277 00000100:00100000:3.0:1780298645.361700:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.361746:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609686272 00000100:00100000:1.0:1780298645.361752:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609686272 00000100:00100000:1.0:1780298645.361779:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10321 00000100:00100000:1.0:1780298645.361786:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:29360:x1866777609686272:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.361795:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.361805:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63035da4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4c52 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.361824:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e63035cd2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b95 expref: 16 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.361832:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(47 122) 00010000:00010000:1.0:1780298645.361836:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(48 122) 00010000:00010000:1.0:1780298645.361845:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63035da4 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c52 expref: 14 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.361865:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e63035cd2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b95 expref: 16 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.361876:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e63035cd2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b95 expref: 16 pid: 8356 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.361887:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481faa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605088384:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298645.361923:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e480 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.362506:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481faa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605088384:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298645.362521:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63035da4 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c52 expref: 14 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.362532:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279200, rc 0) 00000100:00100000:1.0:1780298645.362558:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:29360:x1866777609686272:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 772us (858us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298645.362565:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10321 00000100:00100000:3.0:1780298645.362719:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.362754:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609686400 00000100:00100000:1.0:1780298645.362759:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609686400 00010000:00010000:1.0:1780298645.362773:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e63035cd2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b95 expref: 17 pid: 8356 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.362796:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15278 00000100:00100000:1.0:1780298645.362804:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f1500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2399:x1866777609686400:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298645.362814:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298645.362827:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e63035cd2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b95 expref: 17 pid: 8356 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.362834:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e63035cd2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b95 expref: 17 pid: 8356 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.362846:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000cdbb289/0xaf35132e63035cd2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4b95 expref: 17 pid: 8356 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.362853:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(48 122) 00010000:00010000:1.0:1780298645.362855:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(48 122) 00010000:00010000:1.0:1780298645.362858:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63035da4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c52 expref: 13 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.362863:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63035da4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c52 expref: 13 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.362877:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63035da4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c52 expref: 13 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.362884:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481faa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:8374:1866777605088512:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298645.362914:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e480 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.363560:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481faa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:8374:1866777605088512:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298645.363572:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298645.363592:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f1500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2399:x1866777609686400:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 790us (873us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298645.363597:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15278 00000100:00100000:2.0:1780298645.363865:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.363896:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.363920:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609686528 00000100:00100000:0.0:1780298645.363925:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609686528 00000100:00100000:3.0:1780298645.363933:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609686656 00000100:00100000:3.0:1780298645.363939:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609686656 00002000:00010000:0.0:1780298645.363949:0:19943:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (2255272->18446744073709551615) 00002000:00100000:0.0:1780298645.363954:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f8b1c00 with x1866777609686528 ext(2255272->18446744073709551615) 00002000:00010000:3.0:1780298645.363956:0:8364:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00002000:00100000:3.0:1780298645.363961:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f8b0380 with x1866777609686656 ext(0->18446744073709551615) 00002000:00010000:0.0:1780298645.363962:0:19943:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f8b1c00. 00002000:00010000:3.0:1780298645.363968:0:8364:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f8b0380. 00000100:00100000:0.0:1780298645.363992:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6451 00000100:00100000:0.0:1780298645.364003:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609686528:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:3.0:1780298645.364008:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6452 00000100:00100000:3.0:1780298645.364019:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2399:x1866777609686656:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:3.0:1780298645.364150:0:8364:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:3.0:1780298645.364153:0:8364:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:3.0:1780298645.364156:0:8364:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:3.0:1780298645.364158:0:8364:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:3.0:1780298645.364161:0:8364:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:3.0:1780298645.364192:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2399:x1866777609686656:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 175us (295us total) trans 4294969315 rc 0/0 00000100:00100000:3.0:1780298645.364199:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6452 00002000:00010000:3.0:1780298645.364203:0:8364:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00002000:00100000:3.0:1780298645.364207:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f8b0380 with x1866777609686656 ext(0->18446744073709551615) 00002000:00010000:3.0:1780298645.364211:0:8364:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f8b0380. 00002000:00010000:0.0:1780298645.364455:0:19943:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4149789 -> 2255272 00002000:00010000:0.0:1780298645.364459:0:19943:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298645.364462:0:19943:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298645.364464:0:19943:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298645.364466:0:19943:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 504 -> 376 00000100:00100000:0.0:1780298645.364498:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609686528:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 497us (631us total) trans 4294969775 rc 0/0 00000100:00100000:0.0:1780298645.364507:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6451 00002000:00010000:0.0:1780298645.364511:0:19943:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (2255272->18446744073709551615) 00002000:00100000:0.0:1780298645.364514:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f8b1c00 with x1866777609686528 ext(2255272->18446744073709551615) 00002000:00010000:0.0:1780298645.364519:0:19943:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f8b1c00. 00000100:00100000:2.0:1780298645.365310:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.365344:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609686784 00000100:00100000:0.0:1780298645.365348:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609686784 00000100:00100000:0.0:1780298645.365376:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62069 00000100:00100000:0.0:1780298645.365381:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+29:29360:x1866777609686784:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.365391:0:6477:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.365401:0:6477:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae000/0xaf35132e63035dab lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 13 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4c59 expref: -99 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.365478:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baea00/0xaf35132e63035db2 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 14 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.365492:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baea00/0xaf35132e63035db2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 14 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.365508:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baea00/0xaf35132e63035db2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 14 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1780298645.365536:0:6477:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 2255272 00000004:00200000:0.0:1780298645.365540:0:6477:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1780298645.365565:0:6477:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baea00/0xaf35132e63035db2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 14 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.365589:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae000/0xaf35132e63035dab lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 14 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c59 expref: 31 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.365602:0:6477:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baea00/0xaf35132e63035db2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 13 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c59 expref: 30 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.365611:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298645.365614:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1780298645.365617:0:6477:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baea00, rc 0) 00000100:00100000:0.0:1780298645.365648:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+30:29360:x1866777609686784:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 266us (337us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.365654:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62069 00000100:00100000:3.0:1780298645.366530:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.366564:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609686912 00000100:00100000:1.0:1780298645.366568:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609686912 00000100:00100000:1.0:1780298645.366601:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10322 00000100:00100000:1.0:1780298645.366610:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2400:x1866777609686912:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.366623:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.366631:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63035db9 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4c60 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.366660:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481faa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605088640:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.366706:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e480 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.366711:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.366744:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609687040 00000100:00100000:0.0:1780298645.366748:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609687040 00000100:00100000:0.0:1780298645.366771:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10323 00000100:00100000:0.0:1780298645.366775:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2399:x1866777609687040:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.366782:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.366788:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63035dc0 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4c67 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.366811:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255490380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605088768:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.366840:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.367554:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481faa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605088640:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.367569:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255490380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605088768:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.367575:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63035db9 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c60 expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.367585:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63035dc0 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c67 expref: 17 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.367594:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63035db9 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c60 expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.367598:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8bd3274/0xaf35132e63035dc0 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c67 expref: 17 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.367618:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278c00, rc 0) 00010000:00010000:0.0:1780298645.367618:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae400, rc 0) 00000100:00100000:1.0:1780298645.367651:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2400:x1866777609686912:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1043us (1120us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.367652:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2399:x1866777609687040:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 875us (946us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.367664:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10322 00000100:00100000:0.0:1780298645.367670:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10323 00000100:00100000:3.0:1780298645.368878:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.368909:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.368937:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609687168 00000100:00100000:2.0:1780298645.368942:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609687168 00000100:00100000:0.0:1780298645.368950:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609687296 00000100:00100000:0.0:1780298645.368954:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609687296 00000100:00100000:2.0:1780298645.368978:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10324 00000100:00100000:0.0:1780298645.368984:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10325 00000100:00100000:2.0:1780298645.368991:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2398:x1866777609687168:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298645.368996:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b69880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609687296:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.369002:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.369004:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.369014:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e63035dc7 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4c6e expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.369014:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c37b15a2/0xaf35132e63035dce lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4c75 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.369047:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255490380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605088896:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.369055:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605089024:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.369088:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.369089:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c240 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.369916:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255490380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605088896:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.369938:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8b0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605089024:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.369938:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c37b15a2/0xaf35132e63035dce lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c75 expref: 17 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.369961:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e63035dc7 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c6e expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.369964:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c37b15a2/0xaf35132e63035dce lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c75 expref: 17 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.369979:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005e996b61/0xaf35132e63035dc7 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c6e expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.369987:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baee00, rc 0) 00010000:00010000:2.0:1780298645.369990:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf244eb2400, rc 0) 00000100:00100000:2.0:1780298645.370022:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2398:x1866777609687168:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1035us (1143us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.370022:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b69880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609687296:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1028us (1111us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.370034:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10324 00000100:00100000:0.0:1780298645.370040:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10325 00000100:00100000:2.0:1780298645.371949:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.371987:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609687424 00000100:00100000:1.0:1780298645.371993:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609687424 00002000:00100000:1.0:1780298645.372021:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (114688->180223) req@ffff9cf37f8b0000 x1866777609687424/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.372028:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f8b0000 with x1866777609687424 ext(114688->180223) 00002000:00010000:1.0:1780298645.372031:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f8b0000 00000100:00100000:1.0:1780298645.372048:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6453 00000100:00100000:1.0:1780298645.372060:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609687424:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298645.373167:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609687424:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 1106us (1219us total) trans 0 rc 65536/65536 00000100:00100000:1.0:1780298645.373175:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6453 00002000:00100000:1.0:1780298645.373181:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (114688->180223) req@ffff9cf37f8b0000 x1866777609687424/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/440 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 65536/65536 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.373188:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f8b0000 with x1866777609687424 ext(114688->180223) 00002000:00010000:1.0:1780298645.373191:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f8b0000 00000100:00100000:2.0:1780298645.374286:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.374323:0:17567:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609687552 00000100:00100000:0.0:1780298645.374329:0:17567:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609687552 00000100:00100000:0.0:1780298645.374368:0:17567:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62070 00000100:00100000:0.0:1780298645.374378:0:17567:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+37:29360:x1866777609687552:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.374394:0:17567:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.374408:0:17567:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae600/0xaf35132e63035dd5 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 14 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4c7c expref: -99 pid: 17567 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.374489:0:17567:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafe00/0xaf35132e63035ddc lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 15 type: IBT flags: 0x40000000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.374503:0:17567:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafe00/0xaf35132e63035ddc lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 15 type: IBT flags: 0x50210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.374513:0:17567:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafe00/0xaf35132e63035ddc lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 15 type: IBT flags: 0x40210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1780298645.374540:0:17567:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 2255272 00000004:00200000:0.0:1780298645.374544:0:17567:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1780298645.374563:0:17567:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafe00/0xaf35132e63035ddc lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 15 type: IBT flags: 0x40210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.374587:0:17567:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bae600/0xaf35132e63035dd5 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 15 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c7c expref: 39 pid: 17567 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.374602:0:17567:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafe00/0xaf35132e63035ddc lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 14 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c7c expref: 38 pid: 17567 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.374610:0:17567:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298645.374613:0:17567:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1780298645.374615:0:17567:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bafe00, rc 0) 00000100:00100000:0.0:1780298645.374646:0:17567:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+38:29360:x1866777609687552:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 270us (361us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.374652:0:17567:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62070 00000100:00100000:3.0:1780298645.381028:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.381073:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609687680 00000100:00100000:1.0:1780298645.381083:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609687680 00002000:00100000:1.0:1780298645.381118:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x2c0000401:0x11:0x0] (0->2256895) req@ffff9cf242b69f80 x1866777609687680/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 504/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.381149:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b69f80 with x1866777609687680 ext(0->2256895) 00002000:00010000:1.0:1780298645.381154:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf242b69f80 00000100:00100000:1.0:1780298645.381181:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6454 00000100:00100000:1.0:1780298645.381196:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b69f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609687680:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298645.388067:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b69f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609687680:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 6868us (7039us total) trans 0 rc 2191360/2191360 00000100:00100000:1.0:1780298645.388078:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6454 00002000:00100000:1.0:1780298645.388091:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x2c0000401:0x11:0x0] (0->2256895) req@ffff9cf242b69f80 x1866777609687680/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 504/440 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 2191360/2191360 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.388102:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242b69f80 with x1866777609687680 ext(0->2256895) 00002000:00010000:1.0:1780298645.388108:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf242b69f80 00000100:00100000:3.0:1780298645.389646:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.389668:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.389702:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609687808 00000100:00100000:1.0:1780298645.389711:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609687808 00000100:00100000:0.0:1780298645.389726:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609687936 00000100:00100000:0.0:1780298645.389733:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609687936 00000100:00100000:1.0:1780298645.389759:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10326 00000100:00100000:1.0:1780298645.389768:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b68700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609687808:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298645.389778:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10327 00010000:00010000:1.0:1780298645.389783:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298645.389790:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b6b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609687936:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.389799:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63035de3 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4c83 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.389801:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.389815:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000001a3f03a/0xaf35132e63035dea lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4c8a expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.389854:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481faa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605089152:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.389857:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255490380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605089280:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.389946:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.389947:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e480 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.390758:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255490380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605089280:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.390770:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481faa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605089152:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.390782:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000001a3f03a/0xaf35132e63035dea lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c8a expref: 17 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.390791:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63035de3 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c83 expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.390800:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000001a3f03a/0xaf35132e63035dea lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c8a expref: 17 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.390809:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63035de3 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c83 expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.390811:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242bae600, rc 0) 00010000:00010000:1.0:1780298645.390823:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278c00, rc 0) 00000100:00100000:0.0:1780298645.390839:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b6b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609687936:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1054us (1168us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.390845:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10327 00000100:00100000:1.0:1780298645.390850:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b68700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609687808:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1082us (1202us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.390859:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10326 00000100:00100000:3.0:1780298645.391856:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.391889:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609688064 00000100:00100000:0.0:1780298645.391893:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609688064 00000100:00100000:3.0:1780298645.391908:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.391923:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609688192 00000100:00100000:0.0:1780298645.391926:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609688192 00000100:00100000:0.0:1780298645.391949:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10328 00000100:00100000:1.0:1780298645.391960:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10329 00000100:00100000:0.0:1780298645.391961:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b6a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2398:x1866777609688064:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.391970:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298645.391974:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b68380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609688192:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.391981:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63035df1 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4c91 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.391982:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.391990:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63035df8 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4c98 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.392016:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255490380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605089408:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.392020:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481faa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605089536:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.392056:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.392060:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e480 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.392852:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255490380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605089408:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.392866:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481faa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605089536:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.392873:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63035df1 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c91 expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.392881:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63035df8 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c98 expref: 17 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.392893:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c37b15a2/0xaf35132e63035df1 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c91 expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.392894:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63035df8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c98 expref: 17 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.392918:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278a00, rc 0) 00010000:00010000:0.0:1780298645.392918:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf242baee00, rc 0) 00000100:00100000:1.0:1780298645.392957:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b68380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609688192:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 986us (1047us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.392958:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b6a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2398:x1866777609688064:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 999us (1101us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.392973:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10329 00000100:00100000:0.0:1780298645.392982:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10328 00000100:00100000:2.0:1780298645.393973:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.394014:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609688320 00000100:00100000:3.0:1780298645.394020:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609688320 00000100:00100000:3.0:1780298645.394055:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10330 00000100:00100000:3.0:1780298645.394065:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:29360:x1866777609688320:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.394079:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.394089:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e63035dff lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4c9f expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.394108:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63035d9d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c4b expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.394118:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(49 169) 00010000:00010000:3.0:1780298645.394122:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(50 169) 00010000:00010000:3.0:1780298645.394153:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e63035dff lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1507328->1515519] (req 1507328->1515519) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c9f expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.394230:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63035d9d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c4b expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.394243:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63035d9d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c4b expref: 8 pid: 8356 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.394252:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605089664:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298645.394297:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c240 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.394872:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8b0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605089664:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298645.394892:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e63035dff lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1507328->1515519] (req 1507328->1515519) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c9f expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.394903:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243156400, rc 0) 00000100:00100000:2.0:1780298645.394931:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609688320:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 869us (958us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.394938:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10330 00000100:00100000:3.0:1780298645.395214:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.395257:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609688448 00000100:00100000:1.0:1780298645.395262:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609688448 00002000:00100000:1.0:1780298645.395309:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (143360->151551) req@ffff9cf36e5d9f80 x1866777609688448/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 8680/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.395326:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e5d9f80 with x1866777609688448 ext(143360->151551) 00010000:00010000:1.0:1780298645.395333:0:8363:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63035d9d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c4b expref: 9 pid: 8356 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.395342:0:8363:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63035d9d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c4b expref: 9 pid: 8356 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1780298645.395347:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf36e5d9f80 00000100:00100000:1.0:1780298645.395359:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 217 00000100:00100000:1.0:1780298645.395365:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609688448:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:1.0:1780298645.395834:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609688448:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 469us (621us total) trans 4294969776 rc 0/0 00000100:00100000:1.0:1780298645.395839:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 217 00002000:00100000:1.0:1780298645.395844:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (143360->151551) req@ffff9cf36e5d9f80 x1866777609688448/t4294969776(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 8680/448 e 0 to 0 dl 1780298656 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.395849:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e5d9f80 with x1866777609688448 ext(143360->151551) 00010000:00010000:1.0:1780298645.395853:0:8363:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63035d9d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c4b expref: 11 pid: 8356 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.395862:0:8363:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63035d9d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c4b expref: 11 pid: 8356 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1780298645.395866:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf36e5d9f80 00000100:00100000:0.0:1780298645.398318:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.398376:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609688576 00000100:00100000:2.0:1780298645.398383:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609688576 00010000:00010000:2.0:1780298645.398414:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63035d9d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c4b expref: 11 pid: 8356 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.398448:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15279 00000100:00100000:2.0:1780298645.398458:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3487c5450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609688576:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298645.398470:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298645.398489:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63035d9d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c4b expref: 11 pid: 8356 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.398502:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63035d9d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c4b expref: 11 pid: 8356 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.398525:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001eb6350c/0xaf35132e63035d9d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c4b expref: 11 pid: 8356 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.398540:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(50 169) 00010000:00010000:2.0:1780298645.398543:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(50 169) 00010000:00010000:2.0:1780298645.398549:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e63035dff lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1507328->1515519) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c9f expref: 8 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.398556:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e63035dff lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1507328->1515519) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c9f expref: 8 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.398585:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e63035dff lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1507328->1515519) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c9f expref: 8 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.398594:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b0a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777605089792:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298645.398658:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c240 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.399566:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8b0a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777605089792:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298645.399577:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298645.399601:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3487c5450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609688576:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1146us (1290us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.399607:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15279 00000100:00100000:3.0:1780298645.402981:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.403029:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609688704 00000100:00100000:1.0:1780298645.403037:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609688704 00002000:00100000:1.0:1780298645.403087:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (0->2256895) req@ffff9cf36e5db480 x1866777609688704/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.403098:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e5db480 with x1866777609688704 ext(0->2256895) 00002000:00010000:1.0:1780298645.403102:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf36e5db480 00000100:00100000:1.0:1780298645.403150:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6455 00000100:00100000:1.0:1780298645.403161:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609688704:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298645.408395:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609688704:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 5226us (5406us total) trans 0 rc 2256896/2256896 00000100:00100000:1.0:1780298645.408406:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6455 00002000:00100000:1.0:1780298645.408419:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (0->2256895) req@ffff9cf36e5db480 x1866777609688704/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/440 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 2256896/2256896 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.408429:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e5db480 with x1866777609688704 ext(0->2256895) 00002000:00010000:1.0:1780298645.408435:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf36e5db480 00000100:00100000:3.0:1780298645.410437:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.410509:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609688832 00000100:00100000:1.0:1780298645.410516:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609688832 00000100:00100000:1.0:1780298645.410552:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10331 00000100:00100000:1.0:1780298645.410558:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609688832:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.410572:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.410588:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63035e06 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4ca6 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.410632:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481faa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605089920:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.410690:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e480 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.411497:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481faa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605089920:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.411524:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63035e06 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4ca6 expref: 17 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.411545:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63035e06 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4ca6 expref: 17 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.411557:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278a00, rc 0) 00000100:00100000:1.0:1780298645.411594:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609688832:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1032us (1155us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.411602:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10331 00000100:00100000:0.0:1780298645.412550:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.412592:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609688960 00000100:00100000:2.0:1780298645.412597:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609688960 00000100:00100000:2.0:1780298645.412625:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10332 00000100:00100000:2.0:1780298645.412631:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255490380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609688960:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.412639:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.412646:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ac4408d/0xaf35132e63035e0d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4cad expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.412669:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605090048:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.412714:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.413343:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8b0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605090048:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.413360:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000ac4408d/0xaf35132e63035e0d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cad expref: 17 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.413374:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ac4408d/0xaf35132e63035e0d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cad expref: 17 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.413384:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341499200, rc 0) 00000100:00100000:2.0:1780298645.413414:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255490380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609688960:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 782us (864us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.413422:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10332 00000100:00100000:0.0:1780298645.415829:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.415872:0:9976:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609689216 00000100:00100000:2.0:1780298645.415879:0:9976:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609689216 00000100:00100000:2.0:1780298645.415940:0:9976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62071 00000100:00100000:2.0:1780298645.415952:0:9976:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255490e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+38:29360:x1866777609689216:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:2.0:1780298645.416002:0:9976:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298645.416023:0:9976:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242bafe00/0xaf35132e63035ddc lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 15 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c7c expref: 38 pid: 17567 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.416039:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298645.416043:0:9976:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298645.416239:0:9976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499e00/0xaf35132e63035e14 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 14 type: IBT flags: 0x40000000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.416255:0:9976:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baea00/0xaf35132e63035db2 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 14 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c59 expref: 29 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.416266:0:9976:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499e00/0xaf35132e63035e14 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 14 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.416309:0:9976:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baea00/0xaf35132e63035db2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 14 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c59 expref: 29 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.416323:0:9976:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baea00/0xaf35132e63035db2 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 14 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c59 expref: 29 pid: 6477 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.416338:0:9976:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:9976:1866777605090176:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298645.416393:0:9976:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.417033:0:9976:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8b0a80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:9976:1866777605090176:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298645.417049:0:9976:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499e00/0xaf35132e63035e14 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 14 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.417369:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.417407:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609689344 00000100:00100000:2.0:1780298645.417413:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609689344 00010000:00010000:2.0:1780298645.417433:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baea00/0xaf35132e63035db2 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 14 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c59 expref: 30 pid: 6477 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.417463:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15280 00000100:00100000:2.0:1780298645.417472:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556ab800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+30:2399:x1866777609689344:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298645.417481:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298645.417487:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baea00/0xaf35132e63035db2 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 15 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c59 expref: 30 pid: 6477 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.417504:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baea00/0xaf35132e63035db2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 15 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c59 expref: 30 pid: 6477 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.417518:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf242baea00/0xaf35132e63035db2 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 15 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c59 expref: 30 pid: 6477 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.417526:0:7400:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298645.417529:0:7400:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499e00/0xaf35132e63035e14 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 14 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.417535:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499e00/0xaf35132e63035e14 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 14 type: IBT flags: 0x40290000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.417540:0:7400:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499e00/0xaf35132e63035e14 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 14 type: IBT flags: 0x40290000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.417558:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298645.417581:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556ab800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+29:2399:x1866777609689344:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 110us (211us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.417586:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15280 00010000:00010000:0.0:1780298645.417587:0:9976:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499e00/0xaf35132e63035e14 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.417595:0:9976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499e00/0xaf35132e63035e14 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1780298645.417637:0:9976:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991114 00000004:00010000:0.0:1780298645.417660:0:9976:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 0000000007b54fcf reply state 00000000fb098cc5 transno 4294991115 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499e00/0xaf35132e63035e14 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 13 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.417670:0:9976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341499e00/0xaf35132e63035e14 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 13 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.417675:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298645.417677:0:9976:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1780298645.417678:0:9976:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1780298645.417738:0:9976:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255490e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+40:29360:x1866777609689216:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1789us (1910us total) trans 4294991115 rc 0/0 00000100:00100000:0.0:1780298645.417746:0:9976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62071 00000020:00080000:0.0:1780298645.417750:0:9976:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1780298645.418558:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.418587:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609689472 00000100:00100000:3.0:1780298645.418592:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609689472 00000100:00100000:3.0:1780298645.418618:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10333 00000100:00100000:3.0:1780298645.418624:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:29360:x1866777609689472:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.418633:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.418644:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d7062f72/0xaf35132e63035e1b lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4cb4 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.418660:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e63035dff lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1507328->1515519) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c9f expref: 8 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.418667:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(50 169) 00010000:00010000:3.0:1780298645.418670:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(50 169) 00010000:00010000:3.0:1780298645.418672:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d7062f72/0xaf35132e63035e1b lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cb4 expref: 11 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.418686:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e63035dff lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1507328->1515519) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c9f expref: 8 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.418692:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e63035dff lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1507328->1515519) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c9f expref: 8 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.418699:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605090304:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298645.418731:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2418563c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.419381:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d8e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605090304:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298645.419399:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d7062f72/0xaf35132e63035e1b lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cb4 expref: 11 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.419412:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341692e00, rc 0) 00000100:00100000:3.0:1780298645.419442:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:29360:x1866777609689472:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 818us (883us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298645.419449:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10333 00000100:00100000:2.0:1780298645.422022:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.422071:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609689600 00000100:00100000:0.0:1780298645.422076:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609689600 00010000:00010000:0.0:1780298645.422096:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e63035dff lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1507328->1515519) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c9f expref: 9 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.422120:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15281 00000100:00100000:0.0:1780298645.422152:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609689600:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298645.422164:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298645.422182:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e63035dff lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1507328->1515519) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c9f expref: 9 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.422193:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e63035dff lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1507328->1515519) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c9f expref: 9 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.422209:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000004b2c7ae/0xaf35132e63035dff lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1507328->1515519) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c9f expref: 9 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.422217:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(50 169) 00010000:00010000:0.0:1780298645.422220:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(50 169) 00010000:00010000:0.0:1780298645.422224:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000d7062f72/0xaf35132e63035e1b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cb4 expref: 10 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.422230:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d7062f72/0xaf35132e63035e1b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cb4 expref: 10 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.422257:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000d7062f72/0xaf35132e63035e1b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cb4 expref: 10 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.422280:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433dea00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777605090432:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298645.422329:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1900 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.422947:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433dea00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777605090432:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298645.422960:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298645.422990:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2400:x1866777609689600:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 841us (970us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.422999:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15281 00000100:00100000:2.0:1780298645.423186:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.423211:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.423226:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609689856 00000100:00100000:0.0:1780298645.423231:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609689856 00000100:00100000:3.0:1780298645.423254:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609689728 00002000:00010000:0.0:1780298645.423254:0:19943:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (3076618->18446744073709551615) 00000100:00100000:3.0:1780298645.423260:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609689728 00002000:00100000:0.0:1780298645.423260:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f8b3480 with x1866777609689856 ext(3076618->18446744073709551615) 00002000:00010000:0.0:1780298645.423281:0:19943:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f8b3480. 00002000:00010000:3.0:1780298645.423293:0:19949:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:3.0:1780298645.423298:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f8b1f80 with x1866777609689728 ext(4194304->18446744073709551615) 00000100:00100000:0.0:1780298645.423303:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6456 00002000:00010000:3.0:1780298645.423307:0:19949:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f8b1f80. 00000100:00100000:0.0:1780298645.423312:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2400:x1866777609689856:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:3.0:1780298645.423339:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6457 00000100:00100000:3.0:1780298645.423350:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609689728:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:0.0:1780298645.423478:0:19943:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 0 -> 3076618 00002000:00010000:3.0:1780298645.423479:0:19949:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 2255272 -> 4194304 00002000:00010000:0.0:1780298645.423481:0:19943:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:3.0:1780298645.423486:0:19949:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298645.423486:0:19943:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298645.423491:0:19943:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:3.0:1780298645.423492:0:19949:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298645.423495:0:19943:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 0 -> 0 00002000:00010000:3.0:1780298645.423497:0:19949:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:3.0:1780298645.423500:0:19949:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 376 -> 376 00000100:00100000:0.0:1780298645.423524:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609689856:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 214us (337us total) trans 4294969316 rc 0/0 00000100:00100000:3.0:1780298645.423529:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609689728:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 181us (315us total) trans 4294969777 rc 0/0 00000100:00100000:0.0:1780298645.423537:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6456 00002000:00010000:0.0:1780298645.423541:0:19943:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (3076618->18446744073709551615) 00002000:00100000:0.0:1780298645.423544:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f8b3480 with x1866777609689856 ext(3076618->18446744073709551615) 00000100:00100000:3.0:1780298645.423549:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6457 00002000:00010000:0.0:1780298645.423551:0:19943:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f8b3480. 00002000:00010000:3.0:1780298645.423553:0:19949:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:3.0:1780298645.423558:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f8b1f80 with x1866777609689728 ext(4194304->18446744073709551615) 00002000:00010000:3.0:1780298645.423564:0:19949:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f8b1f80. 00000100:00100000:3.0:1780298645.432334:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.432380:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609689984 00000100:00100000:1.0:1780298645.432389:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609689984 00002000:00100000:1.0:1780298645.432425:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (0->3071999) req@ffff9cf36e5d8e00 x1866777609689984/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 504/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.432434:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e5d8e00 with x1866777609689984 ext(0->3071999) 00002000:00010000:1.0:1780298645.432438:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf36e5d8e00 00000100:00100000:1.0:1780298645.432460:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6458 00000100:00100000:1.0:1780298645.432468:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609689984:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298645.432508:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.432567:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609690112 00000100:00100000:2.0:1780298645.432572:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609690112 00002000:00100000:2.0:1780298645.432598:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (2256896->4194303) req@ffff9cf36e5d9180 x1866777609690112/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.432608:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e5d9180 with x1866777609690112 ext(2256896->4194303) 00002000:00010000:2.0:1780298645.432613:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf36e5d9180 00000100:00100000:2.0:1780298645.432633:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6459 00000100:00100000:2.0:1780298645.432639:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609690112:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298645.442490:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609689984:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 10012us (10151us total) trans 0 rc 2260992/2260992 00000100:00100000:1.0:1780298645.442507:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6458 00002000:00100000:1.0:1780298645.442522:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (0->3071999) req@ffff9cf36e5d8e00 x1866777609689984/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 504/440 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 2260992/2260992 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.442540:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e5d8e00 with x1866777609689984 ext(0->3071999) 00002000:00010000:1.0:1780298645.442547:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf36e5d8e00 00000100:00100000:1.0:1780298645.443453:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609690112:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 10813us (10943us total) trans 0 rc 1937408/1937408 00000100:00100000:1.0:1780298645.443463:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6459 00002000:00100000:1.0:1780298645.443471:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (2256896->4194303) req@ffff9cf36e5d9180 x1866777609690112/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/440 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 1937408/1937408 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.443480:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e5d9180 with x1866777609690112 ext(2256896->4194303) 00002000:00010000:1.0:1780298645.443485:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf36e5d9180 00000100:00100000:3.0:1780298645.447607:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.447680:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609690240 00000100:00100000:1.0:1780298645.447689:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609690240 00002000:00100000:1.0:1780298645.447743:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x12:0x0] (2256896->3080191) req@ffff9cf36e5dbb80 x1866777609690240/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 504/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.447754:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e5dbb80 with x1866777609690240 ext(2256896->3080191) 00002000:00010000:1.0:1780298645.447759:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf36e5dbb80 00000100:00100000:1.0:1780298645.447794:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6460 00000100:00100000:1.0:1780298645.447804:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5dbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2397:x1866777609690240:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298645.451484:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5dbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2397:x1866777609690240:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 3673us (3876us total) trans 0 rc 819200/819200 00000100:00100000:1.0:1780298645.451498:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6460 00002000:00100000:1.0:1780298645.451510:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x12:0x0] (2256896->3080191) req@ffff9cf36e5dbb80 x1866777609690240/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 504/440 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 819200/819200 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.451529:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e5dbb80 with x1866777609690240 ext(2256896->3080191) 00002000:00010000:1.0:1780298645.451536:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf36e5dbb80 00000100:00100000:0.0:1780298645.453999:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.454065:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609690368 00000100:00100000:1.0:1780298645.454076:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609690368 00002000:00100000:1.0:1780298645.454120:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x12:0x0] (3067904->3076617) req@ffff9cf2433ded80 x1866777609690368/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 9208/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.454165:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2433ded80 with x1866777609690368 ext(3067904->3076617) 00002000:00010000:1.0:1780298645.454170:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2433ded80 00000100:00100000:1.0:1780298645.454197:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6461 00000100:00100000:1.0:1780298645.454204:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609690368:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:1.0:1780298645.454883:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2400:x1866777609690368:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 677us (888us total) trans 4294969317 rc 0/0 00000100:00100000:1.0:1780298645.454891:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6461 00002000:00100000:1.0:1780298645.454899:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x12:0x0] (3067904->3076617) req@ffff9cf2433ded80 x1866777609690368/t4294969317(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 9208/448 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.454908:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2433ded80 with x1866777609690368 ext(3067904->3076617) 00002000:00010000:1.0:1780298645.454911:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2433ded80 00000100:00100000:2.0:1780298645.456351:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.456414:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609690496 00000100:00100000:1.0:1780298645.456423:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609690496 00000100:00100000:1.0:1780298645.456467:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10334 00000100:00100000:1.0:1780298645.456478:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2400:x1866777609690496:12345-192.168.203.25@tcp:16:fsx.0 00000001:00080000:2.0:1780298645.457440:0:8351:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0000: transno 4294969317 is committed 00000001:00080000:2.0:1780298645.457468:0:8351:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0000: transno 4294969314 is committed 00000100:00100000:1.0:1780298645.457612:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609690496:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 1135us (1262us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298645.457624:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10334 00000100:00100000:3.0:1780298645.458968:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.459025:0:17578:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609690624 00000100:00100000:1.0:1780298645.459034:0:17578:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609690624 00000100:00100000:1.0:1780298645.459078:0:17578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62072 00000100:00100000:1.0:1780298645.459087:0:17578:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+40:29360:x1866777609690624:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.459109:0:17578:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.459154:0:17578:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278a00/0xaf35132e63035e22 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 14 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4cbb expref: -99 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.459255:0:17578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278c00/0xaf35132e63035e29 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 15 type: IBT flags: 0x40000000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.459279:0:17578:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278c00/0xaf35132e63035e29 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 15 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.459291:0:17578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278c00/0xaf35132e63035e29 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 15 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1780298645.459315:0:17578:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 7270922 00000004:00200000:1.0:1780298645.459318:0:17578:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1780298645.459334:0:17578:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278c00/0xaf35132e63035e29 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 15 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.459353:0:17578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278a00/0xaf35132e63035e22 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 15 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cbb expref: 42 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.459367:0:17578:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278c00/0xaf35132e63035e29 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 14 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cbb expref: 41 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.459381:0:17578:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298645.459384:0:17578:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1780298645.459387:0:17578:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278c00, rc 0) 00000100:00100000:1.0:1780298645.459425:0:17578:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+41:29360:x1866777609690624:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 339us (459us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298645.459432:0:17578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62072 00000100:00100000:0.0:1780298645.460842:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.460882:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609690880 00000100:00100000:2.0:1780298645.460887:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609690880 00000100:00100000:2.0:1780298645.460919:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62073 00000100:00100000:2.0:1780298645.460925:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+29:29360:x1866777609690880:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:2.0:1780298645.461102:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ba00/0xaf35132e63035e30 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 15 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.461113:0:6477:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278c00/0xaf35132e63035e29 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 15 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cbb expref: 40 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.461121:0:6477:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ba00/0xaf35132e63035e30 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 15 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.461177:0:6477:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278c00/0xaf35132e63035e29 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 15 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cbb expref: 40 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.461196:0:6477:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278c00/0xaf35132e63035e29 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 15 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cbb expref: 40 pid: 17578 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.461210:0:6477:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b3100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6477:1866777605090560:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298645.461296:0:6477:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c900 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.462267:0:6477:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8b3100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6477:1866777605090560:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298645.462292:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:0.0:1780298645.462298:0:6477:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ba00/0xaf35132e63035e30 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 15 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.462331:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609691008 00000100:00100000:1.0:1780298645.462337:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609691008 00010000:00010000:1.0:1780298645.462364:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278c00/0xaf35132e63035e29 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 15 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cbb expref: 41 pid: 17578 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.462395:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15282 00000100:00100000:1.0:1780298645.462404:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5dad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+41:2399:x1866777609691008:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298645.462415:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298645.462422:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278c00/0xaf35132e63035e29 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 16 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cbb expref: 41 pid: 17578 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.462434:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278c00/0xaf35132e63035e29 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 16 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cbb expref: 41 pid: 17578 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.462451:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278c00/0xaf35132e63035e29 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 16 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cbb expref: 41 pid: 17578 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.462461:0:6463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298645.462464:0:6463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ba00/0xaf35132e63035e30 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 15 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.462472:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ba00/0xaf35132e63035e30 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 15 type: IBT flags: 0x40290000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.462476:0:6463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ba00/0xaf35132e63035e30 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 15 type: IBT flags: 0x40290000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.462495:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1780298645.462515:0:6477:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ba00/0xaf35132e63035e30 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 14 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.462525:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5dad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+40:2399:x1866777609691008:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 120us (231us total) trans 0 rc 0/0 00010000:00010000:0.0:1780298645.462525:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ba00/0xaf35132e63035e30 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 14 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.462540:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15282 00000001:00080000:0.0:1780298645.462586:0:6477:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991116 00000004:00010000:0.0:1780298645.462614:0:6477:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000485d4070 reply state 00000000cf5b3f03 transno 4294991117 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ba00/0xaf35132e63035e30 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 14 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.462623:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ba00/0xaf35132e63035e30 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 14 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.462628:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298645.462630:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1780298645.462632:0:6477:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1780298645.462689:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433dea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+32:29360:x1866777609690880:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1764us (1848us total) trans 4294991117 rc 0/0 00000100:00100000:0.0:1780298645.462701:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62073 00000020:00080000:0.0:1780298645.462707:0:6477:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1780298645.463520:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.463574:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609691136 00000100:00100000:1.0:1780298645.463580:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609691136 00000100:00100000:1.0:1780298645.463624:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10335 00000100:00100000:1.0:1780298645.463631:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5db100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:29360:x1866777609691136:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.463644:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.463657:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63035e37 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4cc2 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.463678:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d7062f72/0xaf35132e63035e1b lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cb4 expref: 11 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.463688:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(50 169) 00010000:00010000:1.0:1780298645.463692:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(51 169) 00010000:00010000:1.0:1780298645.463707:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63035e37 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cc2 expref: 9 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.463735:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d7062f72/0xaf35132e63035e1b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cb4 expref: 11 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.463747:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d7062f72/0xaf35132e63035e1b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cb4 expref: 11 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.463759:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481faa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605090688:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298645.463817:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.464546:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481faa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605090688:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298645.464559:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63035e37 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cc2 expref: 9 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.464570:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278200, rc 0) 00000100:00100000:1.0:1780298645.464597:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5db100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609691136:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 968us (1084us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298645.464608:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10335 00000100:00100000:3.0:1780298645.466771:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.466831:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609691264 00000100:00100000:0.0:1780298645.466841:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609691264 00010000:00010000:0.0:1780298645.466873:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d7062f72/0xaf35132e63035e1b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cb4 expref: 12 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.466920:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15283 00000100:00100000:0.0:1780298645.466931:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5da680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609691264:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298645.466943:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298645.466965:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d7062f72/0xaf35132e63035e1b lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cb4 expref: 12 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.466979:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d7062f72/0xaf35132e63035e1b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cb4 expref: 12 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.467008:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d7062f72/0xaf35132e63035e1b lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cb4 expref: 12 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.467028:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(51 169) 00010000:00010000:0.0:1780298645.467033:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(51 169) 00010000:00010000:0.0:1780298645.467043:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63035e37 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cc2 expref: 8 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.467053:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63035e37 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cc2 expref: 8 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.467084:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63035e37 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cc2 expref: 8 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.467103:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2548b9180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777605090816:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298645.467192:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1d80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.468108:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2548b9180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777605090816:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298645.468160:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:0.0:1780298645.468174:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298645.468229:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609691392 00000100:00100000:2.0:1780298645.468236:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609691392 00000100:00100000:0.0:1780298645.468239:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5da680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609691264:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1308us (1471us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.468252:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15283 00000100:00100000:2.0:1780298645.468305:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10336 00000100:00100000:2.0:1780298645.468318:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609691392:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.468333:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.468347:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0a0f245/0xaf35132e63035e3e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4cc9 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.468368:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63035da4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c52 expref: 6 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.468376:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(48 122) 00010000:00010000:2.0:1780298645.468379:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(49 122) 00010000:00010000:2.0:1780298645.468384:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0a0f245/0xaf35132e63035e3e lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cc9 expref: 10 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.468406:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63035da4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c52 expref: 6 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.468423:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63035da4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c52 expref: 6 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.468434:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b3100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605090944:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298645.468484:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c840 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.469195:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8b3100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605090944:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298645.469214:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e0a0f245/0xaf35132e63035e3e lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cc9 expref: 10 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.469226:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b200, rc 0) 00000100:00100000:2.0:1780298645.469261:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609691392:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 949us (1103us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.469279:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10336 00000100:00100000:0.0:1780298645.472515:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.472572:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609691520 00000100:00100000:1.0:1780298645.472589:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609691520 00010000:00010000:1.0:1780298645.472632:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63035da4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c52 expref: 7 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.472679:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15284 00000100:00100000:1.0:1780298645.472690:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609691520:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298645.472715:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:1.0:1780298645.472734:0:8374:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 0 -> 24 00010000:00010000:1.0:1780298645.472742:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63035da4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c52 expref: 7 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.472758:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63035da4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c52 expref: 7 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.472780:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63035da4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4c52 expref: 7 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.472795:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(49 122) 00010000:00010000:1.0:1780298645.472800:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(49 122) 00010000:00010000:1.0:1780298645.472807:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000e0a0f245/0xaf35132e63035e3e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cc9 expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.472818:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0a0f245/0xaf35132e63035e3e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cc9 expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.472852:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000e0a0f245/0xaf35132e63035e3e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cc9 expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.472866:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481faa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:8374:1866777605091072:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298645.472940:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.473811:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481faa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:8374:1866777605091072:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298645.473827:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298645.473858:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609691520:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1172us (1353us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298645.473868:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15284 00000100:00100000:3.0:1780298645.473977:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.474028:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609691776 00000100:00100000:1.0:1780298645.474035:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609691776 00000100:00100000:3.0:1780298645.474051:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00002000:00010000:1.0:1780298645.474068:0:19949:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (2970630->18446744073709551615) 00002000:00100000:1.0:1780298645.474077:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e5d8a80 with x1866777609691776 ext(2970630->18446744073709551615) 00002000:00010000:1.0:1780298645.474087:0:19949:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf36e5d8a80. 00000100:00100000:1.0:1780298645.474109:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609691648 00000100:00100000:1.0:1780298645.474113:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609691648 00002000:00010000:1.0:1780298645.474158:0:19949:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00000100:00100000:3.0:1780298645.474163:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6462 00002000:00100000:1.0:1780298645.474163:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e5da300 with x1866777609691648 ext(4194304->18446744073709551615) 00002000:00010000:1.0:1780298645.474172:0:19949:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf36e5da300. 00000100:00100000:3.0:1780298645.474179:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609691776:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:2.0:1780298645.474193:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6463 00000100:00100000:2.0:1780298645.474212:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5da300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609691648:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:2.0:1780298645.474365:0:8362:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:2.0:1780298645.474368:0:8362:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298645.474370:0:8362:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298645.474371:0:8362:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298645.474373:0:8362:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 376 -> 376 00000100:00100000:2.0:1780298645.474422:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5da300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609691648:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 219us (371us total) trans 4294969778 rc 0/0 00000100:00100000:2.0:1780298645.474430:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6463 00002000:00010000:2.0:1780298645.474434:0:8362:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:2.0:1780298645.474437:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e5da300 with x1866777609691648 ext(4194304->18446744073709551615) 00002000:00010000:2.0:1780298645.474448:0:8362:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf36e5da300. 00002000:00010000:3.0:1780298645.474560:0:8364:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 3076618 -> 2970630 00002000:00010000:3.0:1780298645.474564:0:8364:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298645 -> 1780298645 00002000:00010000:3.0:1780298645.474567:0:8364:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:3.0:1780298645.474569:0:8364:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298645 -> 1780298645 00002000:00010000:3.0:1780298645.474570:0:8364:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 24 -> 0 00000100:00100000:3.0:1780298645.474607:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609691776:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 432us (633us total) trans 4294969318 rc 0/0 00000100:00100000:3.0:1780298645.474623:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6462 00002000:00010000:3.0:1780298645.474627:0:8364:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (2970630->18446744073709551615) 00002000:00100000:3.0:1780298645.474631:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e5d8a80 with x1866777609691776 ext(2970630->18446744073709551615) 00002000:00010000:3.0:1780298645.474636:0:8364:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf36e5d8a80. 00000100:00100000:3.0:1780298645.475433:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.475487:0:17567:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609691904 00000100:00100000:0.0:1780298645.475493:0:17567:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609691904 00000100:00100000:0.0:1780298645.475529:0:17567:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62074 00000100:00100000:0.0:1780298645.475537:0:17567:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+32:29360:x1866777609691904:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.475554:0:17567:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.475572:0:17567:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2e00/0xaf35132e63035e45 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 15 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4cd0 expref: -99 pid: 17567 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.475670:0:17567:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2000/0xaf35132e63035e4c lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 16 type: IBT flags: 0x40000000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.475685:0:17567:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2000/0xaf35132e63035e4c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 16 type: IBT flags: 0x50210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.475695:0:17567:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2000/0xaf35132e63035e4c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 16 type: IBT flags: 0x40210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1780298645.475720:0:17567:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 7164934 00000004:00200000:0.0:1780298645.475725:0:17567:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1780298645.475740:0:17567:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2000/0xaf35132e63035e4c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 16 type: IBT flags: 0x40210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.475763:0:17567:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2e00/0xaf35132e63035e45 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 16 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cd0 expref: 34 pid: 17567 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.475778:0:17567:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2000/0xaf35132e63035e4c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 15 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cd0 expref: 33 pid: 17567 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.475786:0:17567:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298645.475789:0:17567:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1780298645.475793:0:17567:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2000, rc 0) 00000100:00100000:0.0:1780298645.475826:0:17567:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+33:29360:x1866777609691904:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 290us (394us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.475832:0:17567:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62074 00000100:00100000:0.0:1780298645.477448:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.477493:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609692032 00000100:00100000:3.0:1780298645.477498:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609692032 00000100:00100000:3.0:1780298645.477532:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10337 00000100:00100000:3.0:1780298645.477537:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:29360:x1866777609692032:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.477547:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.477557:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027d3a930/0xaf35132e63035e53 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4cd7 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.477576:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e0a0f245/0xaf35132e63035e3e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cc9 expref: 10 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.477585:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(49 122) 00010000:00010000:3.0:1780298645.477590:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(50 122) 00010000:00010000:3.0:1780298645.477596:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027d3a930/0xaf35132e63035e53 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2240512->2248703] (req 2240512->2248703) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cd7 expref: 7 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.477628:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e0a0f245/0xaf35132e63035e3e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cc9 expref: 10 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.477642:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e0a0f245/0xaf35132e63035e3e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cc9 expref: 10 pid: 8356 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.477653:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605091200:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298645.477712:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2418563c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.478515:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24310ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605091200:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298645.478554:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000027d3a930/0xaf35132e63035e53 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2240512->2248703] (req 2240512->2248703) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cd7 expref: 7 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.478565:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341692200, rc 0) 00000100:00100000:0.0:1780298645.478595:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609692032:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1057us (1152us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.478603:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10337 00000100:00100000:2.0:1780298645.478775:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.478830:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609692160 00000100:00100000:3.0:1780298645.478837:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609692160 00010000:00010000:3.0:1780298645.478862:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e0a0f245/0xaf35132e63035e3e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cc9 expref: 11 pid: 8356 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.478895:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15285 00000100:00100000:3.0:1780298645.478905:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b3100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609692160:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298645.478916:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298645.478937:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e0a0f245/0xaf35132e63035e3e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cc9 expref: 11 pid: 8356 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.478951:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e0a0f245/0xaf35132e63035e3e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cc9 expref: 11 pid: 8356 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.478970:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0a0f245/0xaf35132e63035e3e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cc9 expref: 11 pid: 8356 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.478982:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(50 122) 00010000:00010000:3.0:1780298645.478985:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(50 122) 00010000:00010000:3.0:1780298645.478993:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000027d3a930/0xaf35132e63035e53 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2240512->2248703) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cd7 expref: 6 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.479001:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027d3a930/0xaf35132e63035e53 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2240512->2248703) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cd7 expref: 6 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.479027:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000027d3a930/0xaf35132e63035e53 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2240512->2248703) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cd7 expref: 6 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.479045:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310df80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777605091328:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298645.479093:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.480344:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24310df80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777605091328:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298645.480365:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298645.480399:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b3100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609692160:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1495us (1626us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298645.480406:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15285 00000100:00100000:0.0:1780298645.480698:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.480737:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609692288 00000100:00100000:1.0:1780298645.480746:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609692288 00002000:00100000:1.0:1780298645.480787:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x12:0x0] (2240512->2248703) req@ffff9cf2548bbb80 x1866777609692288/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.480795:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2548bbb80 with x1866777609692288 ext(2240512->2248703) 00002000:00010000:1.0:1780298645.480799:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2548bbb80 00000100:00100000:1.0:1780298645.480821:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6464 00000100:00100000:1.0:1780298645.480828:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609692288:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298645.481014:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609692288:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 185us (317us total) trans 0 rc 8192/8192 00000100:00100000:1.0:1780298645.481021:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6464 00002000:00100000:1.0:1780298645.481030:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x12:0x0] (2240512->2248703) req@ffff9cf2548bbb80 x1866777609692288/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/8632 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 8192/8192 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.481046:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2548bbb80 with x1866777609692288 ext(2240512->2248703) 00002000:00010000:1.0:1780298645.481050:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2548bbb80 00000100:00100000:1.0:1780298645.482172:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.482210:0:9976:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609692416 00000100:00100000:0.0:1780298645.482216:0:9976:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609692416 00000100:00100000:0.0:1780298645.482249:0:9976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62075 00000100:00100000:0.0:1780298645.482256:0:9976:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481faa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+40:29360:x1866777609692416:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.482278:0:9976:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.482300:0:9976:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3a00/0xaf35132e63035e5a lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 16 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4cde expref: -99 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.482383:0:9976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2600/0xaf35132e63035e61 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 17 type: IBT flags: 0x40000000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.482395:0:9976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2600/0xaf35132e63035e61 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 17 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.482407:0:9976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2600/0xaf35132e63035e61 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 17 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1780298645.482437:0:9976:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 7164934 00000004:00200000:0.0:1780298645.482449:0:9976:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1780298645.482470:0:9976:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2600/0xaf35132e63035e61 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 17 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.482489:0:9976:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3a00/0xaf35132e63035e5a lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 17 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cde expref: 42 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.482504:0:9976:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2600/0xaf35132e63035e61 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 16 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cde expref: 41 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.482513:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298645.482516:0:9976:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1780298645.482520:0:9976:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2600, rc 0) 00000100:00100000:0.0:1780298645.482557:0:9976:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481faa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+41:29360:x1866777609692416:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 301us (388us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.482564:0:9976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62075 00000100:00100000:2.0:1780298645.483626:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.483668:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609692544 00000100:00100000:1.0:1780298645.483675:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609692544 00000100:00100000:1.0:1780298645.483714:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10338 00000100:00100000:1.0:1780298645.483722:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609692544:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.483734:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.483745:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e63035e68 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4ce5 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.483796:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605091456:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.483864:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e480 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.484700:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605091456:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.484725:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e63035e68 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4ce5 expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.484749:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e63035e68 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4ce5 expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.484760:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279800, rc 0) 00000100:00100000:1.0:1780298645.484799:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609692544:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1078us (1175us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.484806:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10338 00000100:00100000:3.0:1780298645.485917:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.485963:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609692672 00000100:00100000:1.0:1780298645.485969:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609692672 00000100:00100000:1.0:1780298645.486012:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10339 00000100:00100000:1.0:1780298645.486022:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609692672:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.486034:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.486046:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63035e6f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4cec expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.486092:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605091584:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.486185:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e480 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.490261:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605091584:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.490308:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63035e6f lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cec expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.490333:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63035e6f lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cec expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.490346:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279200, rc 0) 00000100:00100000:1.0:1780298645.490410:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609692672:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 4391us (4501us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.490419:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10339 00000100:00100000:3.0:1780298645.491618:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.491666:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609692800 00000100:00100000:2.0:1780298645.491674:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609692800 00000100:00100000:2.0:1780298645.491739:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10340 00000100:00100000:2.0:1780298645.491749:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609692800:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.491765:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.491781:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e0a0f245/0xaf35132e63035e76 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4cf3 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.491830:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605091712:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.491892:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c840 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.492709:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605091712:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.492734:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e0a0f245/0xaf35132e63035e76 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cf3 expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.492750:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e0a0f245/0xaf35132e63035e76 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cf3 expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.492758:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b200, rc 0) 00000100:00100000:2.0:1780298645.492787:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609692800:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1041us (1176us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.492793:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10340 00000100:00100000:0.0:1780298645.493773:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.493818:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609692928 00000100:00100000:3.0:1780298645.493823:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609692928 00000100:00100000:3.0:1780298645.493860:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10341 00000100:00100000:3.0:1780298645.493869:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609692928:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.493878:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.493887:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e06fa3fe/0xaf35132e63035e7d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4cfa expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.493923:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605091840:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.493970:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856000 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.494691:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24310ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605091840:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.494726:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e06fa3fe/0xaf35132e63035e7d lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cfa expref: 12 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.494740:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e06fa3fe/0xaf35132e63035e7d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cfa expref: 12 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.494748:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341693e00, rc 0) 00000100:00100000:0.0:1780298645.494773:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609692928:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 908us (1004us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.494778:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10341 00000100:00100000:1.0:1780298645.495980:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.496029:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609693056 00000100:00100000:2.0:1780298645.496035:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609693056 00000100:00100000:2.0:1780298645.496075:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10342 00000100:00100000:2.0:1780298645.496083:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609693056:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.496096:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.496110:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e63035e84 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4d01 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.496173:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605091968:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.496230:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.497109:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605091968:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.497164:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e63035e84 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d01 expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.497190:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e63035e84 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d01 expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.497202:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9a600, rc 0) 00000100:00100000:1.0:1780298645.497237:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609693056:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1155us (1263us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.497248:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10342 00000100:00100000:3.0:1780298645.498461:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.498505:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609693184 00000100:00100000:1.0:1780298645.498513:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609693184 00000100:00100000:1.0:1780298645.498553:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10343 00000100:00100000:1.0:1780298645.498563:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609693184:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.498576:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.498590:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63035e8b lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4d08 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.498635:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fa300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605092096:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.498697:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e480 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.499791:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fa300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605092096:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.499815:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63035e8b lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d08 expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.499835:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63035e8b lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d08 expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.499851:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279200, rc 0) 00000100:00100000:1.0:1780298645.499897:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609693184:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1335us (1440us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.499905:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10343 00000100:00100000:2.0:1780298645.501065:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.501111:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609693312 00000100:00100000:3.0:1780298645.501118:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609693312 00000100:00100000:3.0:1780298645.501189:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10344 00000100:00100000:3.0:1780298645.501202:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609693312:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.501217:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.501232:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d1b192d6/0xaf35132e63035e92 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4d0f expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.501254:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(50 122) 00010000:00010000:3.0:1780298645.501259:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(50 122) 00010000:00010000:3.0:1780298645.501280:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d1b192d6/0xaf35132e63035e92 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2408448->2420735) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d0f expref: 11 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.501298:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d1b192d6/0xaf35132e63035e92 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2408448->2420735) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d0f expref: 11 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.501308:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341692400, rc 0) 00000100:00100000:3.0:1780298645.501343:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:29360:x1866777609693312:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 145us (279us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298645.501350:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10344 00000100:00100000:3.0:1780298645.502374:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.502422:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609693440 00000100:00100000:1.0:1780298645.502427:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609693440 00002000:00100000:1.0:1780298645.502462:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x12:0x0] (2408448->2420735) req@ffff9cf24310ed80 x1866777609693440/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.502472:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24310ed80 with x1866777609693440 ext(2408448->2420735) 00002000:00010000:1.0:1780298645.502478:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24310ed80 00000100:00100000:1.0:1780298645.502510:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6465 00000100:00100000:1.0:1780298645.502518:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609693440:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298645.502680:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609693440:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 162us (306us total) trans 0 rc 12288/12288 00000100:00100000:1.0:1780298645.502688:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6465 00002000:00100000:1.0:1780298645.502695:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x12:0x0] (2408448->2420735) req@ffff9cf24310ed80 x1866777609693440/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/12728 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 12288/12288 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.502705:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24310ed80 with x1866777609693440 ext(2408448->2420735) 00002000:00010000:1.0:1780298645.502708:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24310ed80 00000100:00100000:1.0:1780298645.504445:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.504550:0:17578:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609693696 00000100:00100000:3.0:1780298645.504558:0:17578:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609693696 00000100:00100000:3.0:1780298645.504604:0:17578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62076 00000100:00100000:3.0:1780298645.504613:0:17578:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+41:29360:x1866777609693696:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:3.0:1780298645.504659:0:17578:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298645.504685:0:17578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2600/0xaf35132e63035e61 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 17 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cde expref: 41 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.504703:0:17578:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298645.504708:0:17578:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1780298645.504859:0:17578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693400/0xaf35132e63035e99 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 16 type: IBT flags: 0x40000000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.504874:0:17578:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2000/0xaf35132e63035e4c lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 16 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cd0 expref: 32 pid: 17567 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.504884:0:17578:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693400/0xaf35132e63035e99 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 16 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.504917:0:17578:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2000/0xaf35132e63035e4c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 16 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cd0 expref: 32 pid: 17567 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.504931:0:17578:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2000/0xaf35132e63035e4c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 16 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cd0 expref: 32 pid: 17567 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.504946:0:17578:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000_UUID:17578:1866777605092224:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298645.505017:0:17578:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856000 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.506699:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.506712:0:17578:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24310ea00 pname:cluuid:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000_UUID:17578:1866777605092224:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298645.506748:0:17578:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693400/0xaf35132e63035e99 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 16 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.506759:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609693824 00000100:00100000:0.0:1780298645.506766:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609693824 00010000:00010000:0.0:1780298645.506792:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2000/0xaf35132e63035e4c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 16 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cd0 expref: 33 pid: 17567 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.506823:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15286 00000100:00100000:0.0:1780298645.506833:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3487c7c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+33:2399:x1866777609693824:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298645.506845:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298645.506850:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2000/0xaf35132e63035e4c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 17 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cd0 expref: 33 pid: 17567 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.506866:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2000/0xaf35132e63035e4c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 17 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cd0 expref: 33 pid: 17567 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.506883:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2000/0xaf35132e63035e4c lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 17 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cd0 expref: 33 pid: 17567 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.506894:0:7400:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298645.506899:0:7400:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693400/0xaf35132e63035e99 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 16 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.506908:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693400/0xaf35132e63035e99 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 16 type: IBT flags: 0x40290000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.506915:0:7400:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693400/0xaf35132e63035e99 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 16 type: IBT flags: 0x40290000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.506935:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1780298645.506957:0:17578:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693400/0xaf35132e63035e99 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 15 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.506966:0:17578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693400/0xaf35132e63035e99 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 15 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.506980:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3487c7c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+32:2399:x1866777609693824:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 145us (281us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.506988:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15286 00000001:00080000:3.0:1780298645.507041:0:17578:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991118 00000004:00010000:3.0:1780298645.507072:0:17578:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 0000000088117421 reply state 00000000d40b8bad transno 4294991119 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693400/0xaf35132e63035e99 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 15 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.507146:0:17578:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341693400/0xaf35132e63035e99 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 15 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.507155:0:17578:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298645.507158:0:17578:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1780298645.507161:0:17578:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1780298645.507224:0:17578:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+43:29360:x1866777609693696:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 2613us (2785us total) trans 4294991119 rc 0/0 00000100:00100000:3.0:1780298645.507232:0:17578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62076 00000020:00080000:3.0:1780298645.507237:0:17578:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1780298645.508007:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.508047:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609693952 00000100:00100000:0.0:1780298645.508052:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609693952 00000100:00100000:0.0:1780298645.508087:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10345 00000100:00100000:0.0:1780298645.508097:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:29360:x1866777609693952:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.508109:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.508122:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e63035ea0 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4d16 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.508172:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63035e37 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cc2 expref: 9 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.508182:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(51 169) 00010000:00010000:0.0:1780298645.508187:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(52 169) 00010000:00010000:0.0:1780298645.508192:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e63035ea0 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d16 expref: 12 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.508216:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63035e37 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cc2 expref: 9 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.508230:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63035e37 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cc2 expref: 9 pid: 8359 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.508243:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605092352:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298645.508304:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1540 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.508923:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605092352:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298645.508936:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e63035ea0 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d16 expref: 12 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.508947:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3200, rc 0) 00000100:00100000:3.0:1780298645.508967:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:29360:x1866777609693952:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 875us (963us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298645.508972:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10345 00000100:00100000:1.0:1780298645.509121:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.509183:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609694080 00000100:00100000:2.0:1780298645.509188:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609694080 00010000:00010000:2.0:1780298645.509203:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63035e37 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cc2 expref: 10 pid: 8359 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.509224:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15287 00000100:00100000:2.0:1780298645.509229:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609694080:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298645.509236:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298645.509250:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63035e37 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cc2 expref: 10 pid: 8359 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.509263:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63035e37 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cc2 expref: 10 pid: 8359 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.509287:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63035e37 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cc2 expref: 10 pid: 8359 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.509294:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(52 169) 00010000:00010000:2.0:1780298645.509297:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(52 169) 00010000:00010000:2.0:1780298645.509300:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e63035ea0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d16 expref: 11 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.509306:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e63035ea0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d16 expref: 11 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.509323:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e63035ea0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d16 expref: 11 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.509329:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255091f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777605092480:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298645.509358:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c900 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.509991:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255091f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777605092480:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298645.510001:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298645.510036:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609694080:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 806us (915us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.510044:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15287 00000100:00100000:0.0:1780298645.510118:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.510182:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609694208 00000100:00100000:2.0:1780298645.510187:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609694208 00000100:00100000:2.0:1780298645.510220:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10346 00000100:00100000:2.0:1780298645.510230:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609694208:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.510242:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.510252:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e63035ea7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4d1d expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.510286:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000027d3a930/0xaf35132e63035e53 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2240512->2248703) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cd7 expref: 8 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.510297:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d1b192d6/0xaf35132e63035e92 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2408448->2420735) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d0f expref: 10 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.510305:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(50 122) 00010000:00010000:2.0:1780298645.510309:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(50 122) 00010000:00010000:2.0:1780298645.510314:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e63035ea7 lrc: 5/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d1d expref: 8 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.510332:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d1b192d6/0xaf35132e63035e92 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2408448->2420735) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d0f expref: 10 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.510341:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d1b192d6/0xaf35132e63035e92 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2408448->2420735) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d0f expref: 10 pid: 8356 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.510351:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255090a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605092608:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298645.510393:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000027d3a930/0xaf35132e63035e53 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2240512->2248703) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cd7 expref: 8 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.510408:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000027d3a930/0xaf35132e63035e53 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2240512->2248703) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cd7 expref: 8 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.510418:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255093b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605092736:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298645.510440:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.511096:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255090a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605092608:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298645.511252:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255093b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605092736:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298645.511267:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e63035ea7 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d1d expref: 8 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.511291:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9ac00, rc 0) 00000100:00100000:2.0:1780298645.511320:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:29360:x1866777609694208:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1093us (1202us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.511326:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10346 00000100:00100000:0.0:1780298645.511653:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.511701:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609694464 00000100:00100000:1.0:1780298645.511707:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609694464 00010000:00010000:1.0:1780298645.511730:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000027d3a930/0xaf35132e63035e53 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2240512->2248703) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cd7 expref: 8 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.511755:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15288 00000100:00100000:1.0:1780298645.511761:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b43b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609694464:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298645.511771:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298645.511783:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000027d3a930/0xaf35132e63035e53 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2240512->2248703) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cd7 expref: 8 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.511795:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000027d3a930/0xaf35132e63035e53 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2240512->2248703) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cd7 expref: 8 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.511817:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027d3a930/0xaf35132e63035e53 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 2240512->2248703) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4cd7 expref: 8 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.511834:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:1.0:1780298645.511847:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298645.511876:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609694464:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 114us (222us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298645.511883:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15288 00000100:00100000:2.0:1780298645.511887:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609694336 00000100:00100000:2.0:1780298645.511892:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609694336 00010000:00010000:2.0:1780298645.511912:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d1b192d6/0xaf35132e63035e92 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2408448->2420735) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d0f expref: 11 pid: 8356 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.511941:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15289 00000100:00100000:2.0:1780298645.511950:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b43480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609694336:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298645.511959:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298645.511969:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d1b192d6/0xaf35132e63035e92 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2408448->2420735) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d0f expref: 11 pid: 8356 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.511979:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d1b192d6/0xaf35132e63035e92 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2408448->2420735) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d0f expref: 11 pid: 8356 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.511991:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d1b192d6/0xaf35132e63035e92 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2408448->2420735) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d0f expref: 11 pid: 8356 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.512001:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(50 122) 00010000:00010000:2.0:1780298645.512005:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(50 122) 00010000:00010000:2.0:1780298645.512010:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e63035ea7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d1d expref: 6 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.512018:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e63035ea7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d1d expref: 6 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.512039:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e63035ea7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d1d expref: 6 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.512050:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255093b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6461:1866777605092864:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298645.512085:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.512836:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255093b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6461:1866777605092864:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298645.512845:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298645.512875:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609694336:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 927us (1060us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.512887:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15289 00000100:00100000:3.0:1780298645.513070:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.513106:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609694592 00000100:00100000:3.0:1780298645.513107:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.513112:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609694592 00000100:00100000:2.0:1780298645.513160:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609694720 00002000:00010000:1.0:1780298645.513161:0:16972:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (814241->18446744073709551615) 00000100:00100000:2.0:1780298645.513165:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609694720 00002000:00100000:1.0:1780298645.513166:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24310ea00 with x1866777609694592 ext(814241->18446744073709551615) 00002000:00010000:1.0:1780298645.513174:0:16972:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24310ea00. 00002000:00010000:2.0:1780298645.513179:0:19949:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00002000:00100000:2.0:1780298645.513183:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f755180 with x1866777609694720 ext(0->18446744073709551615) 00002000:00010000:2.0:1780298645.513189:0:19949:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f755180. 00000100:00100000:1.0:1780298645.513192:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6466 00000100:00100000:1.0:1780298645.513201:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609694592:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:0.0:1780298645.513204:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6467 00000100:00100000:0.0:1780298645.513216:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f755180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609694720:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:0.0:1780298645.515847:0:19951:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 2970630 -> 0 00002000:00010000:0.0:1780298645.515852:0:19951:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298645 -> 1780298646 00002000:00010000:0.0:1780298645.515854:0:19951:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298645.515856:0:19951:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298645 -> 1780298646 00002000:00010000:0.0:1780298645.515858:0:19951:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:0.0:1780298645.515894:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f755180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609694720:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 2680us (2787us total) trans 4294969319 rc 0/0 00000100:00100000:0.0:1780298645.515901:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6467 00002000:00010000:0.0:1780298645.515904:0:19951:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00002000:00100000:0.0:1780298645.515907:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f755180 with x1866777609694720 ext(0->18446744073709551615) 00002000:00010000:0.0:1780298645.515911:0:19951:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f755180. 00002000:00010000:1.0:1780298645.516072:0:16972:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 814241 00002000:00010000:1.0:1780298645.516077:0:16972:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298645 -> 1780298646 00002000:00010000:1.0:1780298645.516080:0:16972:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298645.516082:0:16972:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298645 -> 1780298646 00002000:00010000:1.0:1780298645.516084:0:16972:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 376 -> 376 00000100:00100000:1.0:1780298645.516117:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609694592:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 2918us (3047us total) trans 4294969779 rc 0/0 00000100:00100000:1.0:1780298645.516142:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6466 00002000:00010000:1.0:1780298645.516145:0:16972:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (814241->18446744073709551615) 00002000:00100000:1.0:1780298645.516149:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24310ea00 with x1866777609694592 ext(814241->18446744073709551615) 00002000:00010000:1.0:1780298645.516154:0:16972:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24310ea00. 00000100:00100000:3.0:1780298645.517009:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.517044:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609694848 00000100:00100000:0.0:1780298645.517050:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609694848 00000100:00100000:0.0:1780298645.517081:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62077 00000100:00100000:0.0:1780298645.517087:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f754380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+43:29360:x1866777609694848:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.517103:0:6477:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.517117:0:6477:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3600/0xaf35132e63035eae lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 16 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4d24 expref: -99 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.517203:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2200/0xaf35132e63035eb5 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 17 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.517216:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2200/0xaf35132e63035eb5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 17 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.517228:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2200/0xaf35132e63035eb5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 17 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1780298645.517246:0:6477:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 814241 00000004:00200000:0.0:1780298645.517249:0:6477:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1780298645.517268:0:6477:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2200/0xaf35132e63035eb5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 17 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.517303:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3600/0xaf35132e63035eae lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 17 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d24 expref: 45 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.517314:0:6477:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2200/0xaf35132e63035eb5 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 16 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d24 expref: 44 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.517320:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298645.517322:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1780298645.517325:0:6477:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2200, rc 0) 00000100:00100000:0.0:1780298645.517347:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f754380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+44:29360:x1866777609694848:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 261us (343us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.517351:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62077 00000100:00100000:0.0:1780298645.518469:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.518503:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609694976 00000100:00100000:1.0:1780298645.518508:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609694976 00000100:00100000:1.0:1780298645.518533:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10347 00000100:00100000:1.0:1780298645.518539:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556ab800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609694976:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.518551:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.518559:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63035ebc lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4d2b expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.518576:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e63035ea0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d16 expref: 12 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.518583:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(52 169) 00010000:00010000:1.0:1780298645.518586:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(53 169) 00010000:00010000:1.0:1780298645.518589:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63035ebc lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [503808->516095] (req 503808->516095) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d2b expref: 10 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.518613:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e63035ea0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d16 expref: 12 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.518624:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e63035ea0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d16 expref: 12 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.518634:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605092992:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298645.518679:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e900 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.519305:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605092992:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298645.519320:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63035ebc lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [503808->516095] (req 503808->516095) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d2b expref: 10 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.519329:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279200, rc 0) 00000100:00100000:1.0:1780298645.519352:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556ab800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609694976:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 814us (885us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298645.519360:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10347 00000100:00100000:3.0:1780298645.519461:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.519489:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609695104 00000100:00100000:1.0:1780298645.519493:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609695104 00010000:00010000:1.0:1780298645.519506:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e63035ea0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d16 expref: 13 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.519527:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15290 00000100:00100000:1.0:1780298645.519531:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f754a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2399:x1866777609695104:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298645.519540:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298645.519552:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e63035ea0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d16 expref: 13 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.519564:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e63035ea0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d16 expref: 13 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.519581:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e63035ea0 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d16 expref: 13 pid: 12101 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.519592:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(53 169) 00010000:00010000:1.0:1780298645.519596:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(53 169) 00010000:00010000:1.0:1780298645.519604:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63035ebc lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 503808->516095) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d2b expref: 9 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.519614:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63035ebc lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 503808->516095) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d2b expref: 9 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.519631:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63035ebc lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 503808->516095) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d2b expref: 9 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.519638:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fa680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777605093120:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298645.519665:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e900 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.520347:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fa680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777605093120:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298645.520358:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298645.520389:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f754a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609695104:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 856us (927us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298645.520398:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15290 00000100:00100000:3.0:1780298645.520659:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.520704:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609695232 00000100:00100000:2.0:1780298645.520709:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609695232 00002000:00100000:2.0:1780298645.520737:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (503808->516095) req@ffff9cf37f755f80 x1866777609695232/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.520746:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f755f80 with x1866777609695232 ext(503808->516095) 00002000:00010000:2.0:1780298645.520750:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f755f80 00000100:00100000:2.0:1780298645.520771:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6468 00000100:00100000:2.0:1780298645.520778:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f755f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609695232:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298645.520942:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f755f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609695232:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 165us (284us total) trans 0 rc 12288/12288 00000100:00100000:2.0:1780298645.520948:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6468 00002000:00100000:2.0:1780298645.520952:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (503808->516095) req@ffff9cf37f755f80 x1866777609695232/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/12728 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 12288/12288 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.520959:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f755f80 with x1866777609695232 ext(503808->516095) 00002000:00010000:2.0:1780298645.520962:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f755f80 00000100:00100000:3.0:1780298645.522029:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.522069:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609695360 00000100:00100000:2.0:1780298645.522073:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609695360 00000100:00100000:2.0:1780298645.522109:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10348 00000100:00100000:2.0:1780298645.522119:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f757100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609695360:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.522166:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.522179:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63035ec3 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4d32 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.522197:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(53 169) 00010000:00010000:2.0:1780298645.522201:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(53 169) 00010000:00010000:2.0:1780298645.522208:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63035ec3 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d32 expref: 13 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.522221:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63035ec3 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d32 expref: 13 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.522231:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9aa00, rc 0) 00000100:00100000:2.0:1780298645.522283:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f757100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609695360:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 155us (242us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.522291:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10348 00000100:00100000:0.0:1780298645.523410:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.523438:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609695488 00000100:00100000:3.0:1780298645.523443:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609695488 00000100:00100000:3.0:1780298645.523479:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10349 00000100:00100000:3.0:1780298645.523487:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5dad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609695488:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.523495:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.523503:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e06fa3fe/0xaf35132e63035eca lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4d39 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.523534:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f754000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605093248:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.523587:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.524244:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f754000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605093248:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.524284:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e06fa3fe/0xaf35132e63035eca lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d39 expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.524296:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e06fa3fe/0xaf35132e63035eca lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d39 expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.524304:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341693e00, rc 0) 00000100:00100000:0.0:1780298645.524326:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5dad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609695488:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 842us (919us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.524331:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10349 00000100:00100000:1.0:1780298645.525296:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.525335:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609695616 00000100:00100000:2.0:1780298645.525340:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609695616 00002000:00100000:2.0:1780298645.525364:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (352256->356351) req@ffff9cf3481fa680 x1866777609695616/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.525371:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fa680 with x1866777609695616 ext(352256->356351) 00002000:00010000:2.0:1780298645.525374:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481fa680 00000100:00100000:2.0:1780298645.525390:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6469 00000100:00100000:2.0:1780298645.525395:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609695616:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298645.525495:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609695616:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 100us (201us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298645.525500:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6469 00002000:00100000:2.0:1780298645.525505:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (352256->356351) req@ffff9cf3481fa680 x1866777609695616/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/4536 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.525510:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fa680 with x1866777609695616 ext(352256->356351) 00002000:00010000:2.0:1780298645.525513:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481fa680 00000100:00100000:2.0:1780298645.526551:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.526587:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609695744 00000100:00100000:1.0:1780298645.526591:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609695744 00002000:00100000:1.0:1780298645.526610:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (356352->360447) req@ffff9cf255093b80 x1866777609695744/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.526617:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255093b80 with x1866777609695744 ext(356352->360447) 00002000:00010000:1.0:1780298645.526619:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255093b80 00000100:00100000:1.0:1780298645.526634:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6470 00000100:00100000:1.0:1780298645.526639:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255093b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609695744:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298645.526703:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255093b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609695744:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 63us (154us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298645.526714:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6470 00002000:00100000:1.0:1780298645.526722:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (356352->360447) req@ffff9cf255093b80 x1866777609695744/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/4536 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.526730:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255093b80 with x1866777609695744 ext(356352->360447) 00002000:00010000:1.0:1780298645.526734:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255093b80 00000100:00100000:3.0:1780298645.527649:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.527685:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609695872 00000100:00100000:2.0:1780298645.527689:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609695872 00002000:00100000:2.0:1780298645.527707:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (331776->397311) req@ffff9cf37f756680 x1866777609695872/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 504/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.527716:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f756680 with x1866777609695872 ext(331776->397311) 00002000:00010000:2.0:1780298645.527720:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f756680 00000100:00100000:2.0:1780298645.527739:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6471 00000100:00100000:2.0:1780298645.527746:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f756680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609695872:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298645.528640:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f756680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609695872:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 894us (991us total) trans 0 rc 57344/57344 00000100:00100000:2.0:1780298645.528646:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6471 00002000:00100000:2.0:1780298645.528651:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (331776->397311) req@ffff9cf37f756680 x1866777609695872/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 504/440 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 57344/57344 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.528657:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f756680 with x1866777609695872 ext(331776->397311) 00002000:00010000:2.0:1780298645.528660:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f756680 00000100:00100000:3.0:1780298645.529733:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.529768:0:17567:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609696128 00000100:00100000:0.0:1780298645.529773:0:17567:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609696128 00000100:00100000:0.0:1780298645.529805:0:17567:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62078 00000100:00100000:0.0:1780298645.529812:0:17567:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f755500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+32:29360:x1866777609696128:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:0.0:1780298645.529980:0:17567:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2a00/0xaf35132e63035ed1 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 17 type: IBT flags: 0x40000000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.529994:0:17567:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2200/0xaf35132e63035eb5 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 17 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d24 expref: 43 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.530002:0:17567:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2a00/0xaf35132e63035ed1 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 17 type: IBT flags: 0x50210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.530036:0:17567:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2200/0xaf35132e63035eb5 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 17 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d24 expref: 43 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.530047:0:17567:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2200/0xaf35132e63035eb5 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 17 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d24 expref: 43 pid: 6477 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.530056:0:17567:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:17567:1866777605093376:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298645.530112:0:17567:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1540 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.530746:0:17567:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:17567:1866777605093376:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298645.530765:0:17567:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2a00/0xaf35132e63035ed1 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 17 type: IBT flags: 0x40210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.531039:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.531085:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609696256 00000100:00100000:2.0:1780298645.531090:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609696256 00010000:00010000:2.0:1780298645.531108:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2200/0xaf35132e63035eb5 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 17 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d24 expref: 44 pid: 6477 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.531153:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15291 00000100:00100000:2.0:1780298645.531162:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f756300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+44:2399:x1866777609696256:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298645.531174:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298645.531179:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2200/0xaf35132e63035eb5 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 18 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d24 expref: 44 pid: 6477 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.531191:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2200/0xaf35132e63035eb5 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 18 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d24 expref: 44 pid: 6477 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.531209:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2200/0xaf35132e63035eb5 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 18 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d24 expref: 44 pid: 6477 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.531219:0:6462:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298645.531222:0:6462:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2a00/0xaf35132e63035ed1 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 17 type: IBT flags: 0x40210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.531228:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2a00/0xaf35132e63035ed1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 17 type: IBT flags: 0x40290000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.531234:0:6462:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2a00/0xaf35132e63035ed1 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 17 type: IBT flags: 0x40290000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.531256:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1780298645.531289:0:17567:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2a00/0xaf35132e63035ed1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 16 type: IBT flags: 0x40210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.531296:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f756300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+43:2399:x1866777609696256:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 134us (254us total) trans 0 rc 0/0 00010000:00010000:0.0:1780298645.531303:0:17567:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2a00/0xaf35132e63035ed1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 16 type: IBT flags: 0x40210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.531309:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15291 00000001:00080000:0.0:1780298645.531370:0:17567:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991120 00000004:00010000:0.0:1780298645.531393:0:17567:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000bc8fca25 reply state 000000007b36f77b transno 4294991121 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2a00/0xaf35132e63035ed1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 16 type: IBT flags: 0x40210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.531402:0:17567:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2a00/0xaf35132e63035ed1 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 16 type: IBT flags: 0x50210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.531407:0:17567:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298645.531409:0:17567:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1780298645.531411:0:17567:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1780298645.531457:0:17567:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f755500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+35:29360:x1866777609696128:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1646us (1724us total) trans 4294991121 rc 0/0 00000100:00100000:0.0:1780298645.531462:0:17567:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62078 00000020:00080000:0.0:1780298645.531465:0:17567:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1780298645.532048:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.532080:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609696384 00000100:00100000:2.0:1780298645.532084:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609696384 00000100:00100000:2.0:1780298645.532107:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10350 00000100:00100000:2.0:1780298645.532111:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f757b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609696384:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.532119:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.532146:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63035ed8 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4d40 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.532162:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63035ebc lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 503808->516095) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d2b expref: 11 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.532169:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63035ec3 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d32 expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.532174:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(53 169) 00010000:00010000:2.0:1780298645.532177:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(53 169) 00010000:00010000:2.0:1780298645.532179:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63035ed8 lrc: 5/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d40 expref: 11 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.532192:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63035ec3 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d32 expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.532199:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63035ec3 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d32 expref: 12 pid: 8356 timeout: 934 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.532207:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255090a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605093504:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298645.532243:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63035ebc lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 503808->516095) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d2b expref: 11 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.532254:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63035ebc lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 503808->516095) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d2b expref: 11 pid: 8359 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.532264:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605093632:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298645.532311:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.532845:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255090a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605093504:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298645.533102:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.533152:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605093632:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298645.533157:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609696512 00000100:00100000:0.0:1780298645.533163:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609696512 00010000:00010000:2.0:1780298645.533179:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63035ed8 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d40 expref: 11 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.533182:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63035ec3 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d32 expref: 13 pid: 8356 timeout: 934 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.533208:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9be00, rc 0) 00000100:00100000:0.0:1780298645.533223:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15292 00000100:00100000:0.0:1780298645.533231:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f755c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2399:x1866777609696512:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298645.533241:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1780298645.533245:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f757b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:29360:x1866777609696384:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1129us (1194us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.533254:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10350 00010000:00010000:0.0:1780298645.533277:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63035ec3 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d32 expref: 13 pid: 8356 timeout: 934 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.533297:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63035ec3 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d32 expref: 13 pid: 8356 timeout: 934 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.533312:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63035ec3 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d32 expref: 13 pid: 8356 timeout: 934 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.533323:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298645.533353:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f755c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609696512:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 124us (250us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.533360:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15292 00000100:00100000:3.0:1780298645.533425:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.533457:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609696640 00000100:00100000:0.0:1780298645.533461:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609696640 00010000:00010000:0.0:1780298645.533476:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63035ebc lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 503808->516095) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d2b expref: 11 pid: 8359 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.533498:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15293 00000100:00100000:0.0:1780298645.533505:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3487c4c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609696640:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298645.533513:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298645.533522:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63035ebc lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 503808->516095) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d2b expref: 11 pid: 8359 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.533534:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63035ebc lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 503808->516095) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d2b expref: 11 pid: 8359 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.533553:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63035ebc lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 503808->516095) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d2b expref: 11 pid: 8359 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.533562:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(53 169) 00010000:00010000:0.0:1780298645.533566:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(53 169) 00010000:00010000:0.0:1780298645.533571:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63035ed8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d40 expref: 10 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.533580:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63035ed8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d40 expref: 10 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.533601:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63035ed8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d40 expref: 10 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.533612:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777605093760:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298645.533650:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.534310:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777605093760:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298645.534320:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298645.534343:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3487c4c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609696640:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 838us (916us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.534351:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15293 00000100:00100000:3.0:1780298645.534384:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.534410:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609696768 00000100:00100000:0.0:1780298645.534414:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609696768 00000100:00100000:0.0:1780298645.534445:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10351 00000100:00100000:0.0:1780298645.534454:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f754a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609696768:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.534464:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.534474:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e63035edf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4d47 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.534490:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e63035ea7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d1d expref: 7 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.534498:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(50 122) 00010000:00010000:0.0:1780298645.534501:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(51 122) 00010000:00010000:0.0:1780298645.534506:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e63035edf lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d47 expref: 11 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.534523:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e63035ea7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d1d expref: 7 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.534534:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e63035ea7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d1d expref: 7 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.534544:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605093888:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298645.534576:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4f00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.535089:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605093888:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298645.535103:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e63035edf lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d47 expref: 11 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.535117:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3800, rc 0) 00000100:00100000:0.0:1780298645.535161:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f754a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:29360:x1866777609696768:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 709us (779us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.535166:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10351 00000100:00100000:1.0:1780298645.535365:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.535400:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609696896 00000100:00100000:0.0:1780298645.535404:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609696896 00010000:00010000:0.0:1780298645.535421:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e63035ea7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d1d expref: 8 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.535454:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15294 00000100:00100000:0.0:1780298645.535462:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609696896:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298645.535471:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298645.535483:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e63035ea7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d1d expref: 8 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.535500:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e63035ea7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d1d expref: 8 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.535515:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e63035ea7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d1d expref: 8 pid: 8358 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.535524:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(51 122) 00010000:00010000:0.0:1780298645.535527:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(51 122) 00010000:00010000:0.0:1780298645.535533:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e63035edf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d47 expref: 10 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.535549:0:6463:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e63035edf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d47 expref: 10 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.535569:0:6463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e63035edf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d47 expref: 10 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.535579:0:6463:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777605094016:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298645.535616:0:6463:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4f00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.536284:0:6463:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777605094016:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298645.536294:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298645.536320:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609696896:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 859us (956us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.536328:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15294 00000100:00100000:1.0:1780298645.536375:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.536409:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609697024 00000100:00100000:2.0:1780298645.536413:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609697024 00002000:00010000:2.0:1780298645.536429:0:16972:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (1124045->18446744073709551615) 00002000:00100000:2.0:1780298645.536432:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481f9c00 with x1866777609697024 ext(1124045->18446744073709551615) 00002000:00010000:2.0:1780298645.536438:0:16972:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481f9c00. 00000100:00100000:2.0:1780298645.536456:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6472 00000100:00100000:2.0:1780298645.536462:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609697024:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:3.0:1780298645.536516:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.536553:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609697152 00000100:00100000:0.0:1780298645.536558:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609697152 00002000:00010000:0.0:1780298645.536573:0:8364:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00002000:00100000:0.0:1780298645.536576:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f756300 with x1866777609697152 ext(0->18446744073709551615) 00002000:00010000:0.0:1780298645.536582:0:8364:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f756300. 00000100:00100000:0.0:1780298645.536608:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6473 00002000:00010000:2.0:1780298645.536611:0:16972:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 814241 -> 1124045 00002000:00010000:2.0:1780298645.536615:0:16972:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:2.0:1780298645.536619:0:16972:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298645.536622:0:16972:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:2.0:1780298645.536624:0:16972:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 376 -> 376 00000100:00100000:0.0:1780298645.536627:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f756300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609697152:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:2.0:1780298645.536658:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609697024:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 195us (284us total) trans 4294969780 rc 0/0 00000100:00100000:2.0:1780298645.536667:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6472 00002000:00010000:2.0:1780298645.536671:0:16972:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (1124045->18446744073709551615) 00002000:00100000:2.0:1780298645.536674:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481f9c00 with x1866777609697024 ext(1124045->18446744073709551615) 00002000:00010000:2.0:1780298645.536679:0:16972:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481f9c00. 00002000:00010000:0.0:1780298645.536703:0:8364:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:0.0:1780298645.536707:0:8364:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298645.536710:0:8364:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298645.536713:0:8364:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298645.536715:0:8364:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:0.0:1780298645.536748:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f756300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2399:x1866777609697152:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 133us (233us total) trans 4294969320 rc 0/0 00000100:00100000:0.0:1780298645.536756:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6473 00002000:00010000:0.0:1780298645.536760:0:8364:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00002000:00100000:0.0:1780298645.536763:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f756300 with x1866777609697152 ext(0->18446744073709551615) 00002000:00010000:0.0:1780298645.536768:0:8364:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f756300. 00000100:00100000:3.0:1780298645.538222:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.538249:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609697280 00000100:00100000:2.0:1780298645.538253:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609697280 00002000:00100000:2.0:1780298645.538282:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (1089536->1126399) req@ffff9cf37f755c00 x1866777609697280/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.538290:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f755c00 with x1866777609697280 ext(1089536->1126399) 00002000:00010000:2.0:1780298645.538293:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f755c00 00000100:00100000:2.0:1780298645.538308:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6474 00000100:00100000:2.0:1780298645.538313:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f755c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609697280:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298645.539196:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f755c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609697280:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 881us (975us total) trans 0 rc 36864/36864 00000100:00100000:2.0:1780298645.539206:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6474 00002000:00100000:2.0:1780298645.539211:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (1089536->1126399) req@ffff9cf37f755c00 x1866777609697280/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/440 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 36864/36864 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.539217:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f755c00 with x1866777609697280 ext(1089536->1126399) 00002000:00010000:2.0:1780298645.539220:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f755c00 00000100:00100000:1.0:1780298645.540538:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.540572:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609697408 00000100:00100000:3.0:1780298645.540610:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609697408 00002000:00100000:3.0:1780298645.540639:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (1118208->1124044) req@ffff9cf3481f8700 x1866777609697408/t0(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 6328/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298645.540649:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481f8700 with x1866777609697408 ext(1118208->1124044) 00002000:00010000:3.0:1780298645.540653:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481f8700 00000100:00100000:3.0:1780298645.540674:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6475 00000100:00100000:3.0:1780298645.540680:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609697408:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:3.0:1780298645.541118:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609697408:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 437us (580us total) trans 4294969781 rc 0/0 00000100:00100000:3.0:1780298645.541146:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6475 00002000:00100000:3.0:1780298645.541155:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (1118208->1124044) req@ffff9cf3481f8700 x1866777609697408/t4294969781(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 6328/448 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298645.541163:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481f8700 with x1866777609697408 ext(1118208->1124044) 00002000:00010000:3.0:1780298645.541167:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481f8700 00000100:00100000:1.0:1780298645.542262:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.542316:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609697536 00000100:00100000:2.0:1780298645.542322:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609697536 00000100:00100000:2.0:1780298645.542350:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10352 00000100:00100000:2.0:1780298645.542355:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609697536:12345-192.168.203.25@tcp:16:fsx.0 00000001:00080000:0.0:1780298645.543117:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969781 is committed 00000001:00080000:0.0:1780298645.543151:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969779 is committed 00000100:00100000:2.0:1780298645.543228:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609697536:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 872us (965us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.543234:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10352 00000100:00100000:1.0:1780298645.543964:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.543999:0:9976:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609697664 00000100:00100000:0.0:1780298645.544005:0:9976:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609697664 00000100:00100000:0.0:1780298645.544040:0:9976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62079 00000100:00100000:0.0:1780298645.544048:0:9976:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+35:29360:x1866777609697664:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.544064:0:9976:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.544084:0:9976:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3a00/0xaf35132e63035ee6 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 17 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4d4e expref: -99 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.544190:0:9976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2e00/0xaf35132e63035eed lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 18 type: IBT flags: 0x40000000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.544208:0:9976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2e00/0xaf35132e63035eed lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 18 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.544220:0:9976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2e00/0xaf35132e63035eed lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 18 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1780298645.544241:0:9976:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 1124045 00000004:00200000:0.0:1780298645.544246:0:9976:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1780298645.544261:0:9976:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2e00/0xaf35132e63035eed lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 18 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.544298:0:9976:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3a00/0xaf35132e63035ee6 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 18 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d4e expref: 37 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.544314:0:9976:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2e00/0xaf35132e63035eed lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 17 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d4e expref: 36 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.544323:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298645.544325:0:9976:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1780298645.544329:0:9976:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2e00, rc 0) 00000100:00100000:0.0:1780298645.544360:0:9976:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+36:29360:x1866777609697664:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 313us (396us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.544366:0:9976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62079 00000100:00100000:1.0:1780298645.545768:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.545805:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609697792 00000100:00100000:2.0:1780298645.545810:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609697792 00000100:00100000:2.0:1780298645.545840:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6476 00000100:00100000:2.0:1780298645.545849:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609697792:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298645.545978:0:8362:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:2.0:1780298645.545981:0:8362:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:2.0:1780298645.545983:0:8362:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298645.545984:0:8362:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:2.0:1780298645.545986:0:8362:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 0 -> 24 00000100:00100000:2.0:1780298645.546011:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609697792:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 163us (245us total) trans 4294969321 rc 0/0 00000100:00100000:2.0:1780298645.546015:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6476 00000100:00100000:1.0:1780298645.546856:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.546893:0:17578:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609697920 00000100:00100000:3.0:1780298645.546897:0:17578:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609697920 00000100:00100000:3.0:1780298645.546923:0:17578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62080 00000100:00100000:3.0:1780298645.546928:0:17578:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348007b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+43:29360:x1866777609697920:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.546937:0:17578:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.546946:0:17578:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692400/0xaf35132e63035ef4 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 18 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4d55 expref: -99 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.546992:0:17578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692e00/0xaf35132e63035efb lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 19 type: IBT flags: 0x40000000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.547002:0:17578:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692e00/0xaf35132e63035efb lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 19 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.547009:0:17578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692e00/0xaf35132e63035efb lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 19 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1780298645.547024:0:17578:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 1124045 00000004:00200000:3.0:1780298645.547027:0:17578:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1780298645.547040:0:17578:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692e00/0xaf35132e63035efb lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 19 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.547071:0:17578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692400/0xaf35132e63035ef4 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 19 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d55 expref: 45 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.547087:0:17578:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692e00/0xaf35132e63035efb lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 18 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d55 expref: 44 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.547093:0:17578:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298645.547095:0:17578:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1780298645.547096:0:17578:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341692e00, rc 0) 00000100:00100000:3.0:1780298645.547121:0:17578:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348007b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+44:29360:x1866777609697920:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 192us (267us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298645.547158:0:17578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62080 00000100:00100000:1.0:1780298645.547931:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.547970:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609698048 00000100:00100000:0.0:1780298645.547975:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609698048 00000100:00100000:0.0:1780298645.548010:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10353 00000100:00100000:0.0:1780298645.548018:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348004e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609698048:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.548027:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.548038:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000488a7b88/0xaf35132e63035f02 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4d5c expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.548055:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.548091:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605094144:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.548107:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609698176 00000100:00100000:2.0:1780298645.548112:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609698176 00000100:00100000:2.0:1780298645.548175:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10354 00000100:00100000:2.0:1780298645.548185:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f754000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609698176:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298645.548186:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4f00 going to sleep for 11 seconds 00010000:00010000:2.0:1780298645.548196:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.548206:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e63035f09 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4d63 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.548238:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605094272:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.548284:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00002000:00010000:0.0:1780298645.548952:0:12101:0:(ofd_lvb.c:271:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime: 1780298645 -> 1780298646 00002000:00010000:0.0:1780298645.548955:0:12101:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks: 376 -> 392 00000100:00100000:0.0:1780298645.548966:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605094144:192.168.203.25@tcp:106: 00002000:00010000:2.0:1780298645.548983:0:8358:0:(ofd_lvb.c:271:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime: 1780298645 -> 1780298646 00010000:00010000:0.0:1780298645.548988:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000488a7b88/0xaf35132e63035f02 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d5c expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.548997:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605094272:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.549004:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000488a7b88/0xaf35132e63035f02 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d5c expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.549013:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2200, rc 0) 00010000:00010000:2.0:1780298645.549019:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e63035f09 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d63 expref: 8 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.549034:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e63035f09 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d63 expref: 8 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.549042:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348004e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609698048:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1025us (1111us total) trans 0 rc 301/301 00010000:00010000:2.0:1780298645.549049:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b000, rc 0) 00000100:00100000:0.0:1780298645.549049:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10353 00000100:00100000:2.0:1780298645.549079:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f754000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609698176:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 896us (1022us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.549086:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10354 00000100:00100000:1.0:1780298645.550217:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.550236:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.550253:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609698304 00000100:00100000:0.0:1780298645.550258:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609698304 00000100:00100000:0.0:1780298645.550294:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609698432 00000100:00100000:0.0:1780298645.550296:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609698432 00000100:00100000:0.0:1780298645.550317:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10355 00000100:00100000:3.0:1780298645.550325:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10356 00000100:00100000:0.0:1780298645.550328:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348006680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609698304:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298645.550337:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348007480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609698432:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.550339:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.550344:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.550352:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000658b0d9c/0xaf35132e63035f10 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4d6a expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.550353:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027d3a930/0xaf35132e63035f17 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4d71 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.550395:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244adad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605094400:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.550401:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605094528:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.550440:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2418563c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.550442:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4f00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.551195:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605094528:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.551213:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000658b0d9c/0xaf35132e63035f10 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d6a expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.551224:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244adad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605094400:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.551243:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000658b0d9c/0xaf35132e63035f10 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d6a expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.551252:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2c00, rc 0) 00010000:00010000:3.0:1780298645.551260:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000027d3a930/0xaf35132e63035f17 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d71 expref: 8 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.551289:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348006680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609698304:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 965us (1073us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.551298:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10355 00010000:00010000:3.0:1780298645.551299:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027d3a930/0xaf35132e63035f17 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d71 expref: 8 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.551312:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341692200, rc 0) 00000100:00100000:3.0:1780298645.551337:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348007480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609698432:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1004us (1098us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.551344:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10356 00000100:00100000:1.0:1780298645.552399:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.552446:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609698560 00000100:00100000:0.0:1780298645.552451:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609698560 00000100:00100000:0.0:1780298645.552487:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6477 00000100:00100000:0.0:1780298645.552499:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348004380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609698560:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298645.552665:0:19943:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 1124045 -> 1124045 00002000:00010000:0.0:1780298645.552668:0:19943:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298645.552670:0:19943:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298646 -> 1780298645 00002000:00010000:0.0:1780298645.552671:0:19943:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298645.552673:0:19943:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 392 -> 392 00000100:00100000:0.0:1780298645.552698:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348004380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609698560:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 206us (303us total) trans 4294969782 rc 0/0 00000100:00100000:0.0:1780298645.552702:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6477 00000100:00100000:1.0:1780298645.555105:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.555172:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609698816 00000100:00100000:0.0:1780298645.555181:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609698816 00000100:00100000:0.0:1780298645.555228:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62081 00000100:00100000:0.0:1780298645.555239:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348005c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+36:29360:x1866777609698816:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:0.0:1780298645.555309:0:6477:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298645.555335:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2e00/0xaf35132e63035eed lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 19 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d4e expref: 36 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.555353:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298645.555358:0:6477:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1780298645.555513:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3e00/0xaf35132e63035f1e lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 18 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.555524:0:6477:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692e00/0xaf35132e63035efb lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 18 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d55 expref: 43 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.555532:0:6477:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3e00/0xaf35132e63035f1e lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 18 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.555558:0:6477:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692e00/0xaf35132e63035efb lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 18 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d55 expref: 43 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.555569:0:6477:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692e00/0xaf35132e63035efb lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 18 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d55 expref: 43 pid: 17578 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.555591:0:6477:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6477:1866777605094656:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298645.555658:0:6477:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba49c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.556396:0:6477:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6477:1866777605094656:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298645.556413:0:6477:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3e00/0xaf35132e63035f1e lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 18 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.556491:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.556521:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609698944 00000100:00100000:3.0:1780298645.556525:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609698944 00010000:00010000:3.0:1780298645.556572:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692e00/0xaf35132e63035efb lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 18 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d55 expref: 44 pid: 17578 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.556597:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15295 00000100:00100000:3.0:1780298645.556604:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242861c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+44:2399:x1866777609698944:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298645.556614:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298645.556619:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692e00/0xaf35132e63035efb lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 19 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d55 expref: 44 pid: 17578 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.556630:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692e00/0xaf35132e63035efb lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 19 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d55 expref: 44 pid: 17578 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.556643:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692e00/0xaf35132e63035efb lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 19 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d55 expref: 44 pid: 17578 timeout: 934 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.556654:0:6461:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298645.556657:0:6461:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3e00/0xaf35132e63035f1e lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 18 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.556664:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3e00/0xaf35132e63035f1e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 18 type: IBT flags: 0x40290000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.556669:0:6461:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3e00/0xaf35132e63035f1e lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 18 type: IBT flags: 0x40290000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.556687:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1780298645.556702:0:6477:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3e00/0xaf35132e63035f1e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 17 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.556713:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3e00/0xaf35132e63035f1e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 17 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.556718:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242861c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+43:2399:x1866777609698944:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 115us (227us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298645.556724:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15295 00000001:00080000:0.0:1780298645.556764:0:6477:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991122 00000004:00010000:0.0:1780298645.556790:0:6477:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000fcad4318 reply state 00000000804f6454 transno 4294991123 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3e00/0xaf35132e63035f1e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 17 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.556802:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3e00/0xaf35132e63035f1e lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 17 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.556809:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298645.556811:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1780298645.556813:0:6477:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1780298645.556869:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348005c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+38:29360:x1866777609698816:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1632us (1771us total) trans 4294991123 rc 0/0 00000100:00100000:0.0:1780298645.556877:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62081 00000020:00080000:0.0:1780298645.556881:0:6477:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1780298645.557733:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.557772:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609699072 00000100:00100000:0.0:1780298645.557777:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609699072 00002000:00010000:0.0:1780298645.557798:0:8364:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:0.0:1780298645.557803:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242863b80 with x1866777609699072 ext(4194304->18446744073709551615) 00002000:00010000:0.0:1780298645.557823:0:8364:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf242863b80. 00000100:00100000:1.0:1780298645.557843:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.557856:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609699200 00000100:00100000:0.0:1780298645.557861:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609699200 00002000:00010000:0.0:1780298645.557876:0:8364:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (2213475->18446744073709551615) 00002000:00100000:0.0:1780298645.557881:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242860000 with x1866777609699200 ext(2213475->18446744073709551615) 00000100:00100000:2.0:1780298645.557882:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6478 00002000:00010000:0.0:1780298645.557890:0:8364:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf242860000. 00000100:00100000:2.0:1780298645.557897:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242863b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609699072:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:0.0:1780298645.557914:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6479 00000100:00100000:0.0:1780298645.557925:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242860000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609699200:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:2.0:1780298645.558027:0:19949:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 1124045 -> 4194304 00002000:00010000:2.0:1780298645.558031:0:19949:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:2.0:1780298645.558034:0:19949:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298645.558037:0:19949:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:2.0:1780298645.558041:0:19949:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 392 -> 392 00002000:00010000:0.0:1780298645.558046:0:8364:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 0 -> 2213475 00002000:00010000:0.0:1780298645.558050:0:8364:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298645.558052:0:8364:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298646 -> 1780298645 00002000:00010000:0.0:1780298645.558055:0:8364:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298645.558057:0:8364:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 24 -> 24 00000100:00100000:2.0:1780298645.558089:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242863b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609699072:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 196us (356us total) trans 4294969783 rc 0/0 00000100:00100000:0.0:1780298645.558095:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242860000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609699200:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 170us (251us total) trans 4294969322 rc 0/0 00000100:00100000:2.0:1780298645.558103:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6478 00002000:00010000:2.0:1780298645.558108:0:19949:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00000100:00100000:0.0:1780298645.558111:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6479 00002000:00100000:2.0:1780298645.558113:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242863b80 with x1866777609699072 ext(4194304->18446744073709551615) 00002000:00010000:2.0:1780298645.558120:0:19949:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf242863b80. 00002000:00010000:0.0:1780298645.558121:0:8364:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (2213475->18446744073709551615) 00002000:00100000:0.0:1780298645.558166:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242860000 with x1866777609699200 ext(2213475->18446744073709551615) 00002000:00010000:0.0:1780298645.558171:0:8364:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf242860000. 00000100:00100000:1.0:1780298645.558825:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.558861:0:17567:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609699328 00000100:00100000:0.0:1780298645.558865:0:17567:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609699328 00000100:00100000:0.0:1780298645.558890:0:17567:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62082 00000100:00100000:0.0:1780298645.558895:0:17567:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242863800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+43:29360:x1866777609699328:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.558903:0:17567:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.558911:0:17567:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2200/0xaf35132e63035f25 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 18 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4d78 expref: -99 pid: 17567 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.558962:0:17567:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3a00/0xaf35132e63035f2c lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 19 type: IBT flags: 0x40000000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.558969:0:17567:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3a00/0xaf35132e63035f2c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 19 type: IBT flags: 0x50210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.558977:0:17567:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3a00/0xaf35132e63035f2c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 19 type: IBT flags: 0x40210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1780298645.559003:0:17567:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 6407779 00000004:00200000:0.0:1780298645.559007:0:17567:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1780298645.559021:0:17567:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3a00/0xaf35132e63035f2c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 19 type: IBT flags: 0x40210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.559041:0:17567:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2200/0xaf35132e63035f25 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 19 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d78 expref: 45 pid: 17567 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.559055:0:17567:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3a00/0xaf35132e63035f2c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 18 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d78 expref: 44 pid: 17567 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.559063:0:17567:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298645.559066:0:17567:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1780298645.559068:0:17567:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3a00, rc 0) 00000100:00100000:0.0:1780298645.559100:0:17567:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242863800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+44:29360:x1866777609699328:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 204us (274us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.559107:0:17567:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62082 00000100:00100000:1.0:1780298645.560170:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.560279:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609699456 00000100:00100000:3.0:1780298645.560285:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609699456 00000100:00100000:1.0:1780298645.560326:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.560353:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609699584 00000100:00100000:3.0:1780298645.560356:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609699584 00000100:00100000:0.0:1780298645.560370:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10357 00000100:00100000:0.0:1780298645.560381:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242861f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609699456:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.560391:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298645.560395:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10358 00010000:00010000:0.0:1780298645.560403:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e63035f33 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4d7f expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.560404:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242861180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609699584:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.560413:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.560422:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027d3a930/0xaf35132e63035f3a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4d86 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.560454:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244adad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605094912:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.560455:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605094784:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.560514:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2418563c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.560515:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4f00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.561360:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244adad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605094912:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.561392:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000027d3a930/0xaf35132e63035f3a lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d86 expref: 8 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.561392:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605094784:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.561416:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027d3a930/0xaf35132e63035f3a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d86 expref: 8 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.561421:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e63035f33 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d7f expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.561426:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341692200, rc 0) 00010000:00010000:0.0:1780298645.561443:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e63035f33 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d7f expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.561454:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3c00, rc 0) 00000100:00100000:3.0:1780298645.561462:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242861180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609699584:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1059us (1135us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.561470:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10358 00000100:00100000:0.0:1780298645.561482:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242861f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609699456:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1102us (1311us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.561490:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10357 00000100:00100000:1.0:1780298645.562636:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.562675:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609699712 00000100:00100000:0.0:1780298645.562682:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609699712 00000100:00100000:1.0:1780298645.562703:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.562729:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609699840 00000100:00100000:0.0:1780298645.562733:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609699840 00000100:00100000:0.0:1780298645.562765:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10359 00000100:00100000:2.0:1780298645.562776:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10360 00000100:00100000:0.0:1780298645.562777:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242862d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609699712:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.562790:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298645.562795:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243845180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609699840:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.562802:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008361787d/0xaf35132e63035f41 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4d8d expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.562806:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.562817:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e63035f48 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4d94 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.562843:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605095040:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.562854:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605095168:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.562899:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.562899:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4f00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.563611:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605095168:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.563614:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605095040:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.563629:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e63035f48 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d94 expref: 8 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.563636:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008361787d/0xaf35132e63035f41 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d8d expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.563645:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e63035f48 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d94 expref: 8 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.563674:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b000, rc 0) 00010000:00010000:0.0:1780298645.563677:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008361787d/0xaf35132e63035f41 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d8d expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.563690:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2800, rc 0) 00000100:00100000:2.0:1780298645.563702:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243845180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609699840:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 916us (1000us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.563707:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10360 00000100:00100000:0.0:1780298645.563716:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242862d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609699712:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 942us (1081us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.563724:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10359 00000100:00100000:1.0:1780298645.564712:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.564749:0:9976:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609700096 00000100:00100000:0.0:1780298645.564755:0:9976:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609700096 00000100:00100000:0.0:1780298645.564793:0:9976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62083 00000100:00100000:0.0:1780298645.564801:0:9976:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243846a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+44:29360:x1866777609700096:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:0.0:1780298645.564841:0:9976:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298645.564857:0:9976:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3a00/0xaf35132e63035f2c lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 19 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d78 expref: 44 pid: 17567 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.564871:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298645.564875:0:9976:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1780298645.565013:0:9976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2c00/0xaf35132e63035f4f lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 18 type: IBT flags: 0x40000000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.565027:0:9976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2c00/0xaf35132e63035f4f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 18 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.565036:0:9976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2c00/0xaf35132e63035f4f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 18 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1780298645.565079:0:9976:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991124 00000004:00010000:0.0:1780298645.565098:0:9976:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 000000009ed1caa7 reply state 00000000f3a7918e transno 4294991125 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2c00/0xaf35132e63035f4f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 18 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.565109:0:9976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2c00/0xaf35132e63035f4f lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 18 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.565116:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298645.565118:0:9976:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1780298645.565121:0:9976:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1780298645.565197:0:9976:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243846a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+46:29360:x1866777609700096:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 397us (485us total) trans 4294991125 rc 0/0 00000100:00100000:0.0:1780298645.565205:0:9976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62083 00000020:00080000:0.0:1780298645.565210:0:9976:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1780298645.565891:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.565924:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609700224 00000100:00100000:2.0:1780298645.565929:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609700224 00000100:00100000:2.0:1780298645.565954:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10361 00000100:00100000:2.0:1780298645.565960:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f734e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:29360:x1866777609700224:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.565966:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.565974:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b4561703/0xaf35132e63035f56 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4d9b expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.565989:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63035ed8 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d40 expref: 10 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.565996:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(53 169) 00010000:00010000:2.0:1780298645.565999:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(54 169) 00010000:00010000:2.0:1780298645.566002:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b4561703/0xaf35132e63035f56 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d9b expref: 7 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.566019:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63035ed8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d40 expref: 10 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.566028:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63035ed8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d40 expref: 10 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.566037:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605095296:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298645.566075:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.566768:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605095296:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298645.566781:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b4561703/0xaf35132e63035f56 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d9b expref: 7 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.566790:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b600, rc 0) 00000100:00100000:2.0:1780298645.566813:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f734e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609700224:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 853us (923us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.566820:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10361 00000100:00100000:1.0:1780298645.567012:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.567056:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609700352 00000100:00100000:2.0:1780298645.567061:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609700352 00010000:00010000:2.0:1780298645.567080:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63035ed8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d40 expref: 11 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.567109:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15296 00000100:00100000:2.0:1780298645.567117:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f736a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609700352:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298645.567146:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298645.567160:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63035ed8 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d40 expref: 11 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.567172:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63035ed8 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d40 expref: 11 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.567195:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63035ed8 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d40 expref: 11 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.567206:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(54 169) 00010000:00010000:2.0:1780298645.567209:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(54 169) 00010000:00010000:2.0:1780298645.567215:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b4561703/0xaf35132e63035f56 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d9b expref: 6 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.567224:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b4561703/0xaf35132e63035f56 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d9b expref: 6 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.567246:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000b4561703/0xaf35132e63035f56 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d9b expref: 6 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.567257:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777605095424:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298645.567306:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.567981:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777605095424:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298645.567991:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298645.568030:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f736a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609700352:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 914us (1017us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.568038:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15296 00000100:00100000:1.0:1780298645.568221:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.568254:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609700480 00000100:00100000:0.0:1780298645.568259:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609700480 00000100:00100000:0.0:1780298645.568298:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10362 00000100:00100000:0.0:1780298645.568307:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f737800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609700480:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.568316:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.568327:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000488a7b88/0xaf35132e63035f5d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4da2 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.568344:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e63035edf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d47 expref: 13 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.568352:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(51 122) 00010000:00010000:0.0:1780298645.568355:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(52 122) 00010000:00010000:0.0:1780298645.568360:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000488a7b88/0xaf35132e63035f5d lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4da2 expref: 8 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.568378:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e63035edf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d47 expref: 13 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.568389:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e63035edf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d47 expref: 13 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.568398:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605095552:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298645.568434:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4f00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.569191:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605095552:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298645.569207:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000488a7b88/0xaf35132e63035f5d lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4da2 expref: 8 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.569218:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2200, rc 0) 00000100:00100000:0.0:1780298645.569316:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f737800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:29360:x1866777609700480:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1010us (1092us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.569323:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10362 00000100:00100000:3.0:1780298645.569359:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.569404:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609700608 00000100:00100000:2.0:1780298645.569410:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609700608 00010000:00010000:2.0:1780298645.569427:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e63035edf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d47 expref: 14 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.569488:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15297 00000100:00100000:0.0:1780298645.569497:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609700608:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298645.569507:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298645.569521:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e63035edf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d47 expref: 14 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.569537:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e63035edf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d47 expref: 14 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.569552:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e63035edf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d47 expref: 14 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.569563:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(52 122) 00010000:00010000:0.0:1780298645.569567:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(52 122) 00010000:00010000:0.0:1780298645.569573:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000488a7b88/0xaf35132e63035f5d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4da2 expref: 7 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.569581:0:6463:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000488a7b88/0xaf35132e63035f5d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4da2 expref: 7 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.569600:0:6463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000488a7b88/0xaf35132e63035f5d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4da2 expref: 7 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.569610:0:6463:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777605095680:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298645.569646:0:6463:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4f00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.570318:0:6463:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777605095680:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298645.570328:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298645.570352:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609700608:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 857us (995us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.570362:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15297 00000100:00100000:3.0:1780298645.570535:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.570561:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609700736 00000100:00100000:1.0:1780298645.570567:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609700736 00000100:00100000:3.0:1780298645.570572:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00002000:00010000:1.0:1780298645.570593:0:19928:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:1.0:1780298645.570599:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244ad9f80 with x1866777609700736 ext(4194304->18446744073709551615) 00000100:00100000:0.0:1780298645.570604:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609700864 00002000:00010000:1.0:1780298645.570607:0:19928:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf244ad9f80. 00000100:00100000:0.0:1780298645.570610:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609700864 00002000:00010000:0.0:1780298645.570625:0:19950:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (2426193->18446744073709551615) 00002000:00100000:0.0:1780298645.570629:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244ad9500 with x1866777609700864 ext(2426193->18446744073709551615) 00000100:00100000:1.0:1780298645.570634:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6480 00002000:00010000:0.0:1780298645.570635:0:19950:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf244ad9500. 00000100:00100000:1.0:1780298645.570648:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609700736:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:2.0:1780298645.570666:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6481 00000100:00100000:2.0:1780298645.570677:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609700864:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:1.0:1780298645.570767:0:19928:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:1.0:1780298645.570771:0:19928:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:1.0:1780298645.570774:0:19928:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298645.570777:0:19928:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:1.0:1780298645.570779:0:19928:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 392 -> 392 00002000:00010000:2.0:1780298645.570803:0:8362:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 2213475 -> 2426193 00002000:00010000:2.0:1780298645.570807:0:8362:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00000100:00100000:1.0:1780298645.570815:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609700736:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 168us (278us total) trans 4294969784 rc 0/0 00002000:00010000:2.0:1780298645.570819:0:8362:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:2.0:1780298645.570822:0:8362:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00000100:00100000:1.0:1780298645.570826:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6480 00002000:00010000:2.0:1780298645.570827:0:8362:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 24 -> 24 00002000:00010000:1.0:1780298645.570832:0:19928:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:1.0:1780298645.570836:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244ad9f80 with x1866777609700736 ext(4194304->18446744073709551615) 00002000:00010000:1.0:1780298645.570841:0:19928:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf244ad9f80. 00000100:00100000:2.0:1780298645.570858:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609700864:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 182us (284us total) trans 4294969323 rc 0/0 00000100:00100000:2.0:1780298645.570866:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6481 00002000:00010000:2.0:1780298645.570869:0:8362:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (2426193->18446744073709551615) 00002000:00100000:2.0:1780298645.570872:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244ad9500 with x1866777609700864 ext(2426193->18446744073709551615) 00002000:00010000:2.0:1780298645.570877:0:8362:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf244ad9500. 00000100:00100000:1.0:1780298645.571642:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.571670:0:17578:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609700992 00000100:00100000:3.0:1780298645.571675:0:17578:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609700992 00000100:00100000:3.0:1780298645.571704:0:17578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62084 00000100:00100000:3.0:1780298645.571710:0:17578:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+46:29360:x1866777609700992:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.571722:0:17578:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.571732:0:17578:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692e00/0xaf35132e63035f64 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 19 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4da9 expref: -99 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.571786:0:17578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692200/0xaf35132e63035f6b lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 20 type: IBT flags: 0x40000000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.571796:0:17578:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692200/0xaf35132e63035f6b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 20 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.571803:0:17578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692200/0xaf35132e63035f6b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 20 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1780298645.571822:0:17578:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 6620497 00000004:00200000:3.0:1780298645.571825:0:17578:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1780298645.571836:0:17578:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692200/0xaf35132e63035f6b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 20 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.571852:0:17578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692e00/0xaf35132e63035f64 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 20 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4da9 expref: 48 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.571860:0:17578:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692200/0xaf35132e63035f6b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 19 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4da9 expref: 47 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.571865:0:17578:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298645.571867:0:17578:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1780298645.571869:0:17578:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341692200, rc 0) 00000100:00100000:3.0:1780298645.571891:0:17578:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+47:29360:x1866777609700992:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 182us (252us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298645.571895:0:17578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62084 00000100:00100000:3.0:1780298645.573209:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.573237:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609701120 00000100:00100000:2.0:1780298645.573241:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609701120 00000100:00100000:2.0:1780298645.573285:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10363 00000100:00100000:2.0:1780298645.573291:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:29360:x1866777609701120:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.573298:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.573305:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e63035f72 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4db0 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.573319:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000488a7b88/0xaf35132e63035f5d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4da2 expref: 8 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.573328:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(52 122) 00010000:00010000:2.0:1780298645.573331:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(53 122) 00010000:00010000:2.0:1780298645.573334:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e63035f72 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2342912->2355199] (req 2342912->2355199) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4db0 expref: 14 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.573350:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000488a7b88/0xaf35132e63035f5d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4da2 expref: 8 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.573359:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000488a7b88/0xaf35132e63035f5d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4da2 expref: 8 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.573367:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605095808:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298645.573407:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.574074:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605095808:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298645.574092:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e63035f72 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2342912->2355199] (req 2342912->2355199) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4db0 expref: 14 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.574106:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b000, rc 0) 00000100:00100000:2.0:1780298645.574155:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:29360:x1866777609701120:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 863us (945us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.574163:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10363 00000100:00100000:1.0:1780298645.574333:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.574376:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609701248 00000100:00100000:0.0:1780298645.574380:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609701248 00010000:00010000:0.0:1780298645.574395:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000488a7b88/0xaf35132e63035f5d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4da2 expref: 9 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.574418:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15298 00000100:00100000:0.0:1780298645.574426:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609701248:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298645.574435:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298645.574448:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000488a7b88/0xaf35132e63035f5d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4da2 expref: 9 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.574459:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000488a7b88/0xaf35132e63035f5d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4da2 expref: 9 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.574475:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000488a7b88/0xaf35132e63035f5d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4da2 expref: 9 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.574485:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(53 122) 00010000:00010000:0.0:1780298645.574489:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(53 122) 00010000:00010000:0.0:1780298645.574496:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e63035f72 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2342912->2355199) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4db0 expref: 13 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.574505:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e63035f72 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2342912->2355199) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4db0 expref: 13 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.574524:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e63035f72 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2342912->2355199) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4db0 expref: 13 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.574533:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:7400:1866777605095936:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298645.574568:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4f00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.575299:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:7400:1866777605095936:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298645.575316:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298645.575354:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609701248:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 928us (1021us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.575372:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15298 00000100:00100000:3.0:1780298645.575775:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.575815:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609701376 00000100:00100000:0.0:1780298645.575821:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609701376 00002000:00100000:0.0:1780298645.575851:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x12:0x0] (2342912->2355199) req@ffff9cf244ad9180 x1866777609701376/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298645.575862:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244ad9180 with x1866777609701376 ext(2342912->2355199) 00002000:00010000:0.0:1780298645.575867:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf244ad9180 00000100:00100000:0.0:1780298645.575887:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6482 00000100:00100000:0.0:1780298645.575897:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609701376:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298645.576064:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609701376:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 169us (289us total) trans 0 rc 12288/12288 00000100:00100000:0.0:1780298645.576072:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6482 00002000:00100000:0.0:1780298645.576079:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x12:0x0] (2342912->2355199) req@ffff9cf244ad9180 x1866777609701376/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/12728 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 12288/12288 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298645.576088:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244ad9180 with x1866777609701376 ext(2342912->2355199) 00002000:00010000:0.0:1780298645.576092:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf244ad9180 00000100:00100000:3.0:1780298645.577049:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.577091:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609701504 00000100:00100000:0.0:1780298645.577096:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609701504 00000100:00100000:0.0:1780298645.577153:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62085 00000100:00100000:0.0:1780298645.577162:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+38:29360:x1866777609701504:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.577176:0:6477:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.577190:0:6477:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3a00/0xaf35132e63035f79 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 20 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4db7 expref: -99 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.577260:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2800/0xaf35132e63035f80 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 21 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.577284:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2800/0xaf35132e63035f80 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 21 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.577295:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2800/0xaf35132e63035f80 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 21 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1780298645.577317:0:6477:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 6620497 00000004:00200000:0.0:1780298645.577320:0:6477:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1780298645.577332:0:6477:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2800/0xaf35132e63035f80 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 21 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.577352:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3a00/0xaf35132e63035f79 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 21 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4db7 expref: 40 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.577365:0:6477:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2800/0xaf35132e63035f80 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 20 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4db7 expref: 39 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.577373:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298645.577376:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1780298645.577379:0:6477:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2800, rc 0) 00000100:00100000:0.0:1780298645.577408:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+39:29360:x1866777609701504:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 248us (361us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.577414:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62085 00000100:00100000:3.0:1780298645.578177:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.578208:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609701632 00000100:00100000:0.0:1780298645.578213:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609701632 00000100:00100000:0.0:1780298645.578244:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10364 00000100:00100000:0.0:1780298645.578253:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609701632:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.578262:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.578284:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e63035f87 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4dbe expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.578326:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605096064:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.578390:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4240 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.578983:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605096064:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.579001:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e63035f87 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4dbe expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.579017:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e63035f87 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4dbe expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.579027:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3c00, rc 0) 00000100:00100000:0.0:1780298645.579058:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609701632:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 806us (880us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.579065:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10364 00000100:00100000:3.0:1780298645.579968:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.580016:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609701760 00000100:00100000:2.0:1780298645.580021:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609701760 00000100:00100000:2.0:1780298645.580060:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10365 00000100:00100000:2.0:1780298645.580069:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e424050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609701760:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.580079:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.580088:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63035f8e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4dc5 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.580118:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605096192:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.580201:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.580825:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605096192:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.580836:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63035f8e lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4dc5 expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.580846:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63035f8e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4dc5 expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.580852:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9be00, rc 0) 00000100:00100000:2.0:1780298645.580873:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e424050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609701760:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 808us (909us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.580878:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10365 00000100:00100000:0.0:1780298645.581946:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.581974:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609701888 00000100:00100000:3.0:1780298645.581979:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609701888 00000100:00100000:3.0:1780298645.582004:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10366 00000100:00100000:3.0:1780298645.582010:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:29360:x1866777609701888:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.582018:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.582027:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d1b192d6/0xaf35132e63035f95 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4dcc expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.582042:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e63035f72 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2342912->2355199) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4db0 expref: 13 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.582049:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(53 122) 00010000:00010000:3.0:1780298645.582052:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(53 122) 00010000:00010000:3.0:1780298645.582055:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d1b192d6/0xaf35132e63035f95 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3170304->3182591] (req 3170304->3182591) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4dcc expref: 9 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.582070:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e63035f72 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2342912->2355199) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4db0 expref: 13 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.582080:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e63035f72 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2342912->2355199) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4db0 expref: 13 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.582086:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605096320:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298645.582141:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856000 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.582731:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605096320:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298645.582760:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d1b192d6/0xaf35132e63035f95 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3170304->3182591] (req 3170304->3182591) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4dcc expref: 9 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.582770:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341692400, rc 0) 00000100:00100000:0.0:1780298645.582795:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:29360:x1866777609701888:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 786us (852us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.582800:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10366 00000100:00100000:1.0:1780298645.583002:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.583034:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609702016 00000100:00100000:0.0:1780298645.583038:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609702016 00010000:00010000:0.0:1780298645.583052:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e63035f72 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2342912->2355199) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4db0 expref: 14 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.583075:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15299 00000100:00100000:0.0:1780298645.583082:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609702016:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298645.583093:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298645.583105:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e63035f72 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2342912->2355199) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4db0 expref: 14 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.583150:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e63035f72 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2342912->2355199) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4db0 expref: 14 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.583169:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e63035f72 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2342912->2355199) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4db0 expref: 14 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.583179:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(53 122) 00010000:00010000:0.0:1780298645.583183:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(53 122) 00010000:00010000:0.0:1780298645.583190:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000d1b192d6/0xaf35132e63035f95 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3170304->36724735] (req 3170304->3182591) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4dcc expref: 8 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.583199:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d1b192d6/0xaf35132e63035f95 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3170304->36724735] (req 3170304->3182591) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4dcc expref: 8 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.583222:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000d1b192d6/0xaf35132e63035f95 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3170304->36724735] (req 3170304->3182591) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4dcc expref: 8 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.583231:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d8000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:20253:1866777605096448:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298645.583260:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4240 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.583925:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d8000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:20253:1866777605096448:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298645.583936:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298645.583964:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609702016:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 882us (963us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.583971:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15299 00000100:00100000:3.0:1780298645.584572:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.584606:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609702144 00000100:00100000:2.0:1780298645.584611:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609702144 00000100:00100000:2.0:1780298645.584635:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10367 00000100:00100000:2.0:1780298645.584641:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609702144:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.584649:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.584658:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e63035f9c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4dd3 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.584686:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605096576:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.584736:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.584756:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.584788:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609702272 00000100:00100000:2.0:1780298645.584792:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609702272 00000100:00100000:2.0:1780298645.584818:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10368 00000100:00100000:2.0:1780298645.584823:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609702272:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.584828:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.584834:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0a0f245/0xaf35132e63035fa3 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4dda expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.584853:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255090a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605096704:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.584875:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.585424:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605096576:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.585441:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e63035f9c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4dd3 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.585455:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e63035f9c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4dd3 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.585464:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9a600, rc 0) 00000100:00100000:2.0:1780298645.585512:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609702144:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 869us (938us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.585523:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10367 00002000:00010000:2.0:1780298645.585649:0:20252:0:(ofd_lvb.c:259:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size: 2426193 -> 3178729 00000100:00100000:2.0:1780298645.585658:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255090a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605096704:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.585669:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e0a0f245/0xaf35132e63035fa3 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4dda expref: 14 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.585679:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0a0f245/0xaf35132e63035fa3 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4dda expref: 14 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.585685:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b200, rc 0) 00000100:00100000:2.0:1780298645.585769:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609702272:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 944us (1013us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.585776:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10368 00000100:00100000:0.0:1780298645.586570:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.586617:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609702400 00000100:00100000:2.0:1780298645.586622:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609702400 00000100:00100000:2.0:1780298645.586655:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10369 00000100:00100000:2.0:1780298645.586665:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5db100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609702400:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.586675:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.586683:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d71a6aba/0xaf35132e63035faa lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4de1 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.586699:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.586710:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255090a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605096832:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.586727:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609702528 00000100:00100000:0.0:1780298645.586733:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609702528 00000100:00100000:2.0:1780298645.586755:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.586766:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10370 00000100:00100000:0.0:1780298645.586775:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609702528:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.586783:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.586792:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e63035fb1 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4de8 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.586820:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605096960:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.586851:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4240 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.587372:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255090a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605096832:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.587384:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d71a6aba/0xaf35132e63035faa lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4de1 expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.587398:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d71a6aba/0xaf35132e63035faa lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4de1 expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.587405:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9bc00, rc 0) 00000100:00100000:2.0:1780298645.587433:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5db100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609702400:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 770us (863us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.587447:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10369 00000100:00100000:0.0:1780298645.587543:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605096960:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.587560:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e63035fb1 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4de8 expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.587573:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e63035fb1 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4de8 expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.587585:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3800, rc 0) 00000100:00100000:0.0:1780298645.587611:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609702528:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 838us (942us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.587618:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10370 00000100:00100000:3.0:1780298645.588432:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.588508:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.588532:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609702656 00000100:00100000:1.0:1780298645.588538:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609702656 00000100:00100000:2.0:1780298645.588541:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609702784 00000100:00100000:2.0:1780298645.588546:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609702784 00000100:00100000:1.0:1780298645.588573:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10371 00000100:00100000:1.0:1780298645.588584:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609702656:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.588594:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298645.588595:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10372 00010000:00010000:1.0:1780298645.588605:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e63035fb8 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4def expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.588607:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ada300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609702784:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.588625:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.588634:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0cae770/0xaf35132e63035fbf lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4df6 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.588651:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241a82a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605097088:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.588663:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605097216:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.588699:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4240 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.588700:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e900 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.589405:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241a82a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605097088:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.589415:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605097216:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.589422:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e63035fb8 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4def expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.589431:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e0cae770/0xaf35132e63035fbf lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4df6 expref: 14 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.589442:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e63035fb8 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4def expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.589448:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0cae770/0xaf35132e63035fbf lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4df6 expref: 14 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.589451:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279800, rc 0) 00010000:00010000:0.0:1780298645.589462:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2600, rc 0) 00000100:00100000:1.0:1780298645.589482:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609702656:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 902us (1050us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.589490:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ada300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609702784:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 884us (978us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.589493:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10371 00000100:00100000:0.0:1780298645.589504:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10372 00000100:00100000:3.0:1780298645.590361:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.590389:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609702912 00000100:00100000:2.0:1780298645.590393:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609702912 00000100:00100000:2.0:1780298645.590419:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10373 00000100:00100000:2.0:1780298645.590426:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609702912:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.590434:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.590442:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63035fc6 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4dfd expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.590457:0:19992:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000b4561703/0xaf35132e63035f56 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d9b expref: 7 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.590463:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(54 169) 00010000:00010000:2.0:1780298645.590465:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(55 169) 00010000:00010000:2.0:1780298645.590468:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63035fc6 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3702784->3710975] (req 3702784->3710975) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4dfd expref: 11 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.590490:0:19992:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000b4561703/0xaf35132e63035f56 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d9b expref: 7 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.590504:0:19992:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000b4561703/0xaf35132e63035f56 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d9b expref: 7 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.590515:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255090a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605097344:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298645.590555:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.591207:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255090a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605097344:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298645.591222:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63035fc6 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3702784->3710975] (req 3702784->3710975) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4dfd expref: 11 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.591233:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9be00, rc 0) 00000100:00100000:2.0:1780298645.591262:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:29360:x1866777609702912:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 836us (902us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.591279:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10373 00000100:00100000:3.0:1780298645.591447:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.591483:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609703040 00000100:00100000:2.0:1780298645.591487:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609703040 00010000:00010000:2.0:1780298645.591505:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000b4561703/0xaf35132e63035f56 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d9b expref: 8 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.591529:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15300 00000100:00100000:2.0:1780298645.591535:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b1f850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609703040:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298645.591546:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298645.591556:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000b4561703/0xaf35132e63035f56 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d9b expref: 8 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.591566:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000b4561703/0xaf35132e63035f56 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d9b expref: 8 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.591580:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b4561703/0xaf35132e63035f56 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4d9b expref: 8 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.591590:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(55 169) 00010000:00010000:2.0:1780298645.591594:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(55 169) 00010000:00010000:2.0:1780298645.591601:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63035fc6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3702784->37257215] (req 3702784->3710975) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4dfd expref: 10 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.591610:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63035fc6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3702784->37257215] (req 3702784->3710975) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4dfd expref: 10 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.591640:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63035fc6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [3702784->37257215] (req 3702784->3710975) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4dfd expref: 10 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.591649:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255090a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777605097472:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298645.591687:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.592332:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255090a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777605097472:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298645.592341:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298645.592368:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b1f850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609703040:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 833us (957us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.592376:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15300 00000100:00100000:3.0:1780298645.592545:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.592576:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609703168 00000100:00100000:1.0:1780298645.592580:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609703168 00000100:00100000:1.0:1780298645.592609:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6483 00000100:00100000:1.0:1780298645.592614:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609703168:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298645.592759:0:19951:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:1.0:1780298645.592762:0:19951:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:1.0:1780298645.592764:0:19951:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:1.0:1780298645.592765:0:19951:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:1.0:1780298645.592767:0:19951:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 392 -> 408 00000100:00100000:1.0:1780298645.592791:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2399:x1866777609703168:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 176us (275us total) trans 4294969785 rc 0/0 00000100:00100000:1.0:1780298645.592798:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6483 00000100:00100000:0.0:1780298645.593633:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.593663:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609703296 00000100:00100000:3.0:1780298645.593668:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609703296 00000100:00100000:0.0:1780298645.593686:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.593738:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609703424 00000100:00100000:3.0:1780298645.593743:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609703424 00000100:00100000:1.0:1780298645.593748:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10374 00000100:00100000:1.0:1780298645.593759:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609703296:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298645.593765:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10375 00010000:00010000:1.0:1780298645.593770:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298645.593779:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5dad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609703424:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.593780:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e63035fcd lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4e04 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.593793:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.593803:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0a0f245/0xaf35132e63035fd4 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4e0b expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.593812:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241a82a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605097600:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.593834:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255090a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605097728:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.593855:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e900 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.593863:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00002000:00010000:1.0:1780298645.594550:0:8359:0:(ofd_lvb.c:271:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime: 1780298645 -> 1780298646 00000100:00100000:1.0:1780298645.594560:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241a82a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605097600:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.594577:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e63035fcd lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e04 expref: 13 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.594593:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e63035fcd lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e04 expref: 13 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.594605:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279e00, rc 0) 00000100:00100000:2.0:1780298645.594618:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255090a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605097728:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.594635:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e0a0f245/0xaf35132e63035fd4 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e0b expref: 14 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.594640:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609703296:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 882us (1005us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.594654:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10374 00010000:00010000:2.0:1780298645.594655:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0a0f245/0xaf35132e63035fd4 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e0b expref: 14 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.594664:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b200, rc 0) 00000100:00100000:2.0:1780298645.594686:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5dad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609703424:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 912us (1002us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.594693:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10375 00000100:00100000:3.0:1780298645.595578:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.595656:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609703552 00000100:00100000:0.0:1780298645.595660:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609703552 00000100:00100000:3.0:1780298645.595682:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.595703:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609703680 00000100:00100000:0.0:1780298645.595706:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609703680 00000100:00100000:1.0:1780298645.595720:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10376 00000100:00100000:1.0:1780298645.595731:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609703552:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.595740:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298645.595740:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10377 00010000:00010000:1.0:1780298645.595750:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63035fdb lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4e12 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.595752:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609703680:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.595768:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.595778:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000101214ae/0xaf35132e63035fe2 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4e19 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.595804:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241a82a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605097856:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.595808:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b40380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605097984:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.595852:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4240 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.595854:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e900 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.596645:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241a82a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605097856:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.596666:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63035fdb lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e12 expref: 13 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.596670:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b40380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605097984:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.596689:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63035fdb lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e12 expref: 13 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.596694:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000101214ae/0xaf35132e63035fe2 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e19 expref: 14 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.596703:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278200, rc 0) 00010000:00010000:2.0:1780298645.596722:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000101214ae/0xaf35132e63035fe2 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e19 expref: 14 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.596734:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3200, rc 0) 00000100:00100000:1.0:1780298645.596739:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609703552:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1009us (1160us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.596748:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10376 00000100:00100000:2.0:1780298645.596763:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609703680:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1013us (1080us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.596771:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10377 00000100:00100000:3.0:1780298645.597615:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.597654:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609703808 00000100:00100000:0.0:1780298645.597658:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609703808 00000100:00100000:0.0:1780298645.597680:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10378 00000100:00100000:0.0:1780298645.597684:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:29360:x1866777609703808:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.597691:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.597698:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0cae770/0xaf35132e63035fe9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4e20 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.597710:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(53 122) 00010000:00010000:0.0:1780298645.597712:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(53 122) 00010000:00010000:0.0:1780298645.597718:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0cae770/0xaf35132e63035fe9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2539520->3170303] (req 2539520->2543615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e20 expref: 10 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.597727:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e0cae770/0xaf35132e63035fe9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2539520->3170303] (req 2539520->2543615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e20 expref: 10 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.597733:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2600, rc 0) 00000100:00100000:0.0:1780298645.597753:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:29360:x1866777609703808:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 70us (142us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.597757:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10378 00000100:00100000:3.0:1780298645.598422:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.598457:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609703936 00000100:00100000:0.0:1780298645.598460:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609703936 00000100:00100000:0.0:1780298645.598487:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6484 00000100:00100000:0.0:1780298645.598491:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609703936:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298645.598648:0:8364:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 3178729 -> 2542643 00002000:00010000:0.0:1780298645.598651:0:8364:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298645.598653:0:8364:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298645 -> 1780298645 00002000:00010000:0.0:1780298645.598654:0:8364:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298645.598655:0:8364:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 24 -> 32 00000100:00100000:0.0:1780298645.598677:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609703936:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 186us (257us total) trans 4294969324 rc 0/0 00000100:00100000:0.0:1780298645.598681:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6484 00000100:00100000:3.0:1780298645.599516:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.599554:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609704064 00000100:00100000:2.0:1780298645.599559:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609704064 00000100:00100000:2.0:1780298645.599593:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10379 00000100:00100000:3.0:1780298645.599598:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.599603:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609704064:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.599610:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.599618:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d71a6aba/0xaf35132e63035ff0 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4e27 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.599628:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609704192 00000100:00100000:0.0:1780298645.599633:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609704192 00000100:00100000:2.0:1780298645.599648:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255090a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605098112:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.599666:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10380 00000100:00100000:0.0:1780298645.599673:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609704192:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.599680:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.599687:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e63035ff7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4e2e expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.599691:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.599713:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b42300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605098240:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.599735:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4300 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.600354:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255090a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605098112:192.168.203.25@tcp:106: 00002000:00010000:0.0:1780298645.600559:0:12101:0:(ofd_lvb.c:259:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size: 2542643 -> 3178729 00000100:00100000:0.0:1780298645.600567:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b42300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605098240:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.600585:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e63035ff7 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e2e expref: 12 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.600605:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e63035ff7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e2e expref: 12 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.600614:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3800, rc 0) 00000100:00100000:0.0:1780298645.600642:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609704192:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 969us (1049us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.600648:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10380 00010000:00010000:2.0:1780298645.600991:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d71a6aba/0xaf35132e63035ff0 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e27 expref: 8 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.601001:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d71a6aba/0xaf35132e63035ff0 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e27 expref: 8 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.601012:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9bc00, rc 0) 00000100:00100000:2.0:1780298645.601034:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609704064:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1434us (1520us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.601038:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10379 00000100:00100000:3.0:1780298645.601965:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.601983:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609704320 00000100:00100000:2.0:1780298645.601988:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609704320 00000100:00100000:2.0:1780298645.602017:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10381 00000100:00100000:3.0:1780298645.602023:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.602031:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609704320:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.602039:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298645.602040:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609704448 00000100:00100000:1.0:1780298645.602044:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609704448 00010000:00010000:2.0:1780298645.602048:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b4561703/0xaf35132e63035ffe lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4e35 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.602076:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10382 00000100:00100000:1.0:1780298645.602085:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f757100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609704448:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298645.602088:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255090a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605098368:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.602093:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.602101:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e63036005 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4e3c expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.602152:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.602153:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241a82a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605098496:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.602184:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e900 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.602740:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255090a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605098368:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.602752:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b4561703/0xaf35132e63035ffe lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e35 expref: 8 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.602762:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b4561703/0xaf35132e63035ffe lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e35 expref: 8 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.602769:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b600, rc 0) 00000100:00100000:2.0:1780298645.602800:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609704320:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 774us (837us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.602806:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10381 00002000:00010000:2.0:1780298645.602883:0:8359:0:(ofd_lvb.c:271:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime: 1780298645 -> 1780298646 00000100:00100000:2.0:1780298645.602890:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241a82a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605098496:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.602901:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e63036005 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e3c expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.602910:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e63036005 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e3c expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.602917:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279e00, rc 0) 00000100:00100000:2.0:1780298645.602937:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f757100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609704448:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 855us (921us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.602941:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10382 00000100:00100000:3.0:1780298645.604054:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.604151:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609704576 00000100:00100000:2.0:1780298645.604157:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609704576 00002000:00100000:2.0:1780298645.604186:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (3829760->3837951) req@ffff9cf37f757b80 x1866777609704576/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.604196:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f757b80 with x1866777609704576 ext(3829760->3837951) 00002000:00010000:2.0:1780298645.604202:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f757b80 00000100:00100000:2.0:1780298645.604225:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6485 00000100:00100000:1.0:1780298645.604232:0:3623:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1780298645.604234:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f757b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609704576:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298645.604248:0:3623:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3623:1866777605098624:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1780298645.604296:0:3623:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1780298645.604330:0:9133:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777605098624 00000100:00100000:3.0:1780298645.604336:0:9133:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777605098624 00000100:00100000:3.0:1780298645.604370:0:9133:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 590 00000100:00100000:3.0:1780298645.604381:0:9133:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a82300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3623:x1866777605098624:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1780298645.604396:0:9133:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 106496 granted 969661248 pending 0 free 3923107840 avail 3691208704 00000100:00100000:2.0:1780298645.604417:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f757b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609704576:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 184us (364us total) trans 0 rc 8192/8192 00000100:00100000:2.0:1780298645.604425:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6485 00002000:00100000:2.0:1780298645.604433:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (3829760->3837951) req@ffff9cf37f757b80 x1866777609704576/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/8632 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 8192/8192 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.604443:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f757b80 with x1866777609704576 ext(3829760->3837951) 00000100:00100000:3.0:1780298645.604445:0:9133:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a82300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3623:x1866777605098624:12345-0@lo:13:osp-pre-0-1.0 Request processed in 65us (146us total) trans 0 rc 0/0 00002000:00010000:2.0:1780298645.604449:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf37f757b80 00000100:00100000:3.0:1780298645.604458:0:9133:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 590 00000100:00100000:0.0:1780298645.604481:0:3623:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3623:1866777605098624:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1780298645.605464:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.605495:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.605514:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609704704 00000100:00100000:1.0:1780298645.605519:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609704704 00000100:00100000:0.0:1780298645.605534:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609704832 00000100:00100000:0.0:1780298645.605538:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609704832 00000100:00100000:1.0:1780298645.605550:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10383 00000100:00100000:1.0:1780298645.605557:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f754a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609704704:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.605564:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.605575:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e6303600c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4e43 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.605575:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10384 00000100:00100000:0.0:1780298645.605585:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f754000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609704832:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.605594:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.605602:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d4afe0b/0xaf35132e63036013 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4e4a expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.605625:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241a82d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605098752:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.605631:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b42300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605098880:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.605675:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e480 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.605681:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4300 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.606414:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241a82d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605098752:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.606420:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b42300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605098880:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.606443:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e6303600c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e43 expref: 13 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.606445:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001d4afe0b/0xaf35132e63036013 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e4a expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.606472:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d4afe0b/0xaf35132e63036013 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e4a expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.606476:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e6303600c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e43 expref: 13 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.606483:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3000, rc 0) 00010000:00010000:3.0:1780298645.606497:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278200, rc 0) 00000100:00100000:0.0:1780298645.606509:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f754000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609704832:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 927us (1015us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.606515:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10384 00000100:00100000:3.0:1780298645.606525:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f754a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609704704:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 966us (1061us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.606533:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10383 00000100:00100000:3.0:1780298645.607361:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.607390:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.607406:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609704960 00000100:00100000:1.0:1780298645.607410:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609704960 00000100:00100000:2.0:1780298645.607428:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609705088 00000100:00100000:2.0:1780298645.607434:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609705088 00000100:00100000:1.0:1780298645.607440:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10385 00000100:00100000:1.0:1780298645.607448:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348004e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609704960:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.607455:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.607463:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e6303601a lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4e51 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.607471:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10386 00000100:00100000:2.0:1780298645.607484:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348006680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609705088:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298645.607486:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241a82d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605099008:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.607494:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.607504:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0a0f245/0xaf35132e63036021 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4e58 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.607529:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255090a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605099136:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.607537:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e840 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.607556:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c900 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.608267:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241a82d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605099008:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.608299:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e6303601a lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e51 expref: 13 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.608315:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e6303601a lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e51 expref: 13 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.608326:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279c00, rc 0) 00000100:00100000:2.0:1780298645.608341:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255090a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605099136:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.608354:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e0a0f245/0xaf35132e63036021 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e58 expref: 14 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.608364:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0a0f245/0xaf35132e63036021 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e58 expref: 14 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.608370:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b200, rc 0) 00000100:00100000:2.0:1780298645.608390:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348006680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609705088:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 912us (1002us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.608399:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10386 00000100:00100000:3.0:1780298645.608400:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348004e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2398:x1866777609704960:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 950us (1035us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.608410:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10385 00000100:00100000:1.0:1780298645.609377:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.609414:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609705216 00000100:00100000:2.0:1780298645.609419:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609705216 00000100:00100000:2.0:1780298645.609458:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6486 00000100:00100000:2.0:1780298645.609467:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a82a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609705216:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298645.609614:0:8362:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:2.0:1780298645.609618:0:8362:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:2.0:1780298645.609620:0:8362:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298646 -> 1780298645 00002000:00010000:2.0:1780298645.609623:0:8362:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:2.0:1780298645.609625:0:8362:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 408 -> 408 00000100:00100000:2.0:1780298645.609654:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a82a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609705216:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 190us (278us total) trans 4294969786 rc 0/0 00000100:00100000:2.0:1780298645.609662:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6486 00000100:00100000:1.0:1780298645.610498:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.610527:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.610529:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609705344 00000100:00100000:2.0:1780298645.610533:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609705344 00000100:00100000:0.0:1780298645.610563:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609705472 00000100:00100000:2.0:1780298645.610566:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10387 00000100:00100000:0.0:1780298645.610568:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609705472 00000100:00100000:2.0:1780298645.610573:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a82d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609705344:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.610581:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.610591:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d71a6aba/0xaf35132e63036028 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4e5f expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.610602:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10388 00000100:00100000:0.0:1780298645.610609:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243061500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609705472:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.610615:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.610623:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c703984a/0xaf35132e6303602f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4e66 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.610635:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255090a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605099264:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.610650:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b42300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605099392:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.610688:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4300 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.610689:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c900 going to sleep for 11 seconds 00002000:00010000:2.0:1780298645.611434:0:8356:0:(ofd_lvb.c:271:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime: 1780298645 -> 1780298646 00000100:00100000:2.0:1780298645.611444:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255090a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605099264:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.611461:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d71a6aba/0xaf35132e63036028 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e5f expref: 8 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.611477:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d71a6aba/0xaf35132e63036028 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e5f expref: 8 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.611485:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9bc00, rc 0) 00000100:00100000:2.0:1780298645.611510:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a82d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609705344:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 939us (1014us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.611516:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10387 00000100:00100000:0.0:1780298645.611545:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b42300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605099392:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.611566:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c703984a/0xaf35132e6303602f lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e66 expref: 12 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.611579:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c703984a/0xaf35132e6303602f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e66 expref: 12 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.611589:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2000, rc 0) 00000100:00100000:0.0:1780298645.611620:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243061500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609705472:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1011us (1093us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.611628:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10388 00000100:00100000:1.0:1780298645.612526:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.612547:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609705600 00000100:00100000:3.0:1780298645.612552:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609705600 00000100:00100000:2.0:1780298645.612602:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10389 00000100:00100000:2.0:1780298645.612612:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243063800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609705600:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.612622:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.612630:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e63036036 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4e6d expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.612659:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255090a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605099520:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.612670:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.612692:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609705728 00000100:00100000:3.0:1780298645.612697:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609705728 00000100:00100000:2.0:1780298645.612703:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c900 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.612747:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10390 00000100:00100000:2.0:1780298645.612756:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243063100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609705728:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.612763:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.612772:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b0cbb9d/0xaf35132e6303603d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4e74 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.612798:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605099648:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.612831:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c840 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.613291:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255090a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605099520:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.613306:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e63036036 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e6d expref: 8 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.613316:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e63036036 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e6d expref: 8 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.613324:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9a600, rc 0) 00000100:00100000:2.0:1780298645.613347:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243063800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609705600:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 737us (821us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.613355:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10389 00000100:00100000:2.0:1780298645.613468:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605099648:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.613479:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003b0cbb9d/0xaf35132e6303603d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e74 expref: 12 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.613489:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b0cbb9d/0xaf35132e6303603d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e74 expref: 12 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.613496:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b800, rc 0) 00000100:00100000:2.0:1780298645.613517:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243063100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609705728:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 765us (849us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.613522:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10390 00000100:00100000:1.0:1780298645.614476:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.614496:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609705856 00000100:00100000:3.0:1780298645.614500:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609705856 00000100:00100000:1.0:1780298645.614505:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.614525:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609705984 00000100:00100000:3.0:1780298645.614527:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609705984 00000100:00100000:3.0:1780298645.614546:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10391 00000100:00100000:3.0:1780298645.614554:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243060000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609705856:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298645.614558:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10392 00010000:00010000:3.0:1780298645.614563:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298645.614571:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243061f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609705984:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.614572:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e06fa3fe/0xaf35132e63036044 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4e7b expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.614586:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.614595:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b4561703/0xaf35132e6303604b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4e82 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.614602:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf348007480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605099776:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.614632:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605099904:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.614637:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2418563c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.614664:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c840 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.615310:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348007480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605099776:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.615333:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e06fa3fe/0xaf35132e63036044 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e7b expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.615349:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e06fa3fe/0xaf35132e63036044 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e7b expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.615360:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341693e00, rc 0) 00000100:00100000:2.0:1780298645.615391:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605099904:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.615410:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b4561703/0xaf35132e6303604b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e82 expref: 14 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.615425:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b4561703/0xaf35132e6303604b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e82 expref: 14 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.615434:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b600, rc 0) 00000100:00100000:3.0:1780298645.615446:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243060000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609705856:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 892us (969us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.615455:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10391 00000100:00100000:2.0:1780298645.615462:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243061f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609705984:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 895us (956us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.615471:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10392 00000100:00100000:0.0:1780298645.616255:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.616284:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609706112 00000100:00100000:3.0:1780298645.616289:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609706112 00000100:00100000:3.0:1780298645.616318:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10393 00000100:00100000:3.0:1780298645.616326:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:29360:x1866777609706112:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.616334:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.616342:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d7062f72/0xaf35132e63036052 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4e89 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.616358:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(55 169) 00010000:00010000:3.0:1780298645.616362:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(55 169) 00010000:00010000:3.0:1780298645.616371:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d7062f72/0xaf35132e63036052 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2408448->3702783] (req 2408448->2416639) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e89 expref: 14 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.616385:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d7062f72/0xaf35132e63036052 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2408448->3702783] (req 2408448->2416639) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e89 expref: 14 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.616393:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf341692e00, rc 0) 00000100:00100000:3.0:1780298645.616418:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:29360:x1866777609706112:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 93us (163us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298645.616425:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10393 00000100:00100000:0.0:1780298645.617077:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.617156:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609706240 00000100:00100000:1.0:1780298645.617161:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609706240 00000100:00100000:1.0:1780298645.617199:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6487 00000100:00100000:1.0:1780298645.617207:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609706240:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298645.617374:0:19943:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:1.0:1780298645.617377:0:19943:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:1.0:1780298645.617379:0:19943:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298646 -> 1780298645 00002000:00010000:1.0:1780298645.617381:0:19943:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:1.0:1780298645.617382:0:19943:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 408 -> 424 00000100:00100000:1.0:1780298645.617406:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2399:x1866777609706240:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 202us (331us total) trans 4294969787 rc 0/0 00000100:00100000:1.0:1780298645.617410:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6487 00000100:00100000:1.0:1780298645.618234:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.618281:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609706368 00000100:00100000:2.0:1780298645.618287:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609706368 00000100:00100000:0.0:1780298645.618322:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.618326:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10394 00000100:00100000:2.0:1780298645.618336:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243063480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609706368:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298645.618338:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609706496 00000100:00100000:3.0:1780298645.618343:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609706496 00010000:00010000:2.0:1780298645.618346:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.618357:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e63036059 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4e90 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.618372:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10395 00000100:00100000:1.0:1780298645.618383:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609706496:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.618390:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.618399:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017dab243/0xaf35132e63036060 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4e97 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.618405:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605100032:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.618430:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243062300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605100160:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.618466:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.618467:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c840 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.619053:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243062300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605100160:192.168.203.25@tcp:106: 00002000:00010000:2.0:1780298645.619070:0:9128:0:(ofd_lvb.c:271:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime: 1780298645 -> 1780298646 00010000:00010000:3.0:1780298645.619071:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000017dab243/0xaf35132e63036060 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e97 expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.619087:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605100032:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.619088:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017dab243/0xaf35132e63036060 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e97 expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.619101:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279800, rc 0) 00010000:00010000:2.0:1780298645.619113:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e63036059 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e90 expref: 8 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.619151:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609706496:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 770us (831us total) trans 0 rc 301/301 00010000:00010000:2.0:1780298645.619154:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e63036059 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e90 expref: 8 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.619168:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10395 00010000:00010000:2.0:1780298645.619171:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9a200, rc 0) 00000100:00100000:2.0:1780298645.619192:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243063480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609706368:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 859us (959us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.619198:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10394 00000100:00100000:0.0:1780298645.619969:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.620007:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609706624 00000100:00100000:2.0:1780298645.620012:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609706624 00000100:00100000:2.0:1780298645.620046:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10396 00000100:00100000:2.0:1780298645.620056:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609706624:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.620064:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.620074:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e0a0f245/0xaf35132e63036067 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4e9e expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.620079:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.620108:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605100288:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.620109:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609706752 00000100:00100000:3.0:1780298645.620175:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609706752 00000100:00100000:2.0:1780298645.620178:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c840 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.620202:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10397 00000100:00100000:3.0:1780298645.620210:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243062d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609706752:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.620216:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.620223:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000038a88423/0xaf35132e6303606e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4ea5 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.620294:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf348007480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605100416:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.620319:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2418563c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.620731:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605100288:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.620746:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e0a0f245/0xaf35132e63036067 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e9e expref: 8 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.620759:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e0a0f245/0xaf35132e63036067 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e9e expref: 8 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.620769:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b200, rc 0) 00000100:00100000:2.0:1780298645.620799:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609706624:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 745us (830us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.620806:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10396 00000100:00100000:1.0:1780298645.620937:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348007480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605100416:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.620954:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000038a88423/0xaf35132e6303606e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4ea5 expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.620969:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038a88423/0xaf35132e6303606e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4ea5 expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.620979:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93e00, rc 0) 00000100:00100000:1.0:1780298645.621007:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243062d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609706752:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 798us (931us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.621014:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10397 00000100:00100000:0.0:1780298645.621896:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.621952:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609706880 00000100:00100000:3.0:1780298645.621956:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609706880 00000100:00100000:3.0:1780298645.621979:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10398 00000100:00100000:3.0:1780298645.621985:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2398:x1866777609706880:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.621992:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298645.621992:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:3.0:1780298645.622000:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000670f33f0/0xaf35132e63036075 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4eac expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.622027:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf348007480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605100544:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.622032:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609707008 00000100:00100000:2.0:1780298645.622037:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609707008 00000100:00100000:2.0:1780298645.622062:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10399 00000100:00100000:2.0:1780298645.622071:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609707008:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.622080:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.622091:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e6303607c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4eb3 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.622097:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856180 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.622120:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605100672:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.622191:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c840 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.622758:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348007480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605100544:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.622777:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000670f33f0/0xaf35132e63036075 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4eac expref: 16 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.622780:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605100672:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.622798:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e6303607c lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4eb3 expref: 14 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.622807:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000670f33f0/0xaf35132e63036075 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4eac expref: 16 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.622819:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92e00, rc 0) 00010000:00010000:2.0:1780298645.622819:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e6303607c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4eb3 expref: 14 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.622833:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b000, rc 0) 00000100:00100000:3.0:1780298645.622852:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2398:x1866777609706880:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 866us (956us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.622860:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609707008:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 789us (868us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.622862:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10398 00000100:00100000:2.0:1780298645.622872:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10399 00000100:00100000:0.0:1780298645.624168:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.624235:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609707136 00000100:00100000:1.0:1780298645.624251:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609707136 00000100:00100000:1.0:1780298645.624332:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10400 00000100:00100000:1.0:1780298645.624341:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:29360:x1866777609707136:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.624351:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.624360:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b25cdf87/0xaf35132e63036083 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4eba expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.624380:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e0cae770/0xaf35132e63035fe9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2539520->3170303] (req 2539520->2543615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e20 expref: 10 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.624389:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(53 122) 00010000:00010000:1.0:1780298645.624393:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(54 122) 00010000:00010000:1.0:1780298645.624398:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b25cdf87/0xaf35132e63036083 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2990080->2994175] (req 2990080->2994175) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4eba expref: 14 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.624421:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e0cae770/0xaf35132e63035fe9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2539520->3170303] (req 2539520->2543615) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e20 expref: 10 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.624435:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e0cae770/0xaf35132e63035fe9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2539520->3170303] (req 2539520->2543615) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e20 expref: 10 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.624447:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243063b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605100800:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298645.624490:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.625202:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243063b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605100800:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298645.625217:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b25cdf87/0xaf35132e63036083 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2990080->2994175] (req 2990080->2994175) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4eba expref: 14 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.625228:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278200, rc 0) 00000100:00100000:1.0:1780298645.625251:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:29360:x1866777609707136:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 913us (1088us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298645.625257:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10400 00000100:00100000:0.0:1780298645.625471:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.625498:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609707264 00000100:00100000:3.0:1780298645.625504:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609707264 00010000:00010000:3.0:1780298645.625525:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e0cae770/0xaf35132e63035fe9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2539520->3170303] (req 2539520->2543615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e20 expref: 11 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.625553:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15301 00000100:00100000:3.0:1780298645.625561:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609707264:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298645.625571:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298645.625586:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e0cae770/0xaf35132e63035fe9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [2539520->3170303] (req 2539520->2543615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e20 expref: 11 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.625598:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e0cae770/0xaf35132e63035fe9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [2539520->3170303] (req 2539520->2543615) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e20 expref: 11 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.625616:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0cae770/0xaf35132e63035fe9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [2539520->3170303] (req 2539520->2543615) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e20 expref: 11 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.625627:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(54 122) 00010000:00010000:3.0:1780298645.625630:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(54 122) 00010000:00010000:3.0:1780298645.625636:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b25cdf87/0xaf35132e63036083 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2990080->3170303] (req 2990080->2994175) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4eba expref: 13 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.625642:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b25cdf87/0xaf35132e63036083 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2990080->3170303] (req 2990080->2994175) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4eba expref: 13 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.625659:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000b25cdf87/0xaf35132e63036083 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2990080->3170303] (req 2990080->2994175) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4eba expref: 13 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.625666:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf348007480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6461:1866777605100928:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298645.625694:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856180 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.626300:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348007480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6461:1866777605100928:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298645.626311:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298645.626337:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b2a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609707264:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 777us (867us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298645.626345:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15301 00000100:00100000:0.0:1780298645.626484:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.626520:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609707392 00000100:00100000:1.0:1780298645.626524:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609707392 00000100:00100000:1.0:1780298645.626554:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6488 00000100:00100000:1.0:1780298645.626559:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609707392:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298645.626684:0:19951:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 3178729 -> 2992038 00002000:00010000:1.0:1780298645.626687:0:19951:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:1.0:1780298645.626688:0:19951:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298646 -> 1780298645 00002000:00010000:1.0:1780298645.626690:0:19951:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:1.0:1780298645.626691:0:19951:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 32 -> 40 00000100:00100000:1.0:1780298645.626713:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2399:x1866777609707392:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 154us (230us total) trans 4294969325 rc 0/0 00000100:00100000:1.0:1780298645.626717:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6488 00000100:00100000:0.0:1780298645.627544:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.627579:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609707520 00000100:00100000:1.0:1780298645.627583:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609707520 00000100:00100000:1.0:1780298645.627609:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10401 00000100:00100000:1.0:1780298645.627616:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2398:x1866777609707520:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.627622:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298645.627629:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:1.0:1780298645.627631:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e6303608a lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4ec1 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.627657:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243063b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605101056:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.627679:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609707648 00000100:00100000:2.0:1780298645.627685:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609707648 00000100:00100000:1.0:1780298645.627692:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.627716:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10402 00000100:00100000:2.0:1780298645.627725:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2398:x1866777609707648:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.627734:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.627743:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000019780e8d/0xaf35132e63036091 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4ec8 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.627788:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605101184:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.627824:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.628348:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243063b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605101056:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.628370:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e6303608a lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4ec1 expref: 16 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.628387:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e6303608a lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4ec1 expref: 16 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.628398:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278a00, rc 0) 00002000:00010000:2.0:1780298645.628409:0:20252:0:(ofd_lvb.c:259:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size: 2992038 -> 3178729 00002000:00010000:2.0:1780298645.628414:0:20252:0:(ofd_lvb.c:271:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime: 1780298645 -> 1780298646 00000100:00100000:2.0:1780298645.628424:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605101184:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.628429:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2398:x1866777609707520:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 811us (883us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.628438:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10401 00010000:00010000:2.0:1780298645.628450:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000019780e8d/0xaf35132e63036091 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4ec8 expref: 16 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.628482:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000019780e8d/0xaf35132e63036091 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4ec8 expref: 16 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.628496:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9a200, rc 0) 00000100:00100000:2.0:1780298645.628524:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2398:x1866777609707648:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 801us (895us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.628532:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10402 00000100:00100000:0.0:1780298645.629438:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.629464:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.629479:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609707776 00000100:00100000:2.0:1780298645.629483:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609707776 00000100:00100000:3.0:1780298645.629494:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609707904 00000100:00100000:3.0:1780298645.629501:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609707904 00000100:00100000:2.0:1780298645.629515:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10403 00000100:00100000:2.0:1780298645.629522:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25488aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2399:x1866777609707776:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.629529:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.629536:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009fd8d343/0xaf35132e63036098 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4ecf expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.629547:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10404 00000100:00100000:1.0:1780298645.629559:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25488b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2400:x1866777609707904:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298645.629561:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605101312:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.629574:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.629585:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e6303609f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4ed6 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.629612:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.629646:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243063b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605101440:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.629679:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.630249:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255092680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605101312:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.630333:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009fd8d343/0xaf35132e63036098 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4ecf expref: 16 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.630340:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243063b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605101440:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.630362:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e6303609f lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4ed6 expref: 16 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.630364:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009fd8d343/0xaf35132e63036098 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4ecf expref: 16 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.630390:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b000, rc 0) 00010000:00010000:1.0:1780298645.630394:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e6303609f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4ed6 expref: 16 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.630406:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279e00, rc 0) 00000100:00100000:3.0:1780298645.630421:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25488aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2399:x1866777609707776:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 898us (982us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.630428:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10403 00000100:00100000:1.0:1780298645.630435:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25488b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2400:x1866777609707904:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 879us (968us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.630442:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10404 00000100:00100000:3.0:1780298645.631534:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.631574:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609708032 00000100:00100000:0.0:1780298645.631579:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609708032 00000100:00100000:0.0:1780298645.631613:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10405 00000100:00100000:0.0:1780298645.631622:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348007480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:29360:x1866777609708032:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.631630:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.631640:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000488a7b88/0xaf35132e630360a6 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4edd expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.631657:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(55 169) 00010000:00010000:0.0:1780298645.631661:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(55 169) 00010000:00010000:0.0:1780298645.631670:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000488a7b88/0xaf35132e630360a6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [786432->2408447] (req 786432->790527) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4edd expref: 16 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.631689:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000488a7b88/0xaf35132e630360a6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [786432->2408447] (req 786432->790527) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4edd expref: 16 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.631698:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2200, rc 0) 00000100:00100000:0.0:1780298645.631730:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348007480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:29360:x1866777609708032:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 110us (197us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.631737:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10405 00000100:00100000:1.0:1780298645.632370:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.632410:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609708160 00000100:00100000:0.0:1780298645.632413:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609708160 00000100:00100000:0.0:1780298645.632439:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6489 00000100:00100000:0.0:1780298645.632444:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243063b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2398:x1866777609708160:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298645.632572:0:8364:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:0.0:1780298645.632574:0:8364:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298645.632576:0:8364:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298646 -> 1780298645 00002000:00010000:0.0:1780298645.632578:0:8364:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298645.632580:0:8364:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 424 -> 432 00000100:00100000:0.0:1780298645.632608:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243063b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2398:x1866777609708160:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 163us (238us total) trans 4294969788 rc 0/0 00000100:00100000:0.0:1780298645.632615:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6489 00000100:00100000:3.0:1780298645.633373:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.633407:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609708288 00000100:00100000:2.0:1780298645.633412:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609708288 00000100:00100000:2.0:1780298645.633440:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10406 00000100:00100000:2.0:1780298645.633448:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242861c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609708288:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.633456:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.633464:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e0a0f245/0xaf35132e630360ad lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4ee4 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.633501:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255090a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605101568:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.633513:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.633540:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c900 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.633557:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609708416 00000100:00100000:0.0:1780298645.633562:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609708416 00000100:00100000:0.0:1780298645.633590:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10407 00000100:00100000:0.0:1780298645.633599:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243060a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2400:x1866777609708416:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.633608:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.633617:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c0d26bbc/0xaf35132e630360b4 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4eeb expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.633643:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254888000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605101696:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.633669:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4300 going to sleep for 11 seconds 00002000:00010000:2.0:1780298645.634077:0:20252:0:(ofd_lvb.c:271:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime: 1780298645 -> 1780298646 00000100:00100000:2.0:1780298645.634085:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255090a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605101568:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.634097:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e0a0f245/0xaf35132e630360ad lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4ee4 expref: 18 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.634107:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e0a0f245/0xaf35132e630360ad lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4ee4 expref: 18 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.634113:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b200, rc 0) 00000100:00100000:2.0:1780298645.634154:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242861c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609708288:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 708us (784us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.634161:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10406 00000100:00100000:1.0:1780298645.634282:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254888000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605101696:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.634301:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c0d26bbc/0xaf35132e630360b4 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4eeb expref: 16 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.634317:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c0d26bbc/0xaf35132e630360b4 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4eeb expref: 16 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.634327:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3c00, rc 0) 00000100:00100000:1.0:1780298645.634358:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243060a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2400:x1866777609708416:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 760us (847us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.634365:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10407 00000100:00100000:0.0:1780298645.635148:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.635171:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609708544 00000100:00100000:1.0:1780298645.635174:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609708544 00000100:00100000:0.0:1780298645.635185:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.635206:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609708672 00000100:00100000:1.0:1780298645.635209:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609708672 00000100:00100000:3.0:1780298645.635218:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10408 00000100:00100000:3.0:1780298645.635253:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25488ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2399:x1866777609708544:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298645.635253:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10409 00010000:00010000:3.0:1780298645.635262:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.635281:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000670f33f0/0xaf35132e630360bb lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4ef2 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.635283:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254889180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2400:x1866777609708672:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.635296:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.635306:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b4561703/0xaf35132e630360c2 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4ef9 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.635320:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242861180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605101824:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.635337:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255090a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605101952:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.635372:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856180 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.635372:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c900 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.635997:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242861180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605101824:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.636011:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000670f33f0/0xaf35132e630360bb lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4ef2 expref: 18 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.636022:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000670f33f0/0xaf35132e630360bb lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4ef2 expref: 18 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.636033:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92e00, rc 0) 00000100:00100000:2.0:1780298645.636043:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255090a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605101952:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.636057:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25488ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2399:x1866777609708544:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 807us (910us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.636063:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10408 00010000:00010000:2.0:1780298645.636063:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b4561703/0xaf35132e630360c2 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4ef9 expref: 16 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.636081:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b4561703/0xaf35132e630360c2 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4ef9 expref: 16 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.636093:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b600, rc 0) 00000100:00100000:2.0:1780298645.636117:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254889180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2400:x1866777609708672:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 853us (931us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.636144:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10409 00000100:00100000:0.0:1780298645.636968:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.636987:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609708928 00000100:00100000:3.0:1780298645.636990:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609708928 00000100:00100000:0.0:1780298645.636998:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.637018:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609708800 00000100:00100000:3.0:1780298645.637020:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609708800 00000100:00100000:2.0:1780298645.637038:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10410 00000100:00100000:3.0:1780298645.637044:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10411 00000100:00100000:2.0:1780298645.637050:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254888380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2397:x1866777609708928:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298645.637056:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254888000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2398:x1866777609708800:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.637060:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.637063:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.637071:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003b0cbb9d/0xaf35132e630360c9 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4f07 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.637073:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000056a806f/0xaf35132e630360d0 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4f00 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.637112:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255090a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605102080:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.637169:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243062300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605102208:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.637191:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c900 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.637203:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.637827:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255090a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605102080:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.637842:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003b0cbb9d/0xaf35132e630360c9 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f07 expref: 16 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.637855:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b0cbb9d/0xaf35132e630360c9 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f07 expref: 16 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.637866:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b800, rc 0) 00000100:00100000:1.0:1780298645.637894:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243062300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605102208:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.637899:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254888380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2397:x1866777609708928:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 852us (930us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.637905:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10410 00010000:00010000:1.0:1780298645.637915:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000056a806f/0xaf35132e630360d0 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f00 expref: 18 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.637931:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000056a806f/0xaf35132e630360d0 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f00 expref: 18 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.637943:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92800, rc 0) 00000100:00100000:1.0:1780298645.637975:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254888000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2398:x1866777609708800:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 922us (976us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.637983:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10411 00000100:00100000:0.0:1780298645.638788:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.638808:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609709056 00000100:00100000:3.0:1780298645.638811:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609709056 00000100:00100000:3.0:1780298645.638832:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10412 00000100:00100000:3.0:1780298645.638836:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e424850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:29360:x1866777609709056:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.638842:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.638848:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e630360d7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4f0e expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.638859:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(54 122) 00010000:00010000:3.0:1780298645.638861:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(54 122) 00010000:00010000:3.0:1780298645.638867:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e630360d7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2719744->2990079] (req 2719744->2723839) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f0e expref: 16 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.638875:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e630360d7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2719744->2990079] (req 2719744->2723839) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f0e expref: 16 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.638880:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93000, rc 0) 00000100:00100000:3.0:1780298645.638900:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e424850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:29360:x1866777609709056:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 64us (113us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298645.638904:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10412 00000100:00100000:0.0:1780298645.639649:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.639681:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609709184 00000100:00100000:2.0:1780298645.639686:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609709184 00002000:00100000:2.0:1780298645.639712:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x12:0x0] (2719744->2723839) req@ffff9cf25488aa00 x1866777609709184/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.639721:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25488aa00 with x1866777609709184 ext(2719744->2723839) 00002000:00010000:2.0:1780298645.639726:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf25488aa00 00000100:00100000:2.0:1780298645.639742:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6490 00000100:00100000:2.0:1780298645.639748:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25488aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2399:x1866777609709184:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298645.639866:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25488aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2399:x1866777609709184:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 119us (219us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298645.639871:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6490 00002000:00100000:2.0:1780298645.639876:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x12:0x0] (2719744->2723839) req@ffff9cf25488aa00 x1866777609709184/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/4536 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.639881:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25488aa00 with x1866777609709184 ext(2719744->2723839) 00002000:00010000:2.0:1780298645.639884:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf25488aa00 00000100:00100000:0.0:1780298645.640837:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.640868:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609709312 00000100:00100000:2.0:1780298645.640872:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609709312 00002000:00100000:2.0:1780298645.640886:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (2723840->2727935) req@ffff9cf25488b800 x1866777609709312/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.640896:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25488b800 with x1866777609709312 ext(2723840->2727935) 00002000:00010000:2.0:1780298645.640898:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf25488b800 00000100:00100000:2.0:1780298645.640913:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6491 00000100:00100000:2.0:1780298645.640916:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25488b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2398:x1866777609709312:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298645.640976:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25488b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2398:x1866777609709312:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 59us (141us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298645.640981:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6491 00002000:00100000:2.0:1780298645.640984:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (2723840->2727935) req@ffff9cf25488b800 x1866777609709312/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/4536 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.640990:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25488b800 with x1866777609709312 ext(2723840->2727935) 00002000:00010000:2.0:1780298645.640993:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf25488b800 00000100:00100000:0.0:1780298645.641854:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.641882:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609709440 00000100:00100000:1.0:1780298645.641886:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609709440 00002000:00100000:1.0:1780298645.641902:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x12:0x0] (2727936->2732031) req@ffff9cf25488ad80 x1866777609709440/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.641909:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25488ad80 with x1866777609709440 ext(2727936->2732031) 00002000:00010000:1.0:1780298645.641911:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf25488ad80 00000100:00100000:1.0:1780298645.641929:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6492 00000100:00100000:1.0:1780298645.641934:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25488ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2398:x1866777609709440:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298645.641999:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25488ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2398:x1866777609709440:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 64us (145us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298645.642004:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6492 00002000:00100000:1.0:1780298645.642008:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x12:0x0] (2727936->2732031) req@ffff9cf25488ad80 x1866777609709440/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/4536 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.642014:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25488ad80 with x1866777609709440 ext(2727936->2732031) 00002000:00010000:1.0:1780298645.642017:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf25488ad80 00000100:00100000:0.0:1780298645.642946:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.642974:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.642984:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609709568 00000100:00100000:2.0:1780298645.642988:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609709568 00000100:00100000:1.0:1780298645.643006:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609709696 00000100:00100000:1.0:1780298645.643010:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609709696 00000100:00100000:2.0:1780298645.643021:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10413 00000100:00100000:2.0:1780298645.643034:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254889180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609709568:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.643041:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298645.643049:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10414 00010000:00010000:2.0:1780298645.643050:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e630360de lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4f15 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.643063:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254888380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609709696:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.643073:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298645.643083:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255090a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605102336:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.643084:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e2cfab1/0xaf35132e630360e5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4f1c expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.643146:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6cf00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.643150:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254888000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605102464:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.643189:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4300 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.643742:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255090a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605102336:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.643757:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e630360de lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f15 expref: 8 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.643769:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e630360de lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f15 expref: 8 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.643776:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9a200, rc 0) 00000100:00100000:2.0:1780298645.643800:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254889180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609709568:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 769us (856us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.643804:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10413 00000100:00100000:1.0:1780298645.643879:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254888000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605102464:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.643897:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003e2cfab1/0xaf35132e630360e5 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f1c expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.643911:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e2cfab1/0xaf35132e630360e5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f1c expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.643923:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2e00, rc 0) 00000100:00100000:1.0:1780298645.643949:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254888380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609709696:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 892us (977us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.643955:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10414 00000100:00100000:0.0:1780298645.644659:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.644687:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609709824 00000100:00100000:1.0:1780298645.644692:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609709824 00000100:00100000:1.0:1780298645.644716:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10415 00000100:00100000:1.0:1780298645.644722:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254888000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609709824:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.644728:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.644735:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e630360ec lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4f23 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.644756:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243062300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605102592:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.644766:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.644795:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e540 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.644816:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609709952 00000100:00100000:3.0:1780298645.644823:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609709952 00000100:00100000:2.0:1780298645.644863:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10416 00000100:00100000:2.0:1780298645.644869:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609709952:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.644876:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.644882:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c1d9addf/0xaf35132e630360f3 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4f2a expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.644901:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255090a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605102720:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.644926:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6cf00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.645321:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243062300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605102592:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.645334:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e630360ec lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f23 expref: 8 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.645345:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e630360ec lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f23 expref: 8 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.645352:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279e00, rc 0) 00000100:00100000:1.0:1780298645.645375:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254888000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609709824:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 653us (717us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.645380:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10415 00000100:00100000:2.0:1780298645.645497:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255090a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605102720:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.645510:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c1d9addf/0xaf35132e630360f3 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f2a expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.645520:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c1d9addf/0xaf35132e630360f3 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f2a expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.645527:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9a600, rc 0) 00000100:00100000:2.0:1780298645.645554:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609709952:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 686us (790us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.645558:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10416 00000100:00100000:0.0:1780298645.646349:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.646379:0:17567:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609710208 00000100:00100000:2.0:1780298645.646383:0:17567:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609710208 00000100:00100000:2.0:1780298645.646411:0:17567:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62086 00000100:00100000:2.0:1780298645.646416:0:17567:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+47:29360:x1866777609710208:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:2.0:1780298645.646449:0:17567:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298645.646461:0:17567:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf341692200/0xaf35132e63035f6b lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 21 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4da9 expref: 47 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.646471:0:17567:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298645.646474:0:17567:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298645.646594:0:17567:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b800/0xaf35132e630360fa lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 20 type: IBT flags: 0x40000000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.646604:0:17567:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2800/0xaf35132e63035f80 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 20 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4db7 expref: 38 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.646612:0:17567:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b800/0xaf35132e630360fa lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 20 type: IBT flags: 0x50210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.646627:0:17567:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2800/0xaf35132e63035f80 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 20 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4db7 expref: 38 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.646636:0:17567:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2800/0xaf35132e63035f80 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 20 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4db7 expref: 38 pid: 6477 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.646647:0:17567:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255090a80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:17567:1866777605102848:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298645.646677:0:17567:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c900 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.647199:0:17567:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255090a80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:17567:1866777605102848:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298645.647209:0:17567:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b800/0xaf35132e630360fa lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 20 type: IBT flags: 0x40210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.647400:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.647435:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609710336 00000100:00100000:2.0:1780298645.647439:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609710336 00010000:00010000:2.0:1780298645.647457:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2800/0xaf35132e63035f80 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 20 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4db7 expref: 39 pid: 6477 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.647483:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15302 00000100:00100000:2.0:1780298645.647491:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+39:2400:x1866777609710336:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298645.647501:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298645.647507:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2800/0xaf35132e63035f80 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 21 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4db7 expref: 39 pid: 6477 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.647518:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2800/0xaf35132e63035f80 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 21 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4db7 expref: 39 pid: 6477 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.647530:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2800/0xaf35132e63035f80 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 21 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4db7 expref: 39 pid: 6477 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.647540:0:8374:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298645.647543:0:8374:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b800/0xaf35132e630360fa lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 20 type: IBT flags: 0x40210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.647552:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b800/0xaf35132e630360fa lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 20 type: IBT flags: 0x40290000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.647559:0:8374:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b800/0xaf35132e630360fa lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 20 type: IBT flags: 0x40290000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.647579:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1780298645.647597:0:17567:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b800/0xaf35132e630360fa lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 19 type: IBT flags: 0x40210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.647607:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+38:2400:x1866777609710336:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 116us (206us total) trans 0 rc 0/0 00010000:00010000:0.0:1780298645.647609:0:17567:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b800/0xaf35132e630360fa lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 19 type: IBT flags: 0x40210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.647622:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15302 00000001:00080000:0.0:1780298645.647664:0:17567:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991126 00000004:00010000:0.0:1780298645.647682:0:17567:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000b7bec9f0 reply state 0000000053f9d30a transno 4294991127 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b800/0xaf35132e630360fa lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 19 type: IBT flags: 0x40210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.647692:0:17567:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b800/0xaf35132e630360fa lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 19 type: IBT flags: 0x50210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.647696:0:17567:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298645.647698:0:17567:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1780298645.647700:0:17567:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1780298645.647744:0:17567:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+49:29360:x1866777609710208:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1328us (1395us total) trans 4294991127 rc 0/0 00000100:00100000:0.0:1780298645.647749:0:17567:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62086 00000020:00080000:0.0:1780298645.647751:0:17567:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1780298645.648418:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.648454:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609710464 00000100:00100000:1.0:1780298645.648458:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609710464 00000100:00100000:1.0:1780298645.648487:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10417 00000100:00100000:1.0:1780298645.648492:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609710464:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.648498:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.648505:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63036101 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4f31 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.648523:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000488a7b88/0xaf35132e630360a6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [786432->2408447] (req 786432->790527) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4edd expref: 16 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.648529:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d7062f72/0xaf35132e63036052 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2408448->3702783] (req 2408448->2416639) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e89 expref: 16 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.648534:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63035fc6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3702784->37257215] (req 3702784->3710975) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4dfd expref: 16 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.648538:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 3 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(55 169) 00010000:00010000:1.0:1780298645.648541:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(58 169) 00010000:00010000:1.0:1780298645.648543:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63036101 lrc: 6/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f31 expref: 8 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.648555:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63035fc6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3702784->37257215] (req 3702784->3710975) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4dfd expref: 16 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.648561:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63035fc6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3702784->37257215] (req 3702784->3710975) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4dfd expref: 16 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.648568:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243062300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605102976:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298645.648610:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d7062f72/0xaf35132e63036052 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2408448->3702783] (req 2408448->2416639) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e89 expref: 16 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.648624:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d7062f72/0xaf35132e63036052 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2408448->3702783] (req 2408448->2416639) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e89 expref: 16 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.648634:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605103104:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298645.648664:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000488a7b88/0xaf35132e630360a6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [786432->2408447] (req 786432->790527) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4edd expref: 16 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.648674:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000488a7b88/0xaf35132e630360a6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [786432->2408447] (req 786432->790527) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4edd expref: 16 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.648683:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243063800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605103232:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298645.648709:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e540 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.649184:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243062300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605102976:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298645.649358:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605103104:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298645.649412:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243063800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605103232:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298645.649429:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63036101 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f31 expref: 8 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.649438:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278a00, rc 0) 00000100:00100000:1.0:1780298645.649461:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:29360:x1866777609710464:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 969us (1045us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298645.649466:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10417 00000100:00100000:0.0:1780298645.649562:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.649591:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609710848 00000100:00100000:1.0:1780298645.649595:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609710848 00010000:00010000:1.0:1780298645.649608:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000488a7b88/0xaf35132e630360a6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [786432->2408447] (req 786432->790527) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4edd expref: 17 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.649628:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15303 00000100:00100000:1.0:1780298645.649633:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609710848:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298645.649639:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298645.649650:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000488a7b88/0xaf35132e630360a6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [786432->2408447] (req 786432->790527) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4edd expref: 17 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.649661:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000488a7b88/0xaf35132e630360a6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [786432->2408447] (req 786432->790527) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4edd expref: 17 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.649678:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000488a7b88/0xaf35132e630360a6 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [786432->2408447] (req 786432->790527) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4edd expref: 17 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.649690:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298645.649702:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.649724:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609710848:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 88us (160us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298645.649731:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15303 00000100:00100000:0.0:1780298645.649736:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609710592 00000100:00100000:0.0:1780298645.649740:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609710592 00010000:00010000:0.0:1780298645.649758:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63035fc6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3702784->37257215] (req 3702784->3710975) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4dfd expref: 16 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.649777:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15304 00000100:00100000:0.0:1780298645.649783:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242861c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2399:x1866777609710592:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298645.649789:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298645.649796:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63035fc6 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3702784->37257215] (req 3702784->3710975) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4dfd expref: 16 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.649803:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63035fc6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3702784->37257215] (req 3702784->3710975) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4dfd expref: 16 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.649813:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63035fc6 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3702784->37257215] (req 3702784->3710975) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4dfd expref: 16 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.649820:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298645.649851:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242861c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2399:x1866777609710592:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 68us (149us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.649855:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15304 00000100:00100000:1.0:1780298645.649876:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.649894:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609710720 00000100:00100000:0.0:1780298645.649898:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609710720 00010000:00010000:0.0:1780298645.649909:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d7062f72/0xaf35132e63036052 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2408448->3702783] (req 2408448->2416639) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e89 expref: 15 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.649929:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15305 00000100:00100000:0.0:1780298645.649935:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243063800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2399:x1866777609710720:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298645.649940:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298645.649946:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d7062f72/0xaf35132e63036052 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2408448->3702783] (req 2408448->2416639) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e89 expref: 15 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.649953:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d7062f72/0xaf35132e63036052 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2408448->3702783] (req 2408448->2416639) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e89 expref: 15 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.649966:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d7062f72/0xaf35132e63036052 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2408448->3702783] (req 2408448->2416639) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4e89 expref: 15 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.649973:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(58 169) 00010000:00010000:0.0:1780298645.649975:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(58 169) 00010000:00010000:0.0:1780298645.649979:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63036101 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f31 expref: 7 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.649984:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63036101 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f31 expref: 7 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.650000:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63036101 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f31 expref: 7 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.650006:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304ed80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777605103360:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298645.650038:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4f00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.650630:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304ed80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777605103360:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298645.650640:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298645.650665:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243063800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609710720:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 731us (790us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.650671:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15305 00000100:00100000:2.0:1780298645.650678:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.650701:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609710976 00000100:00100000:1.0:1780298645.650705:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609710976 00000100:00100000:1.0:1780298645.650731:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10418 00000100:00100000:1.0:1780298645.650736:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255090a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:29360:x1866777609710976:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.650743:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.650751:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e63036108 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4f38 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.650765:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000b25cdf87/0xaf35132e63036083 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [2990080->3170303] (req 2990080->2994175) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4eba expref: 15 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.650771:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d1b192d6/0xaf35132e63035f95 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [3170304->36724735] (req 3170304->3182591) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4dcc expref: 11 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.650777:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e630360d7 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [2719744->2990079] (req 2719744->2723839) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f0e expref: 15 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.650781:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 3 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(54 122) 00010000:00010000:1.0:1780298645.650784:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(57 122) 00010000:00010000:1.0:1780298645.650786:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e63036108 lrc: 6/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f38 expref: 11 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.650799:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e630360d7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [2719744->2990079] (req 2719744->2723839) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f0e expref: 15 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.650806:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e630360d7 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [2719744->2990079] (req 2719744->2723839) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f0e expref: 15 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.650812:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605103488:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298645.650845:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d1b192d6/0xaf35132e63035f95 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [3170304->36724735] (req 3170304->3182591) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4dcc expref: 11 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.650857:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d1b192d6/0xaf35132e63035f95 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [3170304->36724735] (req 3170304->3182591) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4dcc expref: 11 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.650864:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243062300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605103616:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298645.650880:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000b25cdf87/0xaf35132e63036083 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [2990080->3170303] (req 2990080->2994175) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4eba expref: 15 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.650887:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000b25cdf87/0xaf35132e63036083 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [2990080->3170303] (req 2990080->2994175) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4eba expref: 15 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.650893:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243063100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605103744:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298645.650906:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e540 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.651441:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605103488:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298645.651450:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243062300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605103616:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298645.651484:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243063100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605103744:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298645.651499:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e63036108 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f38 expref: 11 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.651511:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278400, rc 0) 00000100:00100000:1.0:1780298645.651535:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255090a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:29360:x1866777609710976:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 799us (859us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298645.651542:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10418 00000100:00100000:0.0:1780298645.651958:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.651993:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.651996:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609711104 00000100:00100000:2.0:1780298645.652001:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609711104 00000100:00100000:0.0:1780298645.652017:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:2.0:1780298645.652024:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e630360d7 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [2719744->2990079] (req 2719744->2723839) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f0e expref: 16 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.652031:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609711232 00000100:00100000:1.0:1780298645.652037:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609711232 00000100:00100000:2.0:1780298645.652056:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609711360 00000100:00100000:2.0:1780298645.652061:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609711360 00000100:00100000:0.0:1780298645.652063:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15306 00002000:00100000:1.0:1780298645.652067:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (3170304->3178728) req@ffff9cf375f57050 x1866777609711232/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 8920/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00000100:00100000:0.0:1780298645.652075:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609711104:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298645.652080:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000b25cdf87/0xaf35132e63036083 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [2990080->3170303] (req 2990080->2994175) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4eba expref: 17 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.652088:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00100000:1.0:1780298645.652092:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf375f57050 with x1866777609711232 ext(3170304->3178728) 00010000:00010000:0.0:1780298645.652103:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e630360d7 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [2719744->2990079] (req 2719744->2723839) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f0e expref: 17 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.652105:0:19951:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000d1b192d6/0xaf35132e63035f95 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [3170304->36724735] (req 3170304->3182591) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4dcc expref: 11 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.652118:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15307 00010000:00010000:0.0:1780298645.652122:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e630360d7 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [2719744->2990079] (req 2719744->2723839) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f0e expref: 17 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.652150:0:19951:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000d1b192d6/0xaf35132e63035f95 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [3170304->36724735] (req 3170304->3182591) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4dcc expref: 11 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.652156:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e630360d7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [2719744->2990079] (req 2719744->2723839) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f0e expref: 17 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.652161:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b1ec50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609711360:12345-192.168.203.25@tcp:103:ldlm_bl.0 00002000:00010000:1.0:1780298645.652168:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9cf375f57050 00010000:00010000:2.0:1780298645.652177:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298645.652179:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298645.652190:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000b25cdf87/0xaf35132e63036083 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [2990080->3170303] (req 2990080->2994175) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4eba expref: 16 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.652196:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 218 00000100:00100000:0.0:1780298645.652205:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609711104:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 132us (245us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298645.652212:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf375f57050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609711232:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:0.0:1780298645.652221:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15306 00010000:00010000:2.0:1780298645.652231:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000b25cdf87/0xaf35132e63036083 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [2990080->3170303] (req 2990080->2994175) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4eba expref: 16 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.652245:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b25cdf87/0xaf35132e63036083 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [2990080->3170303] (req 2990080->2994175) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4eba expref: 15 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.652256:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298645.652292:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b1ec50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609711360:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 138us (275us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.652300:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15307 00000100:00100000:1.0:1780298645.652720:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf375f57050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2397:x1866777609711232:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 514us (728us total) trans 4294969326 rc 0/0 00000100:00100000:1.0:1780298645.652728:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 218 00002000:00100000:1.0:1780298645.652735:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (3170304->3178728) req@ffff9cf375f57050 x1866777609711232/t4294969326(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 8920/448 e 0 to 0 dl 1780298656 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.652743:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf375f57050 with x1866777609711232 ext(3170304->3178728) 00010000:00010000:1.0:1780298645.652749:0:19951:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000d1b192d6/0xaf35132e63035f95 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3170304->36724735] (req 3170304->3182591) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4dcc expref: 13 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.652761:0:19951:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000d1b192d6/0xaf35132e63035f95 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3170304->36724735] (req 3170304->3182591) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4dcc expref: 13 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1780298645.652773:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9cf375f57050 00000100:00100000:0.0:1780298645.653475:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.653511:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609711488 00000100:00100000:1.0:1780298645.653515:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609711488 00010000:00010000:1.0:1780298645.653527:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d1b192d6/0xaf35132e63035f95 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3170304->36724735] (req 3170304->3182591) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4dcc expref: 13 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.653547:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15308 00000100:00100000:1.0:1780298645.653554:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2398:x1866777609711488:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298645.653563:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:1.0:1780298645.653570:0:8374:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 40 -> 64 00010000:00010000:1.0:1780298645.653576:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d1b192d6/0xaf35132e63035f95 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3170304->36724735] (req 3170304->3182591) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4dcc expref: 13 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.653587:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d1b192d6/0xaf35132e63035f95 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3170304->36724735] (req 3170304->3182591) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4dcc expref: 13 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.653599:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d1b192d6/0xaf35132e63035f95 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3170304->36724735] (req 3170304->3182591) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4dcc expref: 13 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.653607:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(57 122) 00010000:00010000:1.0:1780298645.653610:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(57 122) 00010000:00010000:1.0:1780298645.653613:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e63036108 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f38 expref: 12 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.653622:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e63036108 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f38 expref: 12 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.653640:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e63036108 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f38 expref: 12 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.653647:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243063b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:8374:1866777605103872:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298645.653678:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e540 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.654444:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243063b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:8374:1866777605103872:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298645.654456:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298645.654482:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609711488:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 929us (1008us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298645.654490:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15308 00000100:00100000:2.0:1780298645.654572:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.654607:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609711616 00000100:00100000:0.0:1780298645.654613:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609711616 00002000:00010000:0.0:1780298645.654629:0:8364:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (1753737->18446744073709551615) 00002000:00100000:0.0:1780298645.654633:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255090000 with x1866777609711616 ext(1753737->18446744073709551615) 00002000:00010000:0.0:1780298645.654641:0:8364:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255090000. 00000100:00100000:0.0:1780298645.654660:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6493 00000100:00100000:0.0:1780298645.654668:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255090000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609711616:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:1.0:1780298645.654766:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.654807:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609711744 00000100:00100000:2.0:1780298645.654811:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609711744 00002000:00010000:2.0:1780298645.654827:0:16972:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00002000:00100000:2.0:1780298645.654831:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243063b80 with x1866777609711744 ext(0->18446744073709551615) 00002000:00010000:2.0:1780298645.654838:0:16972:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf243063b80. 00000100:00100000:2.0:1780298645.654863:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6494 00000100:00100000:2.0:1780298645.654872:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243063b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609711744:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:0.0:1780298645.654880:0:8364:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 1753737 00002000:00010000:0.0:1780298645.654885:0:8364:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298645.654888:0:8364:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298646 -> 1780298645 00002000:00010000:0.0:1780298645.654891:0:8364:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298645.654895:0:8364:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 432 -> 400 00000100:00100000:0.0:1780298645.654930:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255090000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609711616:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 261us (356us total) trans 4294969789 rc 0/0 00000100:00100000:0.0:1780298645.654938:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6493 00002000:00010000:0.0:1780298645.654942:0:8364:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (1753737->18446744073709551615) 00002000:00100000:0.0:1780298645.654946:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255090000 with x1866777609711616 ext(1753737->18446744073709551615) 00002000:00010000:0.0:1780298645.654951:0:8364:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255090000. 00002000:00010000:2.0:1780298645.655080:0:16972:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 3178729 -> 0 00002000:00010000:2.0:1780298645.655083:0:16972:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:2.0:1780298645.655086:0:16972:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298646 -> 1780298646 00002000:00010000:2.0:1780298645.655088:0:16972:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:2.0:1780298645.655090:0:16972:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 64 -> 0 00000100:00100000:2.0:1780298645.655121:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243063b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609711744:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 251us (356us total) trans 4294969327 rc 0/0 00000100:00100000:2.0:1780298645.655172:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6494 00002000:00010000:2.0:1780298645.655176:0:16972:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00002000:00100000:2.0:1780298645.655179:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243063b80 with x1866777609711744 ext(0->18446744073709551615) 00002000:00010000:2.0:1780298645.655185:0:16972:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf243063b80. 00000100:00100000:1.0:1780298645.655931:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.655972:0:9976:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609711872 00000100:00100000:0.0:1780298645.655977:0:9976:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609711872 00000100:00100000:0.0:1780298645.656009:0:9976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62087 00000100:00100000:0.0:1780298645.656016:0:9976:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243063100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+38:29360:x1866777609711872:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.656029:0:9976:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.656043:0:9976:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3c00/0xaf35132e6303610f lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 20 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4f3f expref: -99 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.656143:0:9976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3200/0xaf35132e63036116 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 21 type: IBT flags: 0x40000000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.656158:0:9976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3200/0xaf35132e63036116 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 21 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.656170:0:9976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3200/0xaf35132e63036116 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 21 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1780298645.656195:0:9976:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 1753737 00000004:00200000:0.0:1780298645.656199:0:9976:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1780298645.656216:0:9976:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3200/0xaf35132e63036116 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 21 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.656242:0:9976:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3c00/0xaf35132e6303610f lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 21 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f3f expref: 40 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.656254:0:9976:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3200/0xaf35132e63036116 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 20 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f3f expref: 39 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.656263:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298645.656266:0:9976:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1780298645.656281:0:9976:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3200, rc 0) 00000100:00100000:0.0:1780298645.656309:0:9976:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243063100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+39:29360:x1866777609711872:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 294us (378us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.656316:0:9976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62087 00000100:00100000:0.0:1780298645.657183:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.657265:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609712000 00000100:00100000:3.0:1780298645.657280:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609712000 00000100:00100000:1.0:1780298645.657297:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.657330:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609712128 00000100:00100000:0.0:1780298645.657336:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609712128 00000100:00100000:3.0:1780298645.657344:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10419 00000100:00100000:3.0:1780298645.657354:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609712000:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.657361:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298645.657362:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10420 00010000:00010000:3.0:1780298645.657371:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e6303611d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4f46 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.657376:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243062300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609712128:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.657389:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.657400:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c1d9addf/0xaf35132e63036124 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4f4d expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.657422:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242861180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605104000:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.657432:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255093100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605104128:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.657473:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856180 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.657473:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c840 going to sleep for 11 seconds 00002000:00010000:3.0:1780298645.658106:0:8356:0:(ofd_lvb.c:271:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime: 1780298645 -> 1780298646 00000100:00100000:3.0:1780298645.658119:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242861180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605104000:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.658199:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e6303611d lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f46 expref: 15 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.658210:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255093100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605104128:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.658220:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e6303611d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f46 expref: 15 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.658227:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c1d9addf/0xaf35132e63036124 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f4d expref: 15 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.658235:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92400, rc 0) 00010000:00010000:2.0:1780298645.658240:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c1d9addf/0xaf35132e63036124 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f4d expref: 15 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.658250:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9a600, rc 0) 00000100:00100000:3.0:1780298645.658281:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609712000:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 918us (1086us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.658288:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243062300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609712128:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 914us (988us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.658295:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10419 00000100:00100000:2.0:1780298645.658304:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10420 00000100:00100000:0.0:1780298645.659162:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.659191:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.659214:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609712256 00000100:00100000:3.0:1780298645.659219:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609712256 00000100:00100000:1.0:1780298645.659226:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609712384 00000100:00100000:1.0:1780298645.659230:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609712384 00000100:00100000:3.0:1780298645.659257:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10421 00000100:00100000:0.0:1780298645.659263:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10422 00000100:00100000:3.0:1780298645.659278:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2398:x1866777609712256:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298645.659286:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2397:x1866777609712384:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.659287:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.659293:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.659302:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0cae770/0xaf35132e6303612b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4f5b expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.659317:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e63036132 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4f54 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.659327:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605104256:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.659347:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605104384:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.659368:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4f00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.659375:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e540 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.660099:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605104256:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.660115:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e0cae770/0xaf35132e6303612b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f5b expref: 15 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.660146:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0cae770/0xaf35132e6303612b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f5b expref: 15 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.660153:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2600, rc 0) 00000100:00100000:2.0:1780298645.660173:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605104384:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.660175:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2397:x1866777609712384:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 904us (983us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.660184:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10422 00010000:00010000:2.0:1780298645.660203:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e63036132 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f54 expref: 15 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.660218:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e63036132 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f54 expref: 15 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.660225:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279e00, rc 0) 00000100:00100000:2.0:1780298645.660249:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2398:x1866777609712256:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 986us (1091us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.660254:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10421 00000100:00100000:0.0:1780298645.662469:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.662494:0:17578:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609712640 00000100:00100000:3.0:1780298645.662502:0:17578:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609712640 00000100:00100000:3.0:1780298645.662537:0:17578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62088 00000100:00100000:3.0:1780298645.662544:0:17578:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+39:29360:x1866777609712640:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:3.0:1780298645.662594:0:17578:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298645.662616:0:17578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3200/0xaf35132e63036116 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 21 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f3f expref: 39 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.662632:0:17578:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298645.662636:0:17578:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1780298645.662780:0:17578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92e00/0xaf35132e63036139 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 20 type: IBT flags: 0x40000000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.662801:0:17578:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92e00/0xaf35132e63036139 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 20 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.662810:0:17578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92e00/0xaf35132e63036139 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 20 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1780298645.662850:0:17578:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991128 00000004:00010000:3.0:1780298645.662868:0:17578:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 000000007c166668 reply state 00000000db3bb5d3 transno 4294991129 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92e00/0xaf35132e63036139 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 20 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.662879:0:17578:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e92e00/0xaf35132e63036139 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 20 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.662885:0:17578:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298645.662887:0:17578:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1780298645.662889:0:17578:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1780298645.662953:0:17578:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+41:29360:x1866777609712640:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 408us (489us total) trans 4294991129 rc 0/0 00000100:00100000:3.0:1780298645.662960:0:17578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62088 00000020:00080000:3.0:1780298645.662965:0:17578:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1780298645.663701:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.663736:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609712768 00000100:00100000:2.0:1780298645.663740:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609712768 00000100:00100000:2.0:1780298645.663766:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10423 00000100:00100000:2.0:1780298645.663772:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243060000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:29360:x1866777609712768:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.663779:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.663788:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e63036140 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4f62 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.663805:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63036101 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f31 expref: 8 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.663811:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(58 169) 00010000:00010000:2.0:1780298645.663815:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(59 169) 00010000:00010000:2.0:1780298645.663818:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e63036140 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f62 expref: 15 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.663843:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63036101 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f31 expref: 8 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.663854:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63036101 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f31 expref: 8 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.663864:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255093100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605104512:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298645.663905:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c840 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.664601:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255093100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605104512:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298645.664621:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e63036140 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f62 expref: 15 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.664635:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9a600, rc 0) 00000100:00100000:2.0:1780298645.664664:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243060000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:29360:x1866777609712768:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 893us (965us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.664670:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10423 00000100:00100000:1.0:1780298645.664734:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.664761:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609712896 00000100:00100000:3.0:1780298645.664767:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609712896 00010000:00010000:3.0:1780298645.664843:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63036101 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f31 expref: 9 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.664864:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15309 00000100:00100000:3.0:1780298645.664869:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243061f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609712896:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298645.664876:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298645.664889:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63036101 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f31 expref: 9 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.664896:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63036101 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f31 expref: 9 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.664906:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63036101 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f31 expref: 9 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.664913:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(59 169) 00010000:00010000:3.0:1780298645.664915:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(59 169) 00010000:00010000:3.0:1780298645.664919:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e63036140 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f62 expref: 14 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.664927:0:6463:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e63036140 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f62 expref: 14 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.664950:0:6463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e63036140 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f62 expref: 14 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.664960:0:6463:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242861180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6463:1866777605104640:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298645.664997:0:6463:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856180 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.665693:0:6463:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242861180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6463:1866777605104640:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298645.665701:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298645.665763:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243061f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609712896:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 894us (1030us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298645.665770:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15309 00000100:00100000:1.0:1780298645.665835:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.665875:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609713024 00000100:00100000:2.0:1780298645.665879:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609713024 00000100:00100000:2.0:1780298645.665902:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10424 00000100:00100000:2.0:1780298645.665908:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243063480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:29360:x1866777609713024:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.665916:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.665924:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e7fd0135/0xaf35132e63036147 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4f69 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.665947:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e63036108 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f38 expref: 12 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.665955:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(57 122) 00010000:00010000:2.0:1780298645.665959:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(58 122) 00010000:00010000:2.0:1780298645.665964:0:20252:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7fd0135/0xaf35132e63036147 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f69 expref: 15 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.665983:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e63036108 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f38 expref: 12 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.665994:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e63036108 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f38 expref: 12 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.666006:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255093100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605104768:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298645.666062:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c840 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.666613:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255093100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605104768:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298645.666625:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e7fd0135/0xaf35132e63036147 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f69 expref: 15 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.666634:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9be00, rc 0) 00000100:00100000:2.0:1780298645.666657:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243063480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:29360:x1866777609713024:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 750us (826us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.666665:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10424 00000100:00100000:0.0:1780298645.666876:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.666904:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609713152 00000100:00100000:3.0:1780298645.666909:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609713152 00010000:00010000:3.0:1780298645.666925:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e63036108 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f38 expref: 13 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.666950:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15310 00000100:00100000:3.0:1780298645.666957:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609713152:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298645.666966:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298645.666977:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e63036108 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f38 expref: 13 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.666989:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e63036108 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f38 expref: 13 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.667003:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e63036108 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f38 expref: 13 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.667013:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(58 122) 00010000:00010000:3.0:1780298645.667016:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(58 122) 00010000:00010000:3.0:1780298645.667022:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000e7fd0135/0xaf35132e63036147 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f69 expref: 14 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.667031:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e7fd0135/0xaf35132e63036147 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f69 expref: 14 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.667055:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000e7fd0135/0xaf35132e63036147 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f69 expref: 14 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.667064:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242861180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6461:1866777605104896:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298645.667099:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856180 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.667774:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242861180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6461:1866777605104896:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298645.667798:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298645.667823:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609713152:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 867us (948us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298645.667830:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15310 00000100:00100000:0.0:1780298645.668102:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.668148:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.668162:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609713280 00000100:00100000:2.0:1780298645.668168:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609713280 00000100:00100000:3.0:1780298645.668188:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609713408 00002000:00010000:2.0:1780298645.668188:0:8362:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00000100:00100000:3.0:1780298645.668193:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609713408 00002000:00100000:2.0:1780298645.668194:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24304ed80 with x1866777609713280 ext(4194304->18446744073709551615) 00002000:00010000:2.0:1780298645.668203:0:8362:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24304ed80. 00002000:00010000:3.0:1780298645.668206:0:19950:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (3838373->18446744073709551615) 00002000:00100000:3.0:1780298645.668210:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24304d500 with x1866777609713408 ext(3838373->18446744073709551615) 00002000:00010000:3.0:1780298645.668214:0:19950:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24304d500. 00000100:00100000:2.0:1780298645.668226:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6495 00000100:00100000:2.0:1780298645.668237:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2400:x1866777609713280:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:3.0:1780298645.668242:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6496 00000100:00100000:3.0:1780298645.668253:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2400:x1866777609713408:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:2.0:1780298645.668374:0:8362:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 1753737 -> 4194304 00002000:00010000:2.0:1780298645.668379:0:8362:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:2.0:1780298645.668382:0:8362:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298646 -> 1780298645 00002000:00010000:2.0:1780298645.668384:0:8362:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:2.0:1780298645.668386:0:8362:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 400 -> 400 00002000:00010000:3.0:1780298645.668388:0:19950:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 0 -> 3838373 00002000:00010000:3.0:1780298645.668392:0:19950:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:3.0:1780298645.668394:0:19950:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298646 -> 1780298646 00002000:00010000:3.0:1780298645.668397:0:19950:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:3.0:1780298645.668399:0:19950:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:2.0:1780298645.668420:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609713280:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 185us (317us total) trans 4294969790 rc 0/0 00000100:00100000:3.0:1780298645.668426:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609713408:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 174us (278us total) trans 4294969328 rc 0/0 00000100:00100000:2.0:1780298645.668434:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6495 00000100:00100000:3.0:1780298645.668440:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6496 00002000:00010000:2.0:1780298645.668442:0:8362:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00010000:3.0:1780298645.668443:0:19950:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (3838373->18446744073709551615) 00002000:00100000:2.0:1780298645.668448:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24304ed80 with x1866777609713280 ext(4194304->18446744073709551615) 00002000:00100000:3.0:1780298645.668449:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24304d500 with x1866777609713408 ext(3838373->18446744073709551615) 00002000:00010000:2.0:1780298645.668456:0:8362:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24304ed80. 00002000:00010000:3.0:1780298645.668457:0:19950:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24304d500. 00000100:00100000:0.0:1780298645.669965:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.670003:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609713536 00000100:00100000:1.0:1780298645.670007:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609713536 00002000:00100000:1.0:1780298645.670029:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (3837952->3842047) req@ffff9cf37f745180 x1866777609713536/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.670037:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f745180 with x1866777609713536 ext(3837952->3842047) 00002000:00010000:1.0:1780298645.670039:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f745180 00000100:00100000:1.0:1780298645.670054:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6497 00000100:00100000:1.0:1780298645.670060:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2399:x1866777609713536:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298645.670179:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2399:x1866777609713536:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 119us (217us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298645.670184:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6497 00002000:00100000:1.0:1780298645.670189:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (3837952->3842047) req@ffff9cf37f745180 x1866777609713536/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/4536 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.670195:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f745180 with x1866777609713536 ext(3837952->3842047) 00002000:00010000:1.0:1780298645.670197:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f745180 00000100:00100000:1.0:1780298645.671314:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.671366:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609713664 00000100:00100000:2.0:1780298645.671371:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609713664 00002000:00100000:2.0:1780298645.671394:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x12:0x0] (3833856->3837951) req@ffff9cf243062d80 x1866777609713664/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.671404:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243062d80 with x1866777609713664 ext(3833856->3837951) 00002000:00010000:2.0:1780298645.671409:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf243062d80 00000100:00100000:2.0:1780298645.671427:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6498 00000100:00100000:2.0:1780298645.671435:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243062d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609713664:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298645.671517:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243062d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609713664:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 83us (204us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298645.671522:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6498 00002000:00100000:2.0:1780298645.671526:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x12:0x0] (3833856->3837951) req@ffff9cf243062d80 x1866777609713664/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/4536 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.671532:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243062d80 with x1866777609713664 ext(3833856->3837951) 00002000:00010000:2.0:1780298645.671536:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf243062d80 00000100:00100000:0.0:1780298645.672584:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.672627:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609713792 00000100:00100000:2.0:1780298645.672631:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609713792 00002000:00100000:2.0:1780298645.672647:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x12:0x0] (3833856->3838372) req@ffff9cf37f745c00 x1866777609713792/t0(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 5008/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.672653:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f745c00 with x1866777609713792 ext(3833856->3838372) 00002000:00010000:2.0:1780298645.672656:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f745c00 00000100:00100000:2.0:1780298645.672670:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6499 00000100:00100000:2.0:1780298645.672675:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609713792:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:2.0:1780298645.673058:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2397:x1866777609713792:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 382us (476us total) trans 4294969329 rc 0/0 00000100:00100000:2.0:1780298645.673064:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6499 00002000:00100000:2.0:1780298645.673069:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x12:0x0] (3833856->3838372) req@ffff9cf37f745c00 x1866777609713792/t4294969329(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 5008/448 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.673079:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f745c00 with x1866777609713792 ext(3833856->3838372) 00002000:00010000:2.0:1780298645.673081:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f745c00 00000100:00100000:0.0:1780298645.673734:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.673772:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609713920 00000100:00100000:2.0:1780298645.673776:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609713920 00000100:00100000:2.0:1780298645.673797:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10425 00000100:00100000:2.0:1780298645.673801:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2398:x1866777609713920:12345-192.168.203.25@tcp:16:fsx.0 00000001:00080000:1.0:1780298645.674472:0:8351:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0000: transno 4294969329 is committed 00000001:00080000:1.0:1780298645.674483:0:8351:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0000: transno 4294969327 is committed 00000100:00100000:2.0:1780298645.674575:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609713920:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 771us (841us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.674586:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10425 00000100:00100000:1.0:1780298645.675436:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.675470:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609714048 00000100:00100000:0.0:1780298645.675476:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609714048 00000100:00100000:0.0:1780298645.675510:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62089 00000100:00100000:0.0:1780298645.675518:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243060a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+49:29360:x1866777609714048:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.675534:0:6477:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.675553:0:6477:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2000/0xaf35132e6303614e lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 21 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4f70 expref: -99 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.675630:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3c00/0xaf35132e63036155 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 22 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.675646:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3c00/0xaf35132e63036155 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 22 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.675659:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3c00/0xaf35132e63036155 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 22 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1780298645.675684:0:6477:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 8032677 00000004:00200000:0.0:1780298645.675688:0:6477:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1780298645.675705:0:6477:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3c00/0xaf35132e63036155 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 22 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.675731:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2000/0xaf35132e6303614e lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 22 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f70 expref: 51 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.675748:0:6477:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3c00/0xaf35132e63036155 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 21 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f70 expref: 50 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.675757:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298645.675759:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1780298645.675763:0:6477:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3c00, rc 0) 00000100:00100000:0.0:1780298645.675796:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243060a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+50:29360:x1866777609714048:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 279us (361us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.675803:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62089 00000100:00100000:0.0:1780298645.676772:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.676822:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609714176 00000100:00100000:2.0:1780298645.676827:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609714176 00000100:00100000:1.0:1780298645.676863:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.676867:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10426 00000100:00100000:2.0:1780298645.676878:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f747100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609714176:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.676888:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298645.676896:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609714304 00010000:00010000:2.0:1780298645.676900:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e6303615c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4f77 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.676901:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609714304 00000100:00100000:0.0:1780298645.676925:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10427 00000100:00100000:0.0:1780298645.676932:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243061500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609714304:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.676939:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.676947:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5a492f0/0xaf35132e63036163 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4f7e expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.676955:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255093100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605105024:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.676979:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f745500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605105152:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.677026:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba40c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.677027:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00002000:00010000:2.0:1780298645.677874:0:12101:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks: 0 -> 16 00002000:00010000:1.0:1780298645.677877:0:19992:0:(ofd_lvb.c:271:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime: 1780298645 -> 1780298646 00000100:00100000:2.0:1780298645.677889:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f745500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605105152:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.677890:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255093100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605105024:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.677914:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e6303615c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f77 expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.677920:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a5a492f0/0xaf35132e63036163 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f7e expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.677930:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e6303615c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f77 expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.677950:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9aa00, rc 0) 00010000:00010000:2.0:1780298645.677959:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5a492f0/0xaf35132e63036163 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f7e expref: 7 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.677971:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3600, rc 0) 00000100:00100000:1.0:1780298645.677978:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f747100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609714176:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1104us (1208us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.677983:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10426 00000100:00100000:2.0:1780298645.677999:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243061500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609714304:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1067us (1136us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.678007:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10427 00000100:00100000:3.0:1780298645.678984:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.679009:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.679024:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609714432 00000100:00100000:2.0:1780298645.679029:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609714432 00000100:00100000:1.0:1780298645.679039:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609714560 00000100:00100000:1.0:1780298645.679043:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609714560 00000100:00100000:2.0:1780298645.679059:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10428 00000100:00100000:2.0:1780298645.679065:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242861180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609714432:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.679073:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298645.679076:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10429 00010000:00010000:2.0:1780298645.679082:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e6303616a lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4f85 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.679090:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242861f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609714560:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.679099:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.679110:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63036171 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4f8c expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.679111:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255092300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605105280:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.679159:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243063800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605105408:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.679192:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.679192:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.679831:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255092300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605105280:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.679845:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e6303616a lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f85 expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.679858:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e6303616a lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f85 expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.679870:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9ac00, rc 0) 00000100:00100000:2.0:1780298645.679897:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242861180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609714432:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 833us (915us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.679902:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10428 00000100:00100000:1.0:1780298645.679938:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243063800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605105408:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.679965:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63036171 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f8c expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.679979:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63036171 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f8c expref: 7 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.679988:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279200, rc 0) 00000100:00100000:1.0:1780298645.680014:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242861f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609714560:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 930us (1003us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.680020:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10429 00000100:00100000:0.0:1780298645.680882:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.680920:0:17567:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609714688 00000100:00100000:2.0:1780298645.680924:0:17567:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609714688 00000100:00100000:2.0:1780298645.680954:0:17567:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62090 00000100:00100000:2.0:1780298645.680964:0:17567:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f746680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+41:29360:x1866777609714688:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.680978:0:17567:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.680991:0:17567:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a200/0xaf35132e63036178 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 22 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4f93 expref: -99 pid: 17567 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.681053:0:17567:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9bc00/0xaf35132e6303617f lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 23 type: IBT flags: 0x40000000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.681063:0:17567:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9bc00/0xaf35132e6303617f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 23 type: IBT flags: 0x50210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.681071:0:17567:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9bc00/0xaf35132e6303617f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 23 type: IBT flags: 0x40210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298645.681086:0:17567:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 8032677 00000004:00200000:2.0:1780298645.681089:0:17567:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298645.681101:0:17567:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9bc00/0xaf35132e6303617f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 23 type: IBT flags: 0x40210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.681115:0:17567:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a200/0xaf35132e63036178 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 23 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f93 expref: 43 pid: 17567 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.681152:0:17567:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9bc00/0xaf35132e6303617f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 22 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f93 expref: 42 pid: 17567 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.681159:0:17567:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298645.681161:0:17567:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298645.681163:0:17567:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9bc00, rc 0) 00000100:00100000:2.0:1780298645.681185:0:17567:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f746680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+42:29360:x1866777609714688:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 226us (307us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.681190:0:17567:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62090 00000100:00100000:3.0:1780298645.682421:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.682464:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609714816 00000100:00100000:1.0:1780298645.682469:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609714816 00000100:00100000:3.0:1780298645.682496:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.682501:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10430 00000100:00100000:1.0:1780298645.682508:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242862d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609714816:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.682516:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298645.682521:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609714944 00010000:00010000:1.0:1780298645.682524:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63036186 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4f9a expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.682525:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609714944 00000100:00100000:1.0:1780298645.682560:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243063800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605105536:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.682562:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10431 00000100:00100000:2.0:1780298645.682571:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242863b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609714944:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.682578:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.682588:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0a0f245/0xaf35132e6303618d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4fa1 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.682620:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255092300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605105664:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.682622:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.682650:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.683247:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243063800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605105536:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.683281:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63036186 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f9a expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.683298:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63036186 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f9a expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.683308:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279a00, rc 0) 00000100:00100000:1.0:1780298645.683343:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242862d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609714816:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 835us (923us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.683353:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10430 00000100:00100000:2.0:1780298645.683380:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255092300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605105664:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.683395:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e0a0f245/0xaf35132e6303618d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fa1 expref: 7 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.683405:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0a0f245/0xaf35132e6303618d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fa1 expref: 7 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.683412:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b200, rc 0) 00000100:00100000:2.0:1780298645.683427:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242863b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609714944:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 858us (932us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.683432:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10431 00000100:00100000:3.0:1780298645.684267:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.684314:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609715072 00000100:00100000:0.0:1780298645.684318:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609715072 00000100:00100000:0.0:1780298645.684342:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10432 00000100:00100000:0.0:1780298645.684347:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242860000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:29360:x1866777609715072:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.684354:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.684361:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000488a7b88/0xaf35132e63036194 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4fa8 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.684375:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e7fd0135/0xaf35132e63036147 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f69 expref: 9 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.684382:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(58 122) 00010000:00010000:0.0:1780298645.684385:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(59 122) 00010000:00010000:0.0:1780298645.684388:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000488a7b88/0xaf35132e63036194 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1531904->1535999] (req 1531904->1535999) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fa8 expref: 7 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.684403:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e7fd0135/0xaf35132e63036147 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f69 expref: 9 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.684413:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e7fd0135/0xaf35132e63036147 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f69 expref: 9 pid: 20252 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.684420:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f745f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605105792:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298645.684456:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba49c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.685040:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f745f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605105792:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298645.685057:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000488a7b88/0xaf35132e63036194 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1531904->1535999] (req 1531904->1535999) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fa8 expref: 7 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.685069:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2200, rc 0) 00000100:00100000:0.0:1780298645.685101:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242860000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609715072:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 752us (835us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.685108:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10432 00000100:00100000:2.0:1780298645.685318:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.685355:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609715200 00000100:00100000:0.0:1780298645.685360:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609715200 00010000:00010000:0.0:1780298645.685380:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e7fd0135/0xaf35132e63036147 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f69 expref: 10 pid: 20252 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.685409:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15311 00000100:00100000:0.0:1780298645.685416:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255092300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609715200:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298645.685426:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298645.685436:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e7fd0135/0xaf35132e63036147 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f69 expref: 10 pid: 20252 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.685448:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e7fd0135/0xaf35132e63036147 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f69 expref: 10 pid: 20252 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.685463:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e7fd0135/0xaf35132e63036147 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f69 expref: 10 pid: 20252 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.685473:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(59 122) 00010000:00010000:0.0:1780298645.685477:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(59 122) 00010000:00010000:0.0:1780298645.685484:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000488a7b88/0xaf35132e63036194 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1531904->1535999) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fa8 expref: 6 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.685493:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000488a7b88/0xaf35132e63036194 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1531904->1535999) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fa8 expref: 6 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.685514:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000488a7b88/0xaf35132e63036194 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1531904->1535999) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fa8 expref: 6 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.685524:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f745f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:20253:1866777605105920:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298645.685559:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba49c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.686292:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f745f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:20253:1866777605105920:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298645.686303:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298645.686326:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255092300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609715200:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 911us (1009us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.686332:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15311 00000100:00100000:3.0:1780298645.686583:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.686625:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609715328 00000100:00100000:0.0:1780298645.686630:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609715328 00002000:00100000:0.0:1780298645.686664:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x12:0x0] (1531904->1535999) req@ffff9cf242861c00 x1866777609715328/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298645.686675:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242861c00 with x1866777609715328 ext(1531904->1535999) 00002000:00010000:0.0:1780298645.686679:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf242861c00 00000100:00100000:0.0:1780298645.686701:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6500 00000100:00100000:0.0:1780298645.686710:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242861c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609715328:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298645.686863:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242861c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609715328:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 155us (281us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298645.686870:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6500 00002000:00100000:0.0:1780298645.686877:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x12:0x0] (1531904->1535999) req@ffff9cf242861c00 x1866777609715328/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/4536 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298645.686886:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf242861c00 with x1866777609715328 ext(1531904->1535999) 00002000:00010000:0.0:1780298645.686889:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf242861c00 00000100:00100000:0.0:1780298645.688000:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.688045:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609715456 00000100:00100000:3.0:1780298645.688050:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609715456 00002000:00100000:3.0:1780298645.688071:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (1536000->1540095) req@ffff9cf37f744000 x1866777609715456/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298645.688081:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f744000 with x1866777609715456 ext(1536000->1540095) 00002000:00010000:3.0:1780298645.688084:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f744000 00000100:00100000:3.0:1780298645.688102:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6501 00000100:00100000:3.0:1780298645.688108:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f744000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609715456:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298645.688220:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f744000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609715456:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 112us (221us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1780298645.688229:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6501 00002000:00100000:3.0:1780298645.688237:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (1536000->1540095) req@ffff9cf37f744000 x1866777609715456/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/4536 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298645.688246:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f744000 with x1866777609715456 ext(1536000->1540095) 00002000:00010000:3.0:1780298645.688250:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f744000 00000100:00100000:0.0:1780298645.689228:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.689262:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609715584 00000100:00100000:1.0:1780298645.689283:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609715584 00000100:00100000:1.0:1780298645.689329:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10433 00000100:00100000:1.0:1780298645.689337:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609715584:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.689351:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.689365:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6416acb/0xaf35132e6303619b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4faf expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.689385:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(59 122) 00010000:00010000:1.0:1780298645.689388:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(59 122) 00010000:00010000:1.0:1780298645.689394:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6416acb/0xaf35132e6303619b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4faf expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.689409:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a6416acb/0xaf35132e6303619b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4faf expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.689419:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278c00, rc 0) 00000100:00100000:1.0:1780298645.689465:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609715584:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 130us (239us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298645.689470:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10433 00000100:00100000:1.0:1780298645.690764:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.690817:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609715712 00000100:00100000:0.0:1780298645.690825:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609715712 00000100:00100000:0.0:1780298645.690872:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6502 00000100:00100000:0.0:1780298645.690882:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243063800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609715712:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298645.691101:0:19951:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:0.0:1780298645.691105:0:19951:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298645.691108:0:19951:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298646 -> 1780298645 00002000:00010000:0.0:1780298645.691110:0:19951:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298645.691112:0:19951:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 400 -> 416 00000100:00100000:0.0:1780298645.691182:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243063800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2397:x1866777609715712:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 302us (416us total) trans 4294969791 rc 0/0 00000100:00100000:0.0:1780298645.691191:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6502 00000100:00100000:0.0:1780298645.691925:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.691959:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609715840 00000100:00100000:3.0:1780298645.691963:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609715840 00000100:00100000:3.0:1780298645.691995:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10434 00000100:00100000:3.0:1780298645.692002:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609715840:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.692012:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.692024:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004cdc8249/0xaf35132e630361a2 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4fb6 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.692063:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605106048:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.692110:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856000 going to sleep for 11 seconds 00002000:00010000:0.0:1780298645.692787:0:8356:0:(ofd_lvb.c:271:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime: 1780298645 -> 1780298646 00000100:00100000:0.0:1780298645.692798:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605106048:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.692823:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004cdc8249/0xaf35132e630361a2 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fb6 expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.692841:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004cdc8249/0xaf35132e630361a2 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fb6 expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.692852:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92200, rc 0) 00000100:00100000:0.0:1780298645.692882:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609715840:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 881us (958us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.692887:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10434 00000100:00100000:1.0:1780298645.693661:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.693705:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609715968 00000100:00100000:2.0:1780298645.693710:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609715968 00000100:00100000:2.0:1780298645.693743:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10435 00000100:00100000:2.0:1780298645.693751:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609715968:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.693760:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.693769:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e630361a9 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4fbd expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.693797:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255090700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605106176:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.693841:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.694496:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255090700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605106176:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.694513:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e630361a9 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fbd expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.694527:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e630361a9 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fbd expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.694536:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9a200, rc 0) 00000100:00100000:1.0:1780298645.694567:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609715968:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 817us (905us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.694574:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10435 00000100:00100000:3.0:1780298645.696602:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.696648:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609716096 00000100:00100000:2.0:1780298645.696655:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609716096 00002000:00100000:2.0:1780298645.696684:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (1073152->1085439) req@ffff9cf36e5db480 x1866777609716096/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.696704:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e5db480 with x1866777609716096 ext(1073152->1085439) 00002000:00010000:2.0:1780298645.696710:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf36e5db480 00000100:00100000:2.0:1780298645.696731:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6503 00000100:00100000:2.0:1780298645.696740:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2397:x1866777609716096:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298645.696911:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5db480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2397:x1866777609716096:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 172us (312us total) trans 0 rc 12288/12288 00000100:00100000:2.0:1780298645.696918:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6503 00002000:00100000:2.0:1780298645.696924:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (1073152->1085439) req@ffff9cf36e5db480 x1866777609716096/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/12728 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 12288/12288 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.696932:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e5db480 with x1866777609716096 ext(1073152->1085439) 00002000:00010000:2.0:1780298645.696936:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf36e5db480 00000100:00100000:3.0:1780298645.697934:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.697971:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609716224 00000100:00100000:0.0:1780298645.697978:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609716224 00000100:00100000:0.0:1780298645.698009:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10436 00000100:00100000:0.0:1780298645.698014:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609716224:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.698021:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.698031:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e630361b0 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4fc4 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.698077:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605106304:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.698150:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4f00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.698821:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605106304:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.698842:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e630361b0 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fc4 expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.698860:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e630361b0 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fc4 expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.698871:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2600, rc 0) 00000100:00100000:2.0:1780298645.698900:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609716224:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 885us (967us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.698907:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10436 00000100:00100000:2.0:1780298645.699779:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.699817:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609716352 00000100:00100000:1.0:1780298645.699823:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609716352 00000100:00100000:1.0:1780298645.699860:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10437 00000100:00100000:1.0:1780298645.699868:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255093b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609716352:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.699879:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.699889:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e630361b7 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4fcb expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.699920:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605106432:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.699974:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.700689:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605106432:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.700706:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e630361b7 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fcb expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.700721:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e630361b7 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fcb expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.700732:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279800, rc 0) 00000100:00100000:1.0:1780298645.700761:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255093b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609716352:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 894us (981us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.700767:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10437 00000100:00100000:0.0:1780298645.701651:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.701689:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609716480 00000100:00100000:3.0:1780298645.701694:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609716480 00000100:00100000:3.0:1780298645.701729:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10438 00000100:00100000:3.0:1780298645.701736:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:29360:x1866777609716480:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.701745:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.701753:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e630361be lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4fd2 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.701772:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e63036140 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f62 expref: 16 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.701782:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(59 169) 00010000:00010000:3.0:1780298645.701786:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(60 169) 00010000:00010000:3.0:1780298645.701791:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e630361be lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1560576->1568767] (req 1560576->1568767) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fd2 expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.701811:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e63036140 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f62 expref: 16 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.701825:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e63036140 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f62 expref: 16 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.701840:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605106560:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298645.701874:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.702415:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605106560:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298645.702430:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e630361be lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1560576->1568767] (req 1560576->1568767) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fd2 expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.702440:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92400, rc 0) 00000100:00100000:3.0:1780298645.702466:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:29360:x1866777609716480:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 731us (817us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298645.702473:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10438 00000100:00100000:1.0:1780298645.702736:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.702771:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609716608 00000100:00100000:0.0:1780298645.702775:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609716608 00010000:00010000:0.0:1780298645.702791:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e63036140 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f62 expref: 17 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.702812:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15312 00000100:00100000:0.0:1780298645.702817:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609716608:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298645.702825:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298645.702833:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e63036140 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f62 expref: 17 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.702841:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e63036140 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f62 expref: 17 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.702853:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e63036140 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f62 expref: 17 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.702864:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(60 169) 00010000:00010000:0.0:1780298645.702866:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(60 169) 00010000:00010000:0.0:1780298645.702872:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e630361be lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1560576->35115007] (req 1560576->1568767) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fd2 expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.702878:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e630361be lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1560576->35115007] (req 1560576->1568767) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fd2 expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.702893:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e630361be lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1560576->35115007] (req 1560576->1568767) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fd2 expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.702899:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255488380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777605106688:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298645.702926:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4300 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.703609:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255488380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777605106688:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298645.703619:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298645.703640:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609716608:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 823us (906us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298645.703647:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15312 00000100:00100000:2.0:1780298645.703882:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.703911:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609716736 00000100:00100000:3.0:1780298645.703915:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609716736 00002000:00100000:3.0:1780298645.703937:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (1560576->1564671) req@ffff9cf255092680 x1866777609716736/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298645.703947:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255092680 with x1866777609716736 ext(1560576->1564671) 00002000:00010000:3.0:1780298645.703951:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255092680 00000100:00100000:3.0:1780298645.703967:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6504 00000100:00100000:3.0:1780298645.703974:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255092680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609716736:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298645.704100:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255092680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609716736:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 126us (218us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1780298645.704107:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6504 00002000:00100000:3.0:1780298645.704113:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (1560576->1564671) req@ffff9cf255092680 x1866777609716736/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/4536 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298645.704121:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255092680 with x1866777609716736 ext(1560576->1564671) 00002000:00010000:3.0:1780298645.704149:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255092680 00000100:00100000:1.0:1780298645.704873:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.704903:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609716864 00000100:00100000:0.0:1780298645.704907:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609716864 00002000:00100000:0.0:1780298645.704922:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (1564672->1568767) req@ffff9cf347cd4e00 x1866777609716864/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298645.704928:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf347cd4e00 with x1866777609716864 ext(1564672->1568767) 00002000:00010000:0.0:1780298645.704930:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf347cd4e00 00000100:00100000:0.0:1780298645.704945:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6505 00000100:00100000:0.0:1780298645.704949:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609716864:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298645.705013:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609716864:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 62us (140us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298645.705023:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6505 00002000:00100000:0.0:1780298645.705030:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (1564672->1568767) req@ffff9cf347cd4e00 x1866777609716864/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/4536 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298645.705038:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf347cd4e00 with x1866777609716864 ext(1564672->1568767) 00002000:00010000:0.0:1780298645.705041:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf347cd4e00 00000100:00100000:0.0:1780298645.706016:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.706046:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609716992 00000100:00100000:1.0:1780298645.706050:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609716992 00000100:00100000:1.0:1780298645.706075:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10439 00000100:00100000:1.0:1780298645.706079:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255489500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609716992:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.706086:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.706095:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e630361c5 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4fd9 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.706139:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605106816:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.706174:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.706807:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605106816:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.706820:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e630361c5 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fd9 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.706831:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e630361c5 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fd9 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.706837:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279a00, rc 0) 00000100:00100000:1.0:1780298645.706861:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255489500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609716992:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 782us (847us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.706866:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10439 00000100:00100000:3.0:1780298645.707617:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.707647:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609717120 00000100:00100000:2.0:1780298645.707651:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609717120 00000100:00100000:2.0:1780298645.707674:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10440 00000100:00100000:2.0:1780298645.707679:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609717120:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.707686:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.707692:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009fd8d343/0xaf35132e630361cc lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4fe0 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.707714:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255091f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605106944:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.707743:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6cf00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.708298:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255091f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605106944:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.708311:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009fd8d343/0xaf35132e630361cc lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fe0 expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.708321:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009fd8d343/0xaf35132e630361cc lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fe0 expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.708327:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b000, rc 0) 00000100:00100000:2.0:1780298645.708350:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609717120:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 672us (736us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.708356:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10440 00000100:00100000:0.0:1780298645.709042:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.709070:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609717248 00000100:00100000:1.0:1780298645.709074:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609717248 00000100:00100000:1.0:1780298645.709098:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10441 00000100:00100000:1.0:1780298645.709103:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:29360:x1866777609717248:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.709110:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.709116:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e630361d3 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4fe7 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.709153:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(60 169) 00010000:00010000:1.0:1780298645.709157:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(60 169) 00010000:00010000:1.0:1780298645.709165:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e630361d3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [389120->1560575] (req 389120->401407) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fe7 expref: 17 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.709174:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e630361d3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [389120->1560575] (req 389120->401407) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fe7 expref: 17 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.709180:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279200, rc 0) 00000100:00100000:1.0:1780298645.709201:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:29360:x1866777609717248:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 99us (160us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298645.709205:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10441 00000100:00100000:3.0:1780298645.709715:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.709749:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609717376 00000100:00100000:2.0:1780298645.709752:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609717376 00000100:00100000:2.0:1780298645.709778:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6506 00000100:00100000:2.0:1780298645.709783:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5dbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2397:x1866777609717376:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298645.709905:0:19950:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:2.0:1780298645.709908:0:19950:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:2.0:1780298645.709909:0:19950:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298646 -> 1780298645 00002000:00010000:2.0:1780298645.709911:0:19950:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:2.0:1780298645.709912:0:19950:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 416 -> 416 00000100:00100000:2.0:1780298645.709934:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5dbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2397:x1866777609717376:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 151us (220us total) trans 4294969792 rc 0/0 00000100:00100000:2.0:1780298645.709939:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6506 00000100:00100000:0.0:1780298645.710658:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.710689:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609717504 00000100:00100000:3.0:1780298645.710693:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609717504 00000100:00100000:3.0:1780298645.710715:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10442 00000100:00100000:3.0:1780298645.710720:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255488380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609717504:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.710725:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.710732:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e630361da lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4fee expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.710754:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605107072:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.710785:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00002000:00010000:3.0:1780298645.711410:0:8356:0:(ofd_lvb.c:271:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime: 1780298645 -> 1780298646 00000100:00100000:3.0:1780298645.711418:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605107072:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.711431:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e630361da lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fee expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.711440:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e630361da lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fee expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.711447:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92600, rc 0) 00000100:00100000:3.0:1780298645.711468:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255488380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609717504:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 748us (811us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.711473:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10442 00000100:00100000:1.0:1780298645.712164:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.712190:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609717632 00000100:00100000:2.0:1780298645.712195:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609717632 00000100:00100000:2.0:1780298645.712219:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10443 00000100:00100000:2.0:1780298645.712226:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609717632:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.712233:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.712241:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e630361e1 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4ff5 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.712290:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255091f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605107200:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.712327:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6cf00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.712906:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255091f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605107200:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.712917:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e630361e1 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4ff5 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.712925:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e630361e1 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4ff5 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.712935:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9ac00, rc 0) 00000100:00100000:2.0:1780298645.712962:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609717632:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 737us (800us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.712969:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10443 00000100:00100000:0.0:1780298645.713847:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.713877:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609717760 00000100:00100000:2.0:1780298645.713881:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609717760 00000100:00100000:2.0:1780298645.713902:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10444 00000100:00100000:2.0:1780298645.713906:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2399:x1866777609717760:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.713912:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.713918:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e0a0f245/0xaf35132e630361e8 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d4ffc expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.713937:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255091f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605107328:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.713971:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6cf00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.714636:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255091f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605107328:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.714647:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e0a0f245/0xaf35132e630361e8 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4ffc expref: 19 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.714656:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e0a0f245/0xaf35132e630361e8 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4ffc expref: 19 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.714662:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b200, rc 0) 00000100:00100000:2.0:1780298645.714683:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2399:x1866777609717760:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 777us (839us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.714689:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10444 00000100:00100000:0.0:1780298645.715621:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.715662:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609717888 00000100:00100000:1.0:1780298645.715666:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609717888 00002000:00100000:1.0:1780298645.715689:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (1945600->1949695) req@ffff9cf244811180 x1866777609717888/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.715696:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244811180 with x1866777609717888 ext(1945600->1949695) 00002000:00010000:1.0:1780298645.715699:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf244811180 00000100:00100000:1.0:1780298645.715715:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6507 00000100:00100000:1.0:1780298645.715721:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244811180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609717888:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298645.715839:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244811180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609717888:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 119us (220us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298645.715847:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6507 00002000:00100000:1.0:1780298645.715851:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x280000401:0x12:0x0] (1945600->1949695) req@ffff9cf244811180 x1866777609717888/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/4536 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.715858:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244811180 with x1866777609717888 ext(1945600->1949695) 00002000:00010000:1.0:1780298645.715861:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf244811180 00000100:00100000:0.0:1780298645.716851:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.716886:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609718016 00000100:00100000:2.0:1780298645.716890:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609718016 00002000:00100000:2.0:1780298645.716906:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x12:0x0] (1949696->1953791) req@ffff9cf244812a00 x1866777609718016/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.716913:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244812a00 with x1866777609718016 ext(1949696->1953791) 00002000:00010000:2.0:1780298645.716916:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf244812a00 00000100:00100000:2.0:1780298645.716937:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6508 00000100:00100000:2.0:1780298645.716945:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244812a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609718016:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298645.717033:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244812a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609718016:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 89us (182us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298645.717040:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6508 00002000:00100000:2.0:1780298645.717046:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x12:0x0] (1949696->1953791) req@ffff9cf244812a00 x1866777609718016/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/4536 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.717054:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244812a00 with x1866777609718016 ext(1949696->1953791) 00002000:00010000:2.0:1780298645.717058:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf244812a00 00000100:00100000:3.0:1780298645.718067:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.718108:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609718144 00000100:00100000:1.0:1780298645.718113:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609718144 00000100:00100000:1.0:1780298645.718153:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10445 00000100:00100000:1.0:1780298645.718160:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5da300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2399:x1866777609718144:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.718167:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.718177:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e630361ef lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5003 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.718209:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605107456:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.718258:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.718923:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605107456:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.718942:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e630361ef lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5003 expref: 19 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.718958:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e630361ef lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5003 expref: 19 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.718967:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279a00, rc 0) 00000100:00100000:1.0:1780298645.718995:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5da300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2399:x1866777609718144:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 836us (929us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.719003:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10445 00000100:00100000:0.0:1780298645.719752:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.719786:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609718272 00000100:00100000:3.0:1780298645.719790:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609718272 00000100:00100000:3.0:1780298645.719812:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10446 00000100:00100000:3.0:1780298645.719817:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244813100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2398:x1866777609718272:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.719823:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.719830:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e630361f6 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d500a expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.719851:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605107584:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.719880:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.720524:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605107584:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.720539:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e630361f6 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d500a expref: 19 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.720552:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e630361f6 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d500a expref: 19 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.720560:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92600, rc 0) 00000100:00100000:0.0:1780298645.720587:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244813100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2398:x1866777609718272:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 769us (836us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.720594:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10446 00000100:00100000:0.0:1780298645.721535:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.721566:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609718400 00000100:00100000:2.0:1780298645.721569:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609718400 00000100:00100000:2.0:1780298645.721600:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10447 00000100:00100000:2.0:1780298645.721607:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244810000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:29360:x1866777609718400:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.721616:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.721624:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e630361fd lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5011 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.721647:0:19992:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e630361be lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1560576->35115007] (req 1560576->1568767) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fd2 expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.721656:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(60 169) 00010000:00010000:2.0:1780298645.721661:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(61 169) 00010000:00010000:2.0:1780298645.721665:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e630361fd lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2834432->2842623] (req 2834432->2842623) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5011 expref: 19 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.721682:0:19992:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e630361be lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1560576->35115007] (req 1560576->1568767) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fd2 expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.721693:0:19992:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e630361be lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1560576->35115007] (req 1560576->1568767) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fd2 expref: 8 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.721701:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255091f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605107712:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298645.721732:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6cf00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.722246:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255091f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605107712:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298645.722262:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e630361fd lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2834432->2842623] (req 2834432->2842623) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5011 expref: 19 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.722286:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9be00, rc 0) 00000100:00100000:1.0:1780298645.722310:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244810000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:29360:x1866777609718400:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 704us (777us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298645.722321:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10447 00000100:00100000:3.0:1780298645.722577:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.722617:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609718528 00000100:00100000:2.0:1780298645.722621:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609718528 00002000:00100000:2.0:1780298645.722640:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (1560576->1568767) req@ffff9cf36e5da680 x1866777609718528/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 8680/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.722647:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e5da680 with x1866777609718528 ext(1560576->1568767) 00010000:00010000:2.0:1780298645.722651:0:16972:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e630361be lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1560576->35115007] (req 1560576->1568767) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fd2 expref: 9 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.722663:0:16972:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e630361be lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1560576->35115007] (req 1560576->1568767) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fd2 expref: 9 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1780298645.722669:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf36e5da680 00000100:00100000:2.0:1780298645.722685:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 219 00000100:00100000:2.0:1780298645.722690:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609718528:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:2.0:1780298645.723120:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609718528:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 430us (542us total) trans 4294969793 rc 0/0 00000100:00100000:2.0:1780298645.723150:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 219 00002000:00100000:2.0:1780298645.723156:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (1560576->1568767) req@ffff9cf36e5da680 x1866777609718528/t4294969793(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 8680/448 e 0 to 0 dl 1780298656 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.723163:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e5da680 with x1866777609718528 ext(1560576->1568767) 00010000:00010000:2.0:1780298645.723167:0:16972:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e630361be lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1560576->35115007] (req 1560576->1568767) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fd2 expref: 11 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.723176:0:16972:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e630361be lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1560576->35115007] (req 1560576->1568767) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fd2 expref: 11 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1780298645.723181:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf36e5da680 00000100:00100000:0.0:1780298645.723834:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.723871:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609718656 00000100:00100000:2.0:1780298645.723875:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609718656 00010000:00010000:2.0:1780298645.723889:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e630361be lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1560576->35115007] (req 1560576->1568767) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fd2 expref: 11 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.723914:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15313 00000100:00100000:2.0:1780298645.723919:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244813800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609718656:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298645.723928:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:2.0:1780298645.723933:0:6462:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 416 -> 432 00010000:00010000:2.0:1780298645.723938:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e630361be lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1560576->35115007] (req 1560576->1568767) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fd2 expref: 11 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.723947:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e630361be lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1560576->35115007] (req 1560576->1568767) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fd2 expref: 11 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.723960:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e630361be lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1560576->35115007] (req 1560576->1568767) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fd2 expref: 11 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.723967:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(61 169) 00010000:00010000:2.0:1780298645.723970:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(61 169) 00010000:00010000:2.0:1780298645.723975:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e630361fd lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2834432->18446744073709551615] (req 2834432->2842623) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5011 expref: 18 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.723982:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e630361fd lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2834432->18446744073709551615] (req 2834432->2842623) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5011 expref: 18 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.723997:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e630361fd lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2834432->18446744073709551615] (req 2834432->2842623) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5011 expref: 18 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.724009:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255091f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777605107840:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298645.724047:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c840 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.724629:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255091f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777605107840:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298645.724637:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298645.724657:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244813800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609718656:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 738us (825us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.724662:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15313 00000100:00100000:3.0:1780298645.724857:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.724882:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609718784 00000100:00100000:0.0:1780298645.724886:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609718784 00002000:00100000:0.0:1780298645.724905:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (2834432->2842623) req@ffff9cf36e5d8000 x1866777609718784/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298645.724911:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e5d8000 with x1866777609718784 ext(2834432->2842623) 00002000:00010000:0.0:1780298645.724914:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf36e5d8000 00000100:00100000:0.0:1780298645.724930:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6509 00000100:00100000:0.0:1780298645.724934:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2399:x1866777609718784:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298645.725029:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2399:x1866777609718784:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 94us (172us total) trans 0 rc 8192/8192 00000100:00100000:0.0:1780298645.725034:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6509 00002000:00100000:0.0:1780298645.725038:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (2834432->2842623) req@ffff9cf36e5d8000 x1866777609718784/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/8632 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 8192/8192 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298645.725044:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e5d8000 with x1866777609718784 ext(2834432->2842623) 00002000:00010000:0.0:1780298645.725046:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf36e5d8000 00000100:00100000:0.0:1780298645.725944:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.725971:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609718912 00000100:00100000:2.0:1780298645.725976:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609718912 00000100:00100000:2.0:1780298645.726005:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10448 00000100:00100000:2.0:1780298645.726014:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244810a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2399:x1866777609718912:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.726023:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.726033:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b4561703/0xaf35132e63036204 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5018 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.726057:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b4561703/0xaf35132e63036204 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5018 expref: 20 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.726070:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b4561703/0xaf35132e63036204 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5018 expref: 20 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.726080:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b600, rc 0) 00000100:00100000:2.0:1780298645.726108:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244810a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2399:x1866777609718912:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 96us (163us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.726116:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10448 00000100:00100000:2.0:1780298645.726777:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.726814:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609719040 00000100:00100000:1.0:1780298645.726820:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609719040 00000100:00100000:1.0:1780298645.726852:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10449 00000100:00100000:1.0:1780298645.726861:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255091f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2400:x1866777609719040:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.726869:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.726878:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e6303620b lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d501f expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.726899:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e6303620b lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d501f expref: 20 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.726913:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e6303620b lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d501f expref: 20 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.726922:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278a00, rc 0) 00000100:00100000:1.0:1780298645.726953:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255091f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2400:x1866777609719040:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 94us (176us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.726961:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10449 00000100:00100000:3.0:1780298645.727658:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.727687:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609719168 00000100:00100000:0.0:1780298645.727692:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609719168 00000100:00100000:0.0:1780298645.727717:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10450 00000100:00100000:0.0:1780298645.727724:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5db100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2399:x1866777609719168:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.727732:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.727740:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e63036212 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5026 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.727763:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e63036212 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5026 expref: 20 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.727775:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e63036212 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5026 expref: 20 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.727785:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3800, rc 0) 00000100:00100000:0.0:1780298645.727808:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5db100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2399:x1866777609719168:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 85us (150us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.727815:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10450 00000100:00100000:0.0:1780298645.728690:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.728721:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609719296 00000100:00100000:2.0:1780298645.728725:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609719296 00002000:00100000:2.0:1780298645.728741:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (4063232->4067327) req@ffff9cf244811c00 x1866777609719296/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.728748:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244811c00 with x1866777609719296 ext(4063232->4067327) 00002000:00010000:2.0:1780298645.728751:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf244811c00 00000100:00100000:2.0:1780298645.728765:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6510 00000100:00100000:2.0:1780298645.728770:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244811c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2397:x1866777609719296:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298645.728857:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244811c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2397:x1866777609719296:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 87us (168us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298645.728862:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6510 00002000:00100000:2.0:1780298645.728865:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (4063232->4067327) req@ffff9cf244811c00 x1866777609719296/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/4536 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.728871:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244811c00 with x1866777609719296 ext(4063232->4067327) 00002000:00010000:2.0:1780298645.728874:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf244811c00 00000100:00100000:2.0:1780298645.729899:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.729940:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609719424 00000100:00100000:1.0:1780298645.729945:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609719424 00002000:00100000:1.0:1780298645.729966:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x2c0000401:0x11:0x0] (4067328->4071423) req@ffff9cf255090700 x1866777609719424/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.729976:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255090700 with x1866777609719424 ext(4067328->4071423) 00002000:00010000:1.0:1780298645.729980:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255090700 00000100:00100000:1.0:1780298645.730001:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6511 00000100:00100000:1.0:1780298645.730009:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255090700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2400:x1866777609719424:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298645.730104:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255090700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2400:x1866777609719424:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 96us (205us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298645.730112:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6511 00002000:00100000:1.0:1780298645.730118:0:19951:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x2c0000401:0x11:0x0] (4067328->4071423) req@ffff9cf255090700 x1866777609719424/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/4536 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.730149:0:19951:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255090700 with x1866777609719424 ext(4067328->4071423) 00002000:00010000:1.0:1780298645.730153:0:19951:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255090700 00000100:00100000:3.0:1780298645.730916:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.730954:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609719552 00000100:00100000:1.0:1780298645.730959:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609719552 00000100:00100000:1.0:1780298645.730990:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10451 00000100:00100000:1.0:1780298645.731000:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609719552:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.731008:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.731018:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63036219 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d502d expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.731040:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63036219 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d502d expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.731054:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63036219 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d502d expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.731065:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278200, rc 0) 00000100:00100000:1.0:1780298645.731092:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609719552:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 95us (175us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.731100:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10451 00000100:00100000:0.0:1780298645.731918:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.731945:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609719680 00000100:00100000:2.0:1780298645.731949:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609719680 00000100:00100000:2.0:1780298645.731971:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10452 00000100:00100000:2.0:1780298645.731976:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244812300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609719680:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.731982:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.731988:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63036220 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5034 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.732001:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63036220 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5034 expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.732010:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63036220 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5034 expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.732016:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9ac00, rc 0) 00000100:00100000:2.0:1780298645.732037:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244812300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609719680:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 61us (120us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.732041:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10452 00000100:00100000:2.0:1780298645.733076:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.733110:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609719808 00000100:00100000:1.0:1780298645.733115:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609719808 00000100:00100000:1.0:1780298645.733159:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10453 00000100:00100000:1.0:1780298645.733167:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255093100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:29360:x1866777609719808:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.733175:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.733184:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e63036227 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d503b expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.733202:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e630361fd lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2834432->18446744073709551615] (req 2834432->2842623) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5011 expref: 18 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.733211:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(61 169) 00010000:00010000:1.0:1780298645.733215:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(62 169) 00010000:00010000:1.0:1780298645.733220:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e63036227 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [5324800->5332991] (req 5324800->5332991) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d503b expref: 11 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.733256:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e630361fd lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2834432->18446744073709551615] (req 2834432->2842623) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5011 expref: 18 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.733281:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e630361fd lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2834432->18446744073709551615] (req 2834432->2842623) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5011 expref: 18 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.733295:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605107968:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298645.733348:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.733871:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605107968:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298645.733888:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e63036227 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [5324800->5332991] (req 5324800->5332991) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d503b expref: 11 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.733899:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278800, rc 0) 00000100:00100000:1.0:1780298645.733922:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255093100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:29360:x1866777609719808:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 757us (848us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298645.733929:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10453 00000100:00100000:0.0:1780298645.734118:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.734160:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609719936 00000100:00100000:1.0:1780298645.734164:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609719936 00010000:00010000:1.0:1780298645.734179:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e630361fd lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2834432->18446744073709551615] (req 2834432->2842623) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5011 expref: 19 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.734202:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15314 00000100:00100000:1.0:1780298645.734208:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b1cc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2400:x1866777609719936:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298645.734217:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298645.734230:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e630361fd lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2834432->18446744073709551615] (req 2834432->2842623) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5011 expref: 19 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.734240:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e630361fd lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2834432->18446744073709551615] (req 2834432->2842623) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5011 expref: 19 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.734262:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e630361fd lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2834432->18446744073709551615] (req 2834432->2842623) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5011 expref: 19 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.734284:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(62 169) 00010000:00010000:1.0:1780298645.734287:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(62 169) 00010000:00010000:1.0:1780298645.734295:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e63036227 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [5324800->38879231] (req 5324800->5332991) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d503b expref: 10 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.734304:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e63036227 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [5324800->38879231] (req 5324800->5332991) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d503b expref: 10 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.734324:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e63036227 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [5324800->38879231] (req 5324800->5332991) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d503b expref: 10 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.734333:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd4380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777605108096:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298645.734365:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.734930:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd4380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777605108096:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298645.734937:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298645.734959:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b1cc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609719936:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 752us (843us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298645.734967:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15314 00000100:00100000:3.0:1780298645.735446:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.735476:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609720064 00000100:00100000:0.0:1780298645.735480:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609720064 00000100:00100000:0.0:1780298645.735506:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10454 00000100:00100000:0.0:1780298645.735511:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5dad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609720064:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.735518:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.735525:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e6303622e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5042 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.735547:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244813b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605108224:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.735580:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba40c0 going to sleep for 11 seconds 00002000:00010000:3.0:1780298645.736080:0:8356:0:(ofd_lvb.c:259:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size: 4194304 -> 5330715 00000100:00100000:3.0:1780298645.736088:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244813b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605108224:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.736101:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e6303622e lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5042 expref: 19 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.736110:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e6303622e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5042 expref: 19 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.736116:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2600, rc 0) 00000100:00100000:3.0:1780298645.736171:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5dad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609720064:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 658us (726us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.736179:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10454 00000100:00100000:2.0:1780298645.736845:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.736874:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609720192 00000100:00100000:1.0:1780298645.736878:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609720192 00000100:00100000:1.0:1780298645.736898:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10455 00000100:00100000:1.0:1780298645.736903:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255092300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609720192:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.736909:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.736916:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e63036235 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5049 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.736936:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605108352:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.736964:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e480 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.737513:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605108352:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.737527:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e63036235 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5049 expref: 19 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.737540:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e63036235 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5049 expref: 19 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.737549:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279e00, rc 0) 00000100:00100000:1.0:1780298645.737575:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255092300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609720192:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 671us (731us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.737581:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10455 00000100:00100000:0.0:1780298645.738398:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.738427:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609720320 00000100:00100000:2.0:1780298645.738431:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609720320 00000100:00100000:2.0:1780298645.738454:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10456 00000100:00100000:2.0:1780298645.738459:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244811f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609720320:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.738464:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.738470:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e6303623c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5050 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.738491:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e56dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605108480:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.738520:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.739172:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e56dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605108480:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.739185:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e6303623c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5050 expref: 12 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.739194:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e6303623c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5050 expref: 12 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.739201:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9a600, rc 0) 00000100:00100000:0.0:1780298645.739230:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244811f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609720320:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 771us (833us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.739236:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10456 00000100:00100000:0.0:1780298645.739982:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.740013:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609720448 00000100:00100000:1.0:1780298645.740018:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609720448 00000100:00100000:1.0:1780298645.740050:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10457 00000100:00100000:1.0:1780298645.740059:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244810700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:29360:x1866777609720448:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.740067:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.740076:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63036243 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5057 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.740097:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(62 169) 00010000:00010000:1.0:1780298645.740101:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(62 169) 00010000:00010000:1.0:1780298645.740108:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63036243 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [4624384->5324799] (req 4624384->4636671) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5057 expref: 12 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.740120:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63036243 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [4624384->5324799] (req 4624384->4636671) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5057 expref: 12 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.740148:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278200, rc 0) 00000100:00100000:1.0:1780298645.740176:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244810700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:29360:x1866777609720448:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 119us (194us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298645.740183:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10457 00000100:00100000:2.0:1780298645.740830:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.740860:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609720576 00000100:00100000:1.0:1780298645.740864:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609720576 00000100:00100000:1.0:1780298645.740892:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6512 00000100:00100000:1.0:1780298645.740898:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609720576:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298645.741065:0:19949:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 5330715 -> 4194304 00002000:00010000:1.0:1780298645.741069:0:19949:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:1.0:1780298645.741072:0:19949:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298646 -> 1780298646 00002000:00010000:1.0:1780298645.741074:0:19949:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:1.0:1780298645.741077:0:19949:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 432 -> 456 00000100:00100000:1.0:1780298645.741103:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609720576:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 204us (273us total) trans 4294969794 rc 0/0 00000100:00100000:1.0:1780298645.741109:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6512 00000100:00100000:2.0:1780298645.741940:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.741970:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609720704 00000100:00100000:0.0:1780298645.741974:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609720704 00000100:00100000:0.0:1780298645.741999:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10458 00000100:00100000:0.0:1780298645.742006:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609720704:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.742014:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.742022:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e6303624a lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d505e expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.742052:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244810e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605108608:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.742086:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba49c0 going to sleep for 11 seconds 00002000:00010000:2.0:1780298645.742756:0:8356:0:(ofd_lvb.c:259:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size: 4194304 -> 5330715 00000100:00100000:2.0:1780298645.742765:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244810e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605108608:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.742778:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e6303624a lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d505e expref: 14 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.742788:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e6303624a lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d505e expref: 14 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.742794:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2000, rc 0) 00000100:00100000:2.0:1780298645.742821:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609720704:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 816us (882us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.742826:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10458 00000100:00100000:2.0:1780298645.743636:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.743666:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609720832 00000100:00100000:1.0:1780298645.743670:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609720832 00000100:00100000:1.0:1780298645.743693:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10459 00000100:00100000:1.0:1780298645.743698:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e426450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2397:x1866777609720832:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.743705:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.743713:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63036251 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5065 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.743742:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605108736:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.743783:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e480 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.744471:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd5880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605108736:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.744482:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63036251 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5065 expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.744492:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63036251 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5065 expref: 14 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.744499:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278a00, rc 0) 00000100:00100000:1.0:1780298645.744523:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e426450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2397:x1866777609720832:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 823us (887us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.744530:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10459 00000100:00100000:0.0:1780298645.745353:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.745388:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609720960 00000100:00100000:2.0:1780298645.745393:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609720960 00000100:00100000:2.0:1780298645.745420:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10460 00000100:00100000:2.0:1780298645.745426:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244812680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:29360:x1866777609720960:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.745433:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.745439:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63036258 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d506c expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.745451:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(62 169) 00010000:00010000:2.0:1780298645.745454:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(62 169) 00010000:00010000:2.0:1780298645.745459:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63036258 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3497984->4624383] (req 3497984->3502079) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d506c expref: 14 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.745470:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63036258 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3497984->4624383] (req 3497984->3502079) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d506c expref: 14 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.745476:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9be00, rc 0) 00000100:00100000:2.0:1780298645.745496:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244812680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:29360:x1866777609720960:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 71us (146us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.745501:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10460 00000100:00100000:2.0:1780298645.746240:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.746290:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609721088 00000100:00100000:3.0:1780298645.746295:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609721088 00002000:00100000:3.0:1780298645.746321:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (3497984->3502079) req@ffff9cf36e56d880 x1866777609721088/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298645.746333:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e56d880 with x1866777609721088 ext(3497984->3502079) 00002000:00010000:3.0:1780298645.746338:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf36e56d880 00000100:00100000:3.0:1780298645.746357:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6513 00000100:00100000:3.0:1780298645.746365:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2399:x1866777609721088:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298645.746516:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2399:x1866777609721088:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 152us (276us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1780298645.746524:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6513 00002000:00100000:3.0:1780298645.746531:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (3497984->3502079) req@ffff9cf36e56d880 x1866777609721088/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/4536 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298645.746540:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e56d880 with x1866777609721088 ext(3497984->3502079) 00002000:00010000:3.0:1780298645.746543:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf36e56d880 00000100:00100000:0.0:1780298645.747586:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.747630:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609721216 00000100:00100000:1.0:1780298645.747634:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609721216 00000100:00100000:1.0:1780298645.747665:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10461 00000100:00100000:1.0:1780298645.747671:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244811880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609721216:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.747684:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.747694:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e6303625f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5073 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.747729:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605108864:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.747772:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e480 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.748522:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605108864:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.748543:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e6303625f lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5073 expref: 19 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.748560:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e6303625f lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5073 expref: 19 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.748570:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279e00, rc 0) 00000100:00100000:1.0:1780298645.748602:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244811880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609721216:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 930us (1018us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.748610:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10461 00000100:00100000:2.0:1780298645.749547:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.749582:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609721344 00000100:00100000:0.0:1780298645.749588:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609721344 00000100:00100000:0.0:1780298645.749623:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10462 00000100:00100000:0.0:1780298645.749631:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2398:x1866777609721344:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.749641:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.749650:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e63036266 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d507a expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.749676:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244813100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605108992:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.749709:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba49c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.750472:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244813100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605108992:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.750493:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e63036266 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d507a expref: 19 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.750508:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e63036266 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d507a expref: 19 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.750518:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2600, rc 0) 00000100:00100000:2.0:1780298645.750550:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2398:x1866777609721344:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 921us (1004us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.750559:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10462 00000100:00100000:3.0:1780298645.751451:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.751490:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609721472 00000100:00100000:1.0:1780298645.751495:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609721472 00000100:00100000:1.0:1780298645.751527:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10463 00000100:00100000:1.0:1780298645.751535:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2399:x1866777609721472:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.751544:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.751553:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e6303626d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5081 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.751592:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605109120:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.751633:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e480 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.752392:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605109120:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.752410:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e6303626d lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5081 expref: 15 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.752426:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e6303626d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5081 expref: 15 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.752436:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278a00, rc 0) 00000100:00100000:1.0:1780298645.752468:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2399:x1866777609721472:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 934us (1017us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.752476:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10463 00000100:00100000:0.0:1780298645.753358:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.753396:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609721600 00000100:00100000:3.0:1780298645.753401:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609721600 00002000:00100000:3.0:1780298645.753430:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x12:0x0] (57344->61439) req@ffff9cf244810000 x1866777609721600/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298645.753437:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244810000 with x1866777609721600 ext(57344->61439) 00002000:00010000:3.0:1780298645.753441:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf244810000 00000100:00100000:3.0:1780298645.753460:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6514 00000100:00100000:3.0:1780298645.753468:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244810000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609721600:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298645.753616:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244810000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609721600:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 148us (258us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1780298645.753622:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6514 00002000:00100000:3.0:1780298645.753628:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x12:0x0] (57344->61439) req@ffff9cf244810000 x1866777609721600/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/4536 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298645.753634:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244810000 with x1866777609721600 ext(57344->61439) 00002000:00010000:3.0:1780298645.753637:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf244810000 00000100:00100000:3.0:1780298645.754735:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.754783:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609721728 00000100:00100000:2.0:1780298645.754787:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609721728 00000100:00100000:2.0:1780298645.754826:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10464 00000100:00100000:2.0:1780298645.754834:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2399:x1866777609721728:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.754844:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.754857:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009fd8d343/0xaf35132e63036274 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5088 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.754900:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e56dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605109248:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.754954:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.755717:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e56dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605109248:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.755736:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009fd8d343/0xaf35132e63036274 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5088 expref: 15 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.755749:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009fd8d343/0xaf35132e63036274 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5088 expref: 15 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.755756:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b000, rc 0) 00000100:00100000:2.0:1780298645.755781:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2399:x1866777609721728:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 949us (1049us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.755786:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10464 00000100:00100000:0.0:1780298645.756597:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.756637:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609721856 00000100:00100000:1.0:1780298645.756642:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609721856 00000100:00100000:1.0:1780298645.756676:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10465 00000100:00100000:1.0:1780298645.756684:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244810a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2399:x1866777609721856:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.756693:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.756702:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e6303627b lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d508f expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.756731:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605109376:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.756772:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e480 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.757365:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605109376:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.757378:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e6303627b lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d508f expref: 15 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.757388:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e6303627b lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d508f expref: 15 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.757394:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279e00, rc 0) 00000100:00100000:1.0:1780298645.757419:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244810a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2399:x1866777609721856:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 736us (822us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.757427:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10465 00000100:00100000:0.0:1780298645.758304:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.758336:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609721984 00000100:00100000:2.0:1780298645.758340:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609721984 00000100:00100000:2.0:1780298645.758368:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10466 00000100:00100000:2.0:1780298645.758377:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244812300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:29360:x1866777609721984:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.758386:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.758396:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e0a0f245/0xaf35132e63036282 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5096 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.758419:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e630361d3 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [389120->1560575] (req 389120->401407) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fe7 expref: 17 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.758429:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(62 169) 00010000:00010000:2.0:1780298645.758433:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(63 169) 00010000:00010000:2.0:1780298645.758437:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e0a0f245/0xaf35132e63036282 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [614400->622591] (req 614400->622591) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5096 expref: 15 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.758458:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e630361d3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [389120->1560575] (req 389120->401407) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fe7 expref: 17 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.758471:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e630361d3 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [389120->1560575] (req 389120->401407) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fe7 expref: 17 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.758481:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e56dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605109504:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298645.758523:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.759122:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e56dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605109504:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298645.759168:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e0a0f245/0xaf35132e63036282 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [614400->622591] (req 614400->622591) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5096 expref: 15 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.759176:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b200, rc 0) 00000100:00100000:2.0:1780298645.759198:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244812300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:29360:x1866777609721984:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 825us (901us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.759203:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10466 00000100:00100000:3.0:1780298645.759370:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.759406:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609722112 00000100:00100000:1.0:1780298645.759409:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609722112 00010000:00010000:1.0:1780298645.759425:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e630361d3 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [389120->1560575] (req 389120->401407) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fe7 expref: 18 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.759445:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15315 00000100:00100000:1.0:1780298645.759449:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5da300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609722112:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298645.759457:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298645.759465:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e630361d3 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [389120->1560575] (req 389120->401407) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fe7 expref: 18 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.759475:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e630361d3 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [389120->1560575] (req 389120->401407) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fe7 expref: 18 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.759489:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e630361d3 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [389120->1560575] (req 389120->401407) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fe7 expref: 18 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.759497:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(63 169) 00010000:00010000:1.0:1780298645.759499:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(63 169) 00010000:00010000:1.0:1780298645.759506:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000e0a0f245/0xaf35132e63036282 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [614400->3497983] (req 614400->622591) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5096 expref: 14 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.759512:0:6463:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e0a0f245/0xaf35132e63036282 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [614400->3497983] (req 614400->622591) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5096 expref: 14 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.759526:0:6463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000e0a0f245/0xaf35132e63036282 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [614400->3497983] (req 614400->622591) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5096 expref: 14 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.759534:0:6463:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd7480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6463:1866777605109632:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298645.759561:0:6463:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e480 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.760287:0:6463:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd7480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6463:1866777605109632:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298645.760295:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298645.760317:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5da300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609722112:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 868us (947us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298645.760322:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15315 00000100:00100000:3.0:1780298645.760528:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.760573:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609722240 00000100:00100000:2.0:1780298645.760577:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609722240 00000100:00100000:2.0:1780298645.760611:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6515 00000100:00100000:2.0:1780298645.760618:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5db100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2400:x1866777609722240:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298645.760718:0:19950:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 5330715 -> 4194304 00002000:00010000:2.0:1780298645.760721:0:19950:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:2.0:1780298645.760722:0:19950:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298646 -> 1780298646 00002000:00010000:2.0:1780298645.760724:0:19950:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:2.0:1780298645.760725:0:19950:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 456 -> 456 00000100:00100000:2.0:1780298645.760748:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5db100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609722240:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 132us (220us total) trans 4294969795 rc 0/0 00000100:00100000:2.0:1780298645.760752:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6515 00000100:00100000:0.0:1780298645.761612:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.761642:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609722368 00000100:00100000:1.0:1780298645.761646:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609722368 00000100:00100000:1.0:1780298645.761669:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10467 00000100:00100000:1.0:1780298645.761675:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244811f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609722368:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.761682:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.761689:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63036289 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d509d expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.761714:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605109760:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.761745:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e480 going to sleep for 11 seconds 00002000:00010000:1.0:1780298645.762453:0:19992:0:(ofd_lvb.c:259:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size: 4194304 -> 5330715 00000100:00100000:1.0:1780298645.762463:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605109760:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.762476:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63036289 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d509d expref: 17 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.762488:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63036289 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d509d expref: 17 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.762494:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278a00, rc 0) 00000100:00100000:1.0:1780298645.762518:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244811f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609722368:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 844us (907us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.762523:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10467 00000100:00100000:3.0:1780298645.763311:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.763340:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609722496 00000100:00100000:2.0:1780298645.763344:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609722496 00000100:00100000:2.0:1780298645.763367:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10468 00000100:00100000:2.0:1780298645.763372:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609722496:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.763379:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.763386:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63036290 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d50a4 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.763407:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e56dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605109888:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.763439:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.764118:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e56dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605109888:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.764171:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63036290 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50a4 expref: 17 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.764187:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63036290 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50a4 expref: 17 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.764194:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9aa00, rc 0) 00000100:00100000:2.0:1780298645.764219:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609722496:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 847us (909us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.764223:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10468 00000100:00100000:0.0:1780298645.765027:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.765057:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609722624 00000100:00100000:1.0:1780298645.765062:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609722624 00000100:00100000:1.0:1780298645.765087:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10469 00000100:00100000:1.0:1780298645.765094:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244810700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609722624:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.765101:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.765108:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e63036297 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d50ab expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.765147:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605110016:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.765182:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e480 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.765851:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605110016:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.765866:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e63036297 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50ab expref: 18 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.765878:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e63036297 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50ab expref: 18 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.765885:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279e00, rc 0) 00000100:00100000:1.0:1780298645.765909:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244810700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609722624:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 816us (883us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.765914:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10469 00000100:00100000:0.0:1780298645.766760:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.766798:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609722752 00000100:00100000:2.0:1780298645.766803:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609722752 00000100:00100000:2.0:1780298645.766831:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10470 00000100:00100000:2.0:1780298645.766835:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244810e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609722752:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.766845:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.766856:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e6303629e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d50b2 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.766877:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000488a7b88/0xaf35132e63036194 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1531904->1535999) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fa8 expref: 6 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.766886:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000a6416acb/0xaf35132e6303619b lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4faf expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.766894:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(59 122) 00010000:00010000:2.0:1780298645.766898:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(59 122) 00010000:00010000:2.0:1780298645.766903:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e6303629e lrc: 5/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2306048->2310143] (req 2306048->2310143) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50b2 expref: 11 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.766926:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000a6416acb/0xaf35132e6303619b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4faf expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.766947:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000a6416acb/0xaf35132e6303619b lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4faf expref: 11 pid: 8359 timeout: 935 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.766957:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e56dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605110144:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298645.766995:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000488a7b88/0xaf35132e63036194 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1531904->1535999) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fa8 expref: 6 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.767003:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000488a7b88/0xaf35132e63036194 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1531904->1535999) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fa8 expref: 6 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.767010:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b69f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605110272:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298645.767026:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.767770:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e56dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605110144:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298645.767780:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b69f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605110272:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298645.767794:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e6303629e lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2306048->2310143] (req 2306048->2310143) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50b2 expref: 11 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.767803:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b000, rc 0) 00000100:00100000:2.0:1780298645.767828:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244810e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:29360:x1866777609722752:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 991us (1068us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.767835:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10470 00000100:00100000:0.0:1780298645.768035:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.768069:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.768093:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609722880 00000100:00100000:2.0:1780298645.768099:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609722880 00010000:00010000:2.0:1780298645.768120:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000a6416acb/0xaf35132e6303619b lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4faf expref: 11 pid: 8359 timeout: 935 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.768146:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609723008 00000100:00100000:3.0:1780298645.768150:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609723008 00010000:00010000:3.0:1780298645.768168:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000488a7b88/0xaf35132e63036194 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1531904->1535999) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fa8 expref: 7 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.768184:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15316 00000100:00100000:2.0:1780298645.768196:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244813b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609722880:12345-192.168.203.25@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1780298645.768197:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15317 00000100:00100000:3.0:1780298645.768209:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244813800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609723008:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298645.768210:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298645.768219:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298645.768230:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000488a7b88/0xaf35132e63036194 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 1531904->1535999) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fa8 expref: 7 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.768230:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000a6416acb/0xaf35132e6303619b lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4faf expref: 11 pid: 8359 timeout: 935 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.768250:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000488a7b88/0xaf35132e63036194 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 1531904->1535999) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fa8 expref: 7 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.768283:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000a6416acb/0xaf35132e6303619b lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4faf expref: 11 pid: 8359 timeout: 935 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.768304:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000488a7b88/0xaf35132e63036194 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 1531904->1535999) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4fa8 expref: 7 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.768311:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6416acb/0xaf35132e6303619b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4faf expref: 11 pid: 8359 timeout: 935 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.768317:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(59 122) 00010000:00010000:3.0:1780298645.768323:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(59 122) 00010000:00010000:3.0:1780298645.768330:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e6303629e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2306048->35860479] (req 2306048->2310143) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50b2 expref: 10 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.768336:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e6303629e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2306048->35860479] (req 2306048->2310143) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50b2 expref: 10 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.768347:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1780298645.768361:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e6303629e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2306048->35860479] (req 2306048->2310143) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50b2 expref: 10 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.768369:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5dad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777605110400:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298645.768381:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244813b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609722880:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 188us (345us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.768390:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15316 00000100:00100000:3.0:1780298645.768414:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856180 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.768997:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5dad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777605110400:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298645.769009:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298645.769034:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244813800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609723008:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 827us (964us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298645.769043:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15317 00000100:00100000:0.0:1780298645.769292:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.769335:0:19951:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609723136 00000100:00100000:1.0:1780298645.769340:0:19951:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609723136 00000100:00100000:1.0:1780298645.769379:0:19951:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6516 00000100:00100000:1.0:1780298645.769388:0:19951:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244813100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609723136:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298645.769555:0:19951:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 3838373 -> 3838373 00002000:00010000:1.0:1780298645.769558:0:19951:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:1.0:1780298645.769560:0:19951:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298646 -> 1780298646 00002000:00010000:1.0:1780298645.769561:0:19951:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:1.0:1780298645.769563:0:19951:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 16 -> 24 00000100:00100000:1.0:1780298645.769590:0:19951:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244813100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609723136:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 204us (299us total) trans 4294969330 rc 0/0 00000100:00100000:1.0:1780298645.769597:0:19951:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6516 00000100:00100000:0.0:1780298645.770657:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.770677:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.770691:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609723264 00000100:00100000:1.0:1780298645.770696:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609723264 00000100:00100000:3.0:1780298645.770699:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609723392 00000100:00100000:3.0:1780298645.770704:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609723392 00000100:00100000:1.0:1780298645.770726:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10471 00000100:00100000:1.0:1780298645.770734:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254918a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2399:x1866777609723264:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298645.770742:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10472 00010000:00010000:1.0:1780298645.770743:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298645.770754:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254919500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609723392:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.770755:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e630362a5 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d50b9 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.770766:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.770775:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b87f9a7e/0xaf35132e630362ac lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d50c0 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.770802:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605110528:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.770803:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5dad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605110656:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.770859:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856180 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.770861:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e480 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.771624:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5dad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605110656:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.771647:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605110528:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.771653:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b87f9a7e/0xaf35132e630362ac lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50c0 expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.771669:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e630362a5 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50b9 expref: 17 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.771674:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b87f9a7e/0xaf35132e630362ac lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50c0 expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.771694:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93600, rc 0) 00010000:00010000:1.0:1780298645.771698:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e630362a5 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50b9 expref: 17 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.771710:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278a00, rc 0) 00000100:00100000:3.0:1780298645.771722:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254919500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609723392:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 971us (1043us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.771728:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10472 00000100:00100000:1.0:1780298645.771739:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254918a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2399:x1866777609723264:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1003us (1080us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.771747:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10471 00000100:00100000:0.0:1780298645.772795:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.772825:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.772850:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609723520 00000100:00100000:1.0:1780298645.772857:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609723520 00000100:00100000:2.0:1780298645.772869:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609723648 00000100:00100000:2.0:1780298645.772875:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609723648 00000100:00100000:1.0:1780298645.772894:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10473 00000100:00100000:1.0:1780298645.772904:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254919c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2397:x1866777609723520:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.772915:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298645.772919:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10474 00010000:00010000:1.0:1780298645.772925:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e630362b3 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d50c7 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.772931:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609723648:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.772946:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.772957:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad103424/0xaf35132e630362ba lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d50ce expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.772968:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605110784:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.772994:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b69f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605110912:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.773017:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e480 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.773027:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.773829:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b69f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605110912:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.773852:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ad103424/0xaf35132e630362ba lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50ce expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.773863:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605110784:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.773871:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad103424/0xaf35132e630362ba lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50ce expref: 7 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.773883:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9aa00, rc 0) 00010000:00010000:1.0:1780298645.773885:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e630362b3 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50c7 expref: 17 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.773902:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e630362b3 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50c7 expref: 17 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.773912:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279e00, rc 0) 00000100:00100000:2.0:1780298645.773918:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2398:x1866777609723648:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 989us (1090us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.773927:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10474 00000100:00100000:1.0:1780298645.773941:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254919c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2397:x1866777609723520:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1038us (1147us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.773949:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10473 00000100:00100000:0.0:1780298645.775038:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.775069:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609723776 00000100:00100000:2.0:1780298645.775074:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609723776 00000100:00100000:2.0:1780298645.775103:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10475 00000100:00100000:2.0:1780298645.775112:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:29360:x1866777609723776:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.775122:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.775152:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c1d9addf/0xaf35132e630362c1 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d50d5 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.775175:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e6303629e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2306048->35860479] (req 2306048->2310143) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50b2 expref: 10 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.775182:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(59 122) 00010000:00010000:2.0:1780298645.775185:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(60 122) 00010000:00010000:2.0:1780298645.775188:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c1d9addf/0xaf35132e630362c1 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2797568->2805759] (req 2797568->2805759) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50d5 expref: 7 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.775204:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e6303629e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2306048->35860479] (req 2306048->2310143) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50b2 expref: 10 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.775223:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e6303629e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2306048->35860479] (req 2306048->2310143) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50b2 expref: 10 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.775232:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b69f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605111040:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298645.775285:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.775945:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b69f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605111040:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298645.775963:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c1d9addf/0xaf35132e630362c1 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2797568->2805759] (req 2797568->2805759) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50d5 expref: 7 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.775974:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9a600, rc 0) 00000100:00100000:3.0:1780298645.776042:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609723776:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 930us (1004us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298645.776051:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10475 00000100:00100000:1.0:1780298645.776095:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.776176:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609723904 00000100:00100000:3.0:1780298645.776179:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609723904 00010000:00010000:3.0:1780298645.776193:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e6303629e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2306048->35860479] (req 2306048->2310143) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50b2 expref: 11 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.776217:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15318 00000100:00100000:3.0:1780298645.776222:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd7480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609723904:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298645.776229:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298645.776238:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e6303629e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2306048->35860479] (req 2306048->2310143) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50b2 expref: 11 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.776247:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e6303629e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2306048->35860479] (req 2306048->2310143) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50b2 expref: 11 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.776264:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e6303629e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2306048->35860479] (req 2306048->2310143) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50b2 expref: 11 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.776285:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(60 122) 00010000:00010000:3.0:1780298645.776289:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(60 122) 00010000:00010000:3.0:1780298645.776298:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c1d9addf/0xaf35132e630362c1 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2797568->2805759) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50d5 expref: 6 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.776307:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c1d9addf/0xaf35132e630362c1 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2797568->2805759) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50d5 expref: 6 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.776333:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000c1d9addf/0xaf35132e630362c1 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2797568->2805759) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50d5 expref: 6 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.776343:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5dad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:7400:1866777605111168:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298645.776390:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856180 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.777055:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5dad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:7400:1866777605111168:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298645.777077:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298645.777108:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd7480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609723904:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 884us (1014us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298645.777120:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15318 00000100:00100000:0.0:1780298645.777417:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.777454:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609724032 00000100:00100000:1.0:1780298645.777464:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609724032 00002000:00100000:1.0:1780298645.777505:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x12:0x0] (2797568->2805759) req@ffff9cf25491b800 x1866777609724032/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.777514:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25491b800 with x1866777609724032 ext(2797568->2805759) 00002000:00010000:1.0:1780298645.777519:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf25491b800 00000100:00100000:1.0:1780298645.777543:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6517 00000100:00100000:1.0:1780298645.777550:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609724032:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298645.777745:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609724032:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 194us (327us total) trans 0 rc 8192/8192 00000100:00100000:1.0:1780298645.777753:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6517 00002000:00100000:1.0:1780298645.777762:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x12:0x0] (2797568->2805759) req@ffff9cf25491b800 x1866777609724032/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/8632 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 8192/8192 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.777771:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25491b800 with x1866777609724032 ext(2797568->2805759) 00002000:00010000:1.0:1780298645.777775:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf25491b800 00000100:00100000:1.0:1780298645.778844:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.778878:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.778889:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609724160 00000100:00100000:2.0:1780298645.778896:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609724160 00000100:00100000:3.0:1780298645.778904:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609724288 00000100:00100000:3.0:1780298645.778909:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609724288 00000100:00100000:2.0:1780298645.778934:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10476 00000100:00100000:3.0:1780298645.778944:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10477 00000100:00100000:2.0:1780298645.778945:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2397:x1866777609724160:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.778959:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298645.778964:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609724288:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.778972:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e630362c8 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d50dc expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.778975:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.778986:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c1a1ebef/0xaf35132e630362cf lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d50e3 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.779003:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(60 122) 00010000:00010000:3.0:1780298645.779007:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(60 122) 00010000:00010000:3.0:1780298645.779014:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c1a1ebef/0xaf35132e630362cf lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50e3 expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.779022:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b69880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605111296:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.779030:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c1a1ebef/0xaf35132e630362cf lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50e3 expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.779043:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93a00, rc 0) 00000100:00100000:2.0:1780298645.779066:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c840 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.779072:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609724288:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 116us (192us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298645.779081:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10477 00000100:00100000:2.0:1780298645.779775:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b69880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605111296:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.779796:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e630362c8 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50dc expref: 18 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.779813:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e630362c8 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50dc expref: 18 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.779823:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9ac00, rc 0) 00000100:00100000:2.0:1780298645.779856:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2397:x1866777609724160:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 915us (1011us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.779865:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10476 00000100:00100000:0.0:1780298645.780779:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.780815:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609724416 00000100:00100000:1.0:1780298645.780821:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609724416 00000100:00100000:1.0:1780298645.780855:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10478 00000100:00100000:1.0:1780298645.780861:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609724416:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.780872:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.780886:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e630362d6 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d50ea expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.780926:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605111424:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.780982:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e900 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.781733:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605111424:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.781751:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e630362d6 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50ea expref: 18 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.781767:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e630362d6 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50ea expref: 18 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.781774:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279a00, rc 0) 00000100:00100000:0.0:1780298645.781803:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609724416:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 943us (1027us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.781808:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10478 00000100:00100000:0.0:1780298645.782913:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.782953:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609724544 00000100:00100000:3.0:1780298645.782959:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609724544 00000100:00100000:3.0:1780298645.783002:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10479 00000100:00100000:3.0:1780298645.783013:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254919880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:29360:x1866777609724544:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.783028:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.783040:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e630362dd lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d50f1 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.783065:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e0a0f245/0xaf35132e63036282 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [614400->3497983] (req 614400->622591) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5096 expref: 15 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.783074:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(63 169) 00010000:00010000:3.0:1780298645.783078:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(64 169) 00010000:00010000:3.0:1780298645.783083:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e630362dd lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [3108864->3112959] (req 3108864->3112959) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50f1 expref: 18 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.783109:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e0a0f245/0xaf35132e63036282 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [614400->3497983] (req 614400->622591) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5096 expref: 15 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.783161:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e0a0f245/0xaf35132e63036282 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [614400->3497983] (req 614400->622591) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5096 expref: 15 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.783176:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5dad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605111552:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298645.783293:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856180 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.784172:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5dad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605111552:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298645.784204:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e630362dd lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [3108864->3112959] (req 3108864->3112959) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50f1 expref: 18 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.784218:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93600, rc 0) 00000100:00100000:0.0:1780298645.784277:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254919880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:29360:x1866777609724544:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1258us (1357us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.784285:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10479 00000100:00100000:1.0:1780298645.784456:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.784499:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609724672 00000100:00100000:2.0:1780298645.784505:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609724672 00010000:00010000:2.0:1780298645.784535:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e0a0f245/0xaf35132e63036282 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [614400->3497983] (req 614400->622591) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5096 expref: 16 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.784574:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15319 00000100:00100000:2.0:1780298645.784583:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2399:x1866777609724672:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298645.784596:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298645.784612:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000e0a0f245/0xaf35132e63036282 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [614400->3497983] (req 614400->622591) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5096 expref: 16 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.784627:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e0a0f245/0xaf35132e63036282 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [614400->3497983] (req 614400->622591) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5096 expref: 16 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.784647:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e0a0f245/0xaf35132e63036282 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [614400->3497983] (req 614400->622591) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5096 expref: 16 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.784659:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(64 169) 00010000:00010000:2.0:1780298645.784663:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(64 169) 00010000:00010000:2.0:1780298645.784672:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e630362dd lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [3108864->3497983] (req 3108864->3112959) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50f1 expref: 17 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.784682:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e630362dd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [3108864->3497983] (req 3108864->3112959) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50f1 expref: 17 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.784710:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e630362dd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [3108864->3497983] (req 3108864->3112959) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50f1 expref: 17 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.784723:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b69880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777605111680:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298645.784768:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.785645:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b69880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777605111680:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298645.785662:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298645.785699:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2399:x1866777609724672:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1118us (1247us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298645.785706:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15319 00000100:00100000:0.0:1780298645.785971:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.786013:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609724800 00000100:00100000:3.0:1780298645.786019:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609724800 00002000:00100000:3.0:1780298645.786098:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (3108864->3112959) req@ffff9cf254918700 x1866777609724800/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298645.786109:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf254918700 with x1866777609724800 ext(3108864->3112959) 00002000:00010000:3.0:1780298645.786113:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf254918700 00000100:00100000:3.0:1780298645.786176:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6518 00000100:00100000:3.0:1780298645.786186:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254918700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609724800:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298645.786369:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254918700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609724800:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 186us (401us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1780298645.786374:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6518 00002000:00100000:3.0:1780298645.786379:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (3108864->3112959) req@ffff9cf254918700 x1866777609724800/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/4536 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298645.786385:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf254918700 with x1866777609724800 ext(3108864->3112959) 00002000:00010000:3.0:1780298645.786388:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf254918700 00000100:00100000:0.0:1780298645.787603:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.787660:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609724928 00000100:00100000:2.0:1780298645.787666:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609724928 00000100:00100000:2.0:1780298645.787702:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10480 00000100:00100000:2.0:1780298645.787712:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254919180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2398:x1866777609724928:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.787724:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.787739:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e630362e4 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d50f8 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.787790:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b6b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605111808:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.787850:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6cf00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.788828:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b6b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605111808:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.788859:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e630362e4 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50f8 expref: 16 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.788879:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e630362e4 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50f8 expref: 16 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.788892:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9aa00, rc 0) 00000100:00100000:0.0:1780298645.788929:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254919180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2398:x1866777609724928:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1219us (1329us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.788940:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10480 00000100:00100000:0.0:1780298645.789930:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.789968:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609725056 00000100:00100000:2.0:1780298645.789975:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609725056 00000100:00100000:2.0:1780298645.790018:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10481 00000100:00100000:2.0:1780298645.790027:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2400:x1866777609725056:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.790038:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.790050:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e630362eb lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d50ff expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.790089:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b68700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605111936:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.790164:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6cf00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.790894:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b68700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605111936:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.790910:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e630362eb lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50ff expref: 16 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.790922:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e630362eb lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50ff expref: 16 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.790932:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9ac00, rc 0) 00000100:00100000:2.0:1780298645.790963:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2400:x1866777609725056:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 938us (1034us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.790972:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10481 00000100:00100000:0.0:1780298645.792554:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.792603:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609725184 00000100:00100000:3.0:1780298645.792609:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609725184 00002000:00100000:3.0:1780298645.792649:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (3424256->3432447) req@ffff9cf25491bb80 x1866777609725184/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298645.792657:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25491bb80 with x1866777609725184 ext(3424256->3432447) 00002000:00010000:3.0:1780298645.792662:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf25491bb80 00000100:00100000:3.0:1780298645.792683:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6519 00000100:00100000:3.0:1780298645.792689:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609725184:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298645.792891:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609725184:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 200us (342us total) trans 0 rc 8192/8192 00000100:00100000:3.0:1780298645.792898:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6519 00002000:00100000:3.0:1780298645.792904:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (3424256->3432447) req@ffff9cf25491bb80 x1866777609725184/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/8632 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 8192/8192 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298645.792910:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25491bb80 with x1866777609725184 ext(3424256->3432447) 00002000:00010000:3.0:1780298645.792912:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf25491bb80 00000100:00100000:0.0:1780298645.794538:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.794600:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609725312 00000100:00100000:1.0:1780298645.794606:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609725312 00000100:00100000:1.0:1780298645.794647:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10482 00000100:00100000:1.0:1780298645.794656:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25491aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609725312:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.794672:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.794688:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e630362f2 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5106 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.794738:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605112064:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.794798:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e900 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.795738:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd4380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605112064:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.795760:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e630362f2 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5106 expref: 19 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.795775:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e630362f2 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5106 expref: 19 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.795784:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278400, rc 0) 00000100:00100000:0.0:1780298645.795817:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25491aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609725312:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1158us (1279us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.795825:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10482 00000100:00100000:0.0:1780298645.796865:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.796903:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609725440 00000100:00100000:2.0:1780298645.796910:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609725440 00000100:00100000:2.0:1780298645.796941:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10483 00000100:00100000:2.0:1780298645.796949:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243808700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2398:x1866777609725440:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.796959:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.796968:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009fd8d343/0xaf35132e630362f9 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d510d expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.796999:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b68700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605112192:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.797051:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6cf00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.797871:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b68700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605112192:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.797887:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009fd8d343/0xaf35132e630362f9 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d510d expref: 19 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.797902:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009fd8d343/0xaf35132e630362f9 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d510d expref: 19 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.797909:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b000, rc 0) 00000100:00100000:0.0:1780298645.797934:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243808700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2398:x1866777609725440:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 987us (1071us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.797940:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10483 00000100:00100000:1.0:1780298645.798891:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.798929:0:9976:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609725696 00000100:00100000:0.0:1780298645.798935:0:9976:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609725696 00000100:00100000:0.0:1780298645.798983:0:9976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62091 00000100:00100000:0.0:1780298645.798990:0:9976:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+42:29360:x1866777609725696:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:0.0:1780298645.799031:0:9976:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298645.799046:0:9976:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9bc00/0xaf35132e6303617f lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 23 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f93 expref: 42 pid: 17567 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.799059:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298645.799063:0:9976:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1780298645.799254:0:9976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3a00/0xaf35132e63036300 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 22 type: IBT flags: 0x40000000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.799278:0:9976:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3c00/0xaf35132e63036155 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 22 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f70 expref: 49 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.799290:0:9976:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3a00/0xaf35132e63036300 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 22 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.799319:0:9976:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3c00/0xaf35132e63036155 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 22 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f70 expref: 49 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.799333:0:9976:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3c00/0xaf35132e63036155 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 22 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f70 expref: 49 pid: 6477 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.799346:0:9976:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243809880 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:9976:1866777605112320:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298645.799391:0:9976:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4f00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.800025:0:9976:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243809880 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:9976:1866777605112320:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298645.800040:0:9976:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3a00/0xaf35132e63036300 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 22 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.800437:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.800482:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609725824 00000100:00100000:2.0:1780298645.800488:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609725824 00010000:00010000:2.0:1780298645.800518:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3c00/0xaf35132e63036155 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 22 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f70 expref: 50 pid: 6477 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.800548:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15320 00000100:00100000:2.0:1780298645.800558:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242861180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+50:2399:x1866777609725824:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298645.800567:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298645.800572:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3c00/0xaf35132e63036155 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 23 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f70 expref: 50 pid: 6477 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.800584:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3c00/0xaf35132e63036155 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 23 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f70 expref: 50 pid: 6477 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.800599:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3c00/0xaf35132e63036155 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 23 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d4f70 expref: 50 pid: 6477 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.800613:0:6461:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298645.800617:0:6461:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3a00/0xaf35132e63036300 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 22 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.800626:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3a00/0xaf35132e63036300 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 22 type: IBT flags: 0x40290000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.800633:0:6461:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3a00/0xaf35132e63036300 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 22 type: IBT flags: 0x40290000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.800653:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1780298645.800666:0:9976:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3a00/0xaf35132e63036300 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 21 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.800674:0:9976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3a00/0xaf35132e63036300 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 21 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.800684:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242861180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+49:2399:x1866777609725824:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 127us (245us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.800693:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15320 00000001:00080000:0.0:1780298645.800734:0:9976:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991130 00000004:00010000:0.0:1780298645.800761:0:9976:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000d6b28ec9 reply state 00000000cfabeacf transno 4294991131 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3a00/0xaf35132e63036300 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 21 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.800774:0:9976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3a00/0xaf35132e63036300 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 21 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.800781:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298645.800784:0:9976:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1780298645.800787:0:9976:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1780298645.800848:0:9976:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+44:29360:x1866777609725696:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1857us (1956us total) trans 4294991131 rc 0/0 00000100:00100000:0.0:1780298645.800855:0:9976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62091 00000020:00080000:0.0:1780298645.800860:0:9976:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1780298645.801621:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.801660:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609725952 00000100:00100000:0.0:1780298645.801665:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609725952 00000100:00100000:0.0:1780298645.801703:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10484 00000100:00100000:0.0:1780298645.801711:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242861f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:29360:x1866777609725952:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.801721:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.801733:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e63036307 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5114 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.801754:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e630362dd lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [3108864->3497983] (req 3108864->3112959) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50f1 expref: 19 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.801763:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63036243 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [4624384->5324799] (req 4624384->4636671) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5057 expref: 14 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.801771:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e63036227 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [5324800->38879231] (req 5324800->5332991) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d503b expref: 14 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.801779:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63036258 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [3497984->4624383] (req 3497984->3502079) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d506c expref: 14 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.801787:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 4 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(64 169) 00010000:00010000:0.0:1780298645.801791:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(68 169) 00010000:00010000:0.0:1780298645.801796:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e63036307 lrc: 7/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5114 expref: 19 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.801818:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63036258 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [3497984->4624383] (req 3497984->3502079) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d506c expref: 14 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.801830:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63036258 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [3497984->4624383] (req 3497984->3502079) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d506c expref: 14 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.801842:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243809880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605112448:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298645.801897:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e63036227 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [5324800->38879231] (req 5324800->5332991) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d503b expref: 14 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.801907:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e63036227 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [5324800->38879231] (req 5324800->5332991) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d503b expref: 14 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.801916:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243808380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605112576:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298645.801947:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63036243 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [4624384->5324799] (req 4624384->4636671) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5057 expref: 14 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.801956:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63036243 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [4624384->5324799] (req 4624384->4636671) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5057 expref: 14 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.801965:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24380a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605112704:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298645.802069:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e630362dd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [3108864->3497983] (req 3108864->3112959) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50f1 expref: 19 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.802085:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e630362dd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [3108864->3497983] (req 3108864->3112959) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50f1 expref: 19 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.802096:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243808a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605112832:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298645.802196:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba40c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.803007:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243809880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605112448:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298645.803024:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243808380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605112576:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298645.803037:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24380a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605112704:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298645.803060:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243808a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605112832:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298645.803078:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:0.0:1780298645.803093:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e63036307 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5114 expref: 19 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.803110:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3800, rc 0) 00000100:00100000:1.0:1780298645.803140:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609726336 00000100:00100000:1.0:1780298645.803146:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609726336 00000100:00100000:0.0:1780298645.803163:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242861f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:29360:x1866777609725952:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1453us (1542us total) trans 0 rc 0/0 00010000:00010000:1.0:1780298645.803172:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63036243 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [4624384->5324799] (req 4624384->4636671) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5057 expref: 15 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.803173:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10484 00000100:00100000:1.0:1780298645.803201:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15321 00000100:00100000:1.0:1780298645.803208:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b1fc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2397:x1866777609726336:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298645.803216:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298645.803230:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63036243 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [4624384->5324799] (req 4624384->4636671) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5057 expref: 15 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.803239:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63036243 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [4624384->5324799] (req 4624384->4636671) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5057 expref: 15 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.803253:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:1.0:1780298645.803259:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63036243 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [4624384->5324799] (req 4624384->4636671) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5057 expref: 15 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.803279:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298645.803294:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.803298:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609726208 00000100:00100000:1.0:1780298645.803304:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b1fc50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2397:x1866777609726336:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 95us (228us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.803305:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609726208 00000100:00100000:1.0:1780298645.803312:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15321 00010000:00010000:2.0:1780298645.803330:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63036258 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3497984->4624383] (req 3497984->3502079) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d506c expref: 14 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.803357:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15322 00000100:00100000:1.0:1780298645.803364:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243808a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609726208:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298645.803371:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:0.0:1780298645.803376:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:1.0:1780298645.803380:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63036258 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [3497984->4624383] (req 3497984->3502079) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d506c expref: 14 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.803388:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63036258 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [3497984->4624383] (req 3497984->3502079) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d506c expref: 14 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.803400:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63036258 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [3497984->4624383] (req 3497984->3502079) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d506c expref: 14 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.803401:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609726080 00000100:00100000:2.0:1780298645.803406:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609726080 00010000:00010000:1.0:1780298645.803411:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298645.803435:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243808a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609726208:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 72us (184us total) trans 0 rc 0/0 00002000:00100000:2.0:1780298645.803440:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (5324800->5330714) req@ffff9cf36e5d8a80 x1866777609726080/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 6408/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00000100:00100000:3.0:1780298645.803446:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609726464 00000100:00100000:1.0:1780298645.803446:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15322 00000100:00100000:3.0:1780298645.803453:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609726464 00002000:00100000:2.0:1780298645.803459:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e5d8a80 with x1866777609726080 ext(5324800->5330714) 00010000:00010000:2.0:1780298645.803467:0:8363:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e63036227 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [5324800->38879231] (req 5324800->5332991) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d503b expref: 13 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00100000:3.0:1780298645.803480:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (3108864->3112959) req@ffff9cf24380a680 x1866777609726464/t0(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 4584/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00010000:00010000:2.0:1780298645.803489:0:8363:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e63036227 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [5324800->38879231] (req 5324800->5332991) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d503b expref: 13 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1780298645.803498:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf36e5d8a80 00000100:00100000:2.0:1780298645.803524:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 220 00000100:00100000:2.0:1780298645.803533:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609726080:12345-192.168.203.25@tcp:4:fsx.0 00002000:00100000:3.0:1780298645.803557:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24380a680 with x1866777609726464 ext(3108864->3112959) 00010000:00010000:3.0:1780298645.803564:0:19950:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e630362dd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3108864->3497983] (req 3108864->3112959) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50f1 expref: 19 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.803575:0:19950:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e630362dd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3108864->3497983] (req 3108864->3112959) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50f1 expref: 19 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1780298645.803582:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf24380a680 00000100:00100000:0.0:1780298645.803630:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 221 00000100:00100000:0.0:1780298645.803638:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24380a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2400:x1866777609726464:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:0.0:1780298645.804113:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24380a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2400:x1866777609726464:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 475us (739us total) trans 4294969796 rc 0/0 00000100:00100000:2.0:1780298645.804115:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2400:x1866777609726080:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 582us (820us total) trans 4294969797 rc 0/0 00000100:00100000:0.0:1780298645.804145:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 221 00002000:00100000:0.0:1780298645.804152:0:8364:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x2c0000401:0x11:0x0] (3108864->3112959) req@ffff9cf24380a680 x1866777609726464/t4294969796(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 4584/448 e 0 to 0 dl 1780298656 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00000100:00100000:2.0:1780298645.804155:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 220 00002000:00100000:0.0:1780298645.804163:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24380a680 with x1866777609726464 ext(3108864->3112959) 00002000:00100000:2.0:1780298645.804168:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (5324800->5330714) req@ffff9cf36e5d8a80 x1866777609726080/t4294969797(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 6408/448 e 0 to 0 dl 1780298656 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00010000:00010000:0.0:1780298645.804170:0:8364:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e630362dd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3108864->3497983] (req 3108864->3112959) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50f1 expref: 21 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00100000:2.0:1780298645.804185:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e5d8a80 with x1866777609726080 ext(5324800->5330714) 00010000:00010000:2.0:1780298645.804194:0:8363:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e63036227 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [5324800->38879231] (req 5324800->5332991) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d503b expref: 15 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.804195:0:8364:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e630362dd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3108864->3497983] (req 3108864->3112959) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50f1 expref: 21 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1780298645.804216:0:8364:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf24380a680 00010000:00010000:2.0:1780298645.804224:0:8363:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e63036227 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [5324800->38879231] (req 5324800->5332991) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d503b expref: 15 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1780298645.804232:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf36e5d8a80 00000100:00100000:1.0:1780298645.805043:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.805074:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.805076:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609726592 00000100:00100000:3.0:1780298645.805082:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609726592 00010000:00010000:3.0:1780298645.805102:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e630362dd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3108864->3497983] (req 3108864->3112959) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50f1 expref: 21 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.805111:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609726720 00000100:00100000:2.0:1780298645.805116:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609726720 00010000:00010000:2.0:1780298645.805153:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e63036227 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [5324800->38879231] (req 5324800->5332991) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d503b expref: 15 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.805159:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15323 00000100:00100000:3.0:1780298645.805172:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242861180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2400:x1866777609726592:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298645.805180:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:3.0:1780298645.805187:0:6462:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 456 -> 480 00010000:00010000:3.0:1780298645.805192:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e630362dd lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3108864->3497983] (req 3108864->3112959) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50f1 expref: 21 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.805198:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15324 00000100:00100000:2.0:1780298645.805211:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242862d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2398:x1866777609726720:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298645.805213:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e630362dd lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3108864->3497983] (req 3108864->3112959) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50f1 expref: 21 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.805223:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298645.805229:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e630362dd lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [3108864->3497983] (req 3108864->3112959) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50f1 expref: 21 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.805236:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e63036227 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [5324800->38879231] (req 5324800->5332991) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d503b expref: 15 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.805240:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298645.805251:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e63036227 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [5324800->38879231] (req 5324800->5332991) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d503b expref: 15 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.805266:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e63036227 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [5324800->38879231] (req 5324800->5332991) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d503b expref: 15 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.805280:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242861180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2400:x1866777609726592:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 101us (225us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298645.805294:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15323 00010000:00010000:2.0:1780298645.805301:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(68 169) 00010000:00010000:2.0:1780298645.805306:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(68 169) 00010000:00010000:2.0:1780298645.805314:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e63036307 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5114 expref: 19 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.805324:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e63036307 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5114 expref: 19 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.805351:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e63036307 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5114 expref: 19 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.805363:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b68380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777605112960:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298645.805412:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c840 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.806091:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b68380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777605112960:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298645.806107:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298645.806208:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242862d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2398:x1866777609726720:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 999us (1130us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.806218:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15324 00000100:00100000:0.0:1780298645.806296:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.806335:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609726848 00000100:00100000:2.0:1780298645.806340:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609726848 00000100:00100000:2.0:1780298645.806377:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10485 00000100:00100000:2.0:1780298645.806386:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243808380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:29360:x1866777609726848:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.806398:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.806412:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e6303630e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d511b expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.806433:0:19992:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000c1d9addf/0xaf35132e630362c1 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2797568->2805759) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50d5 expref: 6 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.806443:0:19992:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000c1a1ebef/0xaf35132e630362cf lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50e3 expref: 12 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.806449:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(60 122) 00010000:00010000:2.0:1780298645.806453:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(60 122) 00010000:00010000:2.0:1780298645.806457:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e6303630e lrc: 5/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d511b expref: 12 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.806476:0:19992:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000c1a1ebef/0xaf35132e630362cf lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50e3 expref: 12 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.806490:0:19992:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000c1a1ebef/0xaf35132e630362cf lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50e3 expref: 12 pid: 9128 timeout: 935 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.806511:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b68380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605113088:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298645.806554:0:19992:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000c1d9addf/0xaf35132e630362c1 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2797568->2805759) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50d5 expref: 6 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.806567:0:19992:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000c1d9addf/0xaf35132e630362c1 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2797568->2805759) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50d5 expref: 6 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.806578:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b6a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605113216:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298645.806602:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c840 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.807242:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b68380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605113088:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298645.807251:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b6a300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605113216:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298645.807266:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e6303630e lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d511b expref: 12 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.807293:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9ac00, rc 0) 00000100:00100000:2.0:1780298645.807319:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243808380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:29360:x1866777609726848:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 936us (1025us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.807326:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10485 00000100:00100000:1.0:1780298645.807577:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.807610:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.807621:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609726976 00000100:00100000:2.0:1780298645.807626:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609726976 00000100:00100000:3.0:1780298645.807637:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609727104 00000100:00100000:3.0:1780298645.807641:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609727104 00010000:00010000:2.0:1780298645.807642:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000c1a1ebef/0xaf35132e630362cf lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50e3 expref: 12 pid: 9128 timeout: 935 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.807656:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000c1d9addf/0xaf35132e630362c1 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2797568->2805759) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50d5 expref: 7 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.807671:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15325 00000100:00100000:2.0:1780298645.807680:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242863b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2399:x1866777609726976:12345-192.168.203.25@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1780298645.807685:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15326 00010000:00010000:2.0:1780298645.807688:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1780298645.807696:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242860000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609727104:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298645.807703:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000c1a1ebef/0xaf35132e630362cf lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50e3 expref: 12 pid: 9128 timeout: 935 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.807706:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298645.807716:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000c1a1ebef/0xaf35132e630362cf lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50e3 expref: 12 pid: 9128 timeout: 935 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.807729:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000c1d9addf/0xaf35132e630362c1 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2797568->2805759) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50d5 expref: 7 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.807735:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c1a1ebef/0xaf35132e630362cf lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50e3 expref: 12 pid: 9128 timeout: 935 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.807750:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000c1d9addf/0xaf35132e630362c1 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 2797568->2805759) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50d5 expref: 7 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.807762:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1780298645.807769:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c1d9addf/0xaf35132e630362c1 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 2797568->2805759) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d50d5 expref: 7 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.807778:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(60 122) 00010000:00010000:3.0:1780298645.807780:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(60 122) 00010000:00010000:3.0:1780298645.807784:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e6303630e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d511b expref: 11 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.807788:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242863b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609726976:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 109us (210us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.807801:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15325 00010000:00010000:3.0:1780298645.807802:0:6463:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e6303630e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d511b expref: 11 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.807829:0:6463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e6303630e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d511b expref: 10 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.807841:0:6463:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d9f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777605113344:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298645.807873:0:6463:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.808656:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.808670:0:6463:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d9f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777605113344:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298645.808695:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:3.0:1780298645.808735:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298645.808743:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609727232 00000100:00100000:0.0:1780298645.808749:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609727232 00000100:00100000:2.0:1780298645.808753:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609727360 00000100:00100000:2.0:1780298645.808759:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609727360 00000100:00100000:3.0:1780298645.808771:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242860000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609727104:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1077us (1158us total) trans 0 rc 0/0 00002000:00010000:0.0:1780298645.808776:0:19943:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00010000:2.0:1780298645.808777:0:19949:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (144846->18446744073709551615) 00000100:00100000:3.0:1780298645.808782:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15326 00002000:00100000:0.0:1780298645.808783:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244813b80 with x1866777609727232 ext(4194304->18446744073709551615) 00002000:00100000:2.0:1780298645.808784:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244813800 with x1866777609727360 ext(144846->18446744073709551615) 00002000:00010000:2.0:1780298645.808796:0:19949:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf244813800. 00002000:00010000:0.0:1780298645.808796:0:19943:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf244813b80. 00000100:00100000:2.0:1780298645.808823:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6520 00000100:00100000:3.0:1780298645.808925:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6521 00000100:00100000:2.0:1780298645.808929:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244813800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609727360:12345-192.168.203.25@tcp:10:fsx.0 00000100:00100000:3.0:1780298645.808940:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244813b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2397:x1866777609727232:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:3.0:1780298645.809233:0:16972:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 5330715 -> 4194304 00002000:00010000:3.0:1780298645.809237:0:16972:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:3.0:1780298645.809239:0:16972:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298646 -> 1780298646 00002000:00010000:3.0:1780298645.809241:0:16972:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:3.0:1780298645.809242:0:16972:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 480 -> 440 00000100:00100000:3.0:1780298645.809284:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244813b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2397:x1866777609727232:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 349us (629us total) trans 4294969798 rc 0/0 00002000:00010000:2.0:1780298645.809291:0:19949:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 3838373 -> 144846 00000100:00100000:3.0:1780298645.809295:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6521 00002000:00010000:2.0:1780298645.809297:0:19949:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:3.0:1780298645.809300:0:16972:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00010000:2.0:1780298645.809301:0:19949:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298646 -> 1780298646 00002000:00100000:3.0:1780298645.809306:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244813b80 with x1866777609727232 ext(4194304->18446744073709551615) 00002000:00010000:2.0:1780298645.809307:0:19949:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:2.0:1780298645.809313:0:19949:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 24 -> 0 00002000:00010000:3.0:1780298645.809314:0:16972:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf244813b80. 00000100:00100000:2.0:1780298645.809343:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244813800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609727360:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 418us (646us total) trans 4294969331 rc 0/0 00000100:00100000:2.0:1780298645.809352:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6520 00002000:00010000:2.0:1780298645.809356:0:19949:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (144846->18446744073709551615) 00002000:00100000:2.0:1780298645.809360:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244813800 with x1866777609727360 ext(144846->18446744073709551615) 00002000:00010000:2.0:1780298645.809365:0:19949:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf244813800. 00000100:00100000:1.0:1780298645.810114:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.810165:0:17578:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609727488 00000100:00100000:3.0:1780298645.810169:0:17578:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609727488 00000100:00100000:3.0:1780298645.810206:0:17578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62092 00000100:00100000:3.0:1780298645.810215:0:17578:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+49:29360:x1866777609727488:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.810227:0:17578:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.810242:0:17578:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93600/0xaf35132e63036315 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 22 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5122 expref: -99 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.810333:0:17578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93c00/0xaf35132e6303631c lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 23 type: IBT flags: 0x40000000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.810349:0:17578:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93c00/0xaf35132e6303631c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 23 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.810359:0:17578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93c00/0xaf35132e6303631c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 23 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1780298645.810389:0:17578:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 4339150 00000004:00200000:3.0:1780298645.810392:0:17578:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1780298645.810414:0:17578:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93c00/0xaf35132e6303631c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 23 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.810432:0:17578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93600/0xaf35132e63036315 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 23 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5122 expref: 51 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.810441:0:17578:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93c00/0xaf35132e6303631c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 22 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5122 expref: 50 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.810447:0:17578:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298645.810449:0:17578:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1780298645.810451:0:17578:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93c00, rc 0) 00000100:00100000:3.0:1780298645.810473:0:17578:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+50:29360:x1866777609727488:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 262us (362us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298645.810477:0:17578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62092 00000100:00100000:0.0:1780298645.811310:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.811357:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609727616 00000100:00100000:1.0:1780298645.811362:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609727616 00000100:00100000:0.0:1780298645.811372:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.811401:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609727744 00000100:00100000:3.0:1780298645.811406:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609727744 00000100:00100000:1.0:1780298645.811407:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10486 00000100:00100000:1.0:1780298645.811419:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243809880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2398:x1866777609727616:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.811428:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.811438:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63036323 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5129 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.811475:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10487 00000100:00100000:1.0:1780298645.811484:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf347cd6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605113472:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.811485:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243808000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609727744:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.811498:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.811508:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f8ed852/0xaf35132e6303632a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5130 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.811538:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605113600:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.811547:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.811569:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.812278:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605113600:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.812278:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf347cd6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605113472:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.812306:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63036323 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5129 expref: 15 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.812312:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002f8ed852/0xaf35132e6303632a lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5130 expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.812332:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f8ed852/0xaf35132e6303632a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5130 expref: 7 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.812335:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63036323 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5129 expref: 15 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.812342:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92c00, rc 0) 00010000:00010000:0.0:1780298645.812354:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278c00, rc 0) 00000100:00100000:3.0:1780298645.812371:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243808000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2397:x1866777609727744:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 890us (1001us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.812377:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10487 00000100:00100000:0.0:1780298645.812384:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243809880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2398:x1866777609727616:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 966us (1075us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.812392:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10486 00000100:00100000:2.0:1780298645.813503:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.813541:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609727872 00000100:00100000:0.0:1780298645.813546:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609727872 00000100:00100000:2.0:1780298645.813553:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.813580:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609728000 00000100:00100000:0.0:1780298645.813582:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609728000 00000100:00100000:2.0:1780298645.813596:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10488 00000100:00100000:0.0:1780298645.813604:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10489 00000100:00100000:2.0:1780298645.813609:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b6a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609727872:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298645.813616:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b68380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609728000:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.813621:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.813626:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.813632:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e63036331 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5137 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.813635:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5a492f0/0xaf35132e63036338 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d513e expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.813684:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b68700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605113728:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.813688:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243809180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605113856:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.813733:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c840 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.813739:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba40c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.814499:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243809180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605113856:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.814502:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b68700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605113728:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.814525:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e63036331 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5137 expref: 15 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.814530:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a5a492f0/0xaf35132e63036338 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d513e expref: 7 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.814547:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e63036331 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5137 expref: 15 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.814551:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5a492f0/0xaf35132e63036338 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d513e expref: 7 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.814563:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9a200, rc 0) 00010000:00010000:0.0:1780298645.814568:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3600, rc 0) 00000100:00100000:2.0:1780298645.814598:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b6a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609727872:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 993us (1094us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.814599:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b68380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2400:x1866777609728000:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 987us (1045us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.814610:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10488 00000100:00100000:0.0:1780298645.814620:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10489 00000100:00100000:1.0:1780298645.815704:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.815740:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609728128 00000100:00100000:2.0:1780298645.815746:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609728128 00000100:00100000:1.0:1780298645.815756:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.815780:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609728256 00000100:00100000:2.0:1780298645.815784:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609728256 00000100:00100000:3.0:1780298645.815792:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10490 00000100:00100000:2.0:1780298645.815811:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10491 00000100:00100000:2.0:1780298645.815818:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609728256:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.815828:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.815838:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0a0f245/0xaf35132e6303633f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d514c expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.815844:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347cd6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609728128:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.815854:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.815862:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000cd0af3fe/0xaf35132e63036346 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5145 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.815870:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b68700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605113984:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.815884:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605114112:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.815912:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.815912:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c840 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.816594:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b68700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605113984:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.816611:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e0a0f245/0xaf35132e6303633f lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d514c expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.816626:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0a0f245/0xaf35132e6303633f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d514c expref: 7 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.816636:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b200, rc 0) 00000100:00100000:2.0:1780298645.816673:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f746300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:2399:x1866777609728256:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 855us (917us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.816680:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10491 00000100:00100000:3.0:1780298645.816805:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605114112:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.816818:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000cd0af3fe/0xaf35132e63036346 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5145 expref: 15 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.816830:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000cd0af3fe/0xaf35132e63036346 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5145 expref: 15 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.816837:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93400, rc 0) 00000100:00100000:3.0:1780298645.816858:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347cd6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609728128:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1059us (1158us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.816862:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10490 00000100:00100000:0.0:1780298645.817675:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.817697:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609728384 00000100:00100000:3.0:1780298645.817702:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609728384 00000100:00100000:3.0:1780298645.817729:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10492 00000100:00100000:3.0:1780298645.817737:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243809180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+6:29360:x1866777609728384:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.817745:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.817754:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e6303634d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5153 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.817769:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e6303630e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d511b expref: 11 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.817776:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(60 122) 00010000:00010000:3.0:1780298645.817779:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(61 122) 00010000:00010000:3.0:1780298645.817782:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e6303634d lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1384448->1392639] (req 1384448->1392639) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5153 expref: 7 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.817797:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e6303630e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d511b expref: 11 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.817807:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e6303630e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d511b expref: 11 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.817815:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605114240:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298645.817846:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.818425:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605114240:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298645.818453:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e6303634d lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1384448->1392639] (req 1384448->1392639) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5153 expref: 7 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.818462:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92a00, rc 0) 00000100:00100000:3.0:1780298645.818528:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243809180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609728384:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 750us (811us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298645.818535:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10492 00000100:00100000:0.0:1780298645.818569:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.818617:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609728512 00000100:00100000:3.0:1780298645.818620:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609728512 00010000:00010000:3.0:1780298645.818633:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e6303630e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d511b expref: 12 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.818657:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15327 00000100:00100000:3.0:1780298645.818664:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243809c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2399:x1866777609728512:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298645.818674:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298645.818685:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e6303630e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d511b expref: 12 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.818698:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e6303630e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d511b expref: 12 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.818712:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e6303630e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d511b expref: 12 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.818719:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(61 122) 00010000:00010000:3.0:1780298645.818721:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(61 122) 00010000:00010000:3.0:1780298645.818727:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e6303634d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1384448->34938879] (req 1384448->1392639) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5153 expref: 6 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.818733:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e6303634d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1384448->34938879] (req 1384448->1392639) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5153 expref: 6 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.818749:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e6303634d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1384448->34938879] (req 1384448->1392639) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5153 expref: 6 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.818755:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d9f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777605114368:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298645.818781:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.819410:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d9f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777605114368:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298645.819437:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298645.819505:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243809c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609728512:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 840us (935us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298645.819512:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15327 00000100:00100000:0.0:1780298645.819549:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.819572:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609728640 00000100:00100000:3.0:1780298645.819576:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609728640 00000100:00100000:3.0:1780298645.819682:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6522 00000100:00100000:3.0:1780298645.819688:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24380b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609728640:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:3.0:1780298645.819844:0:19928:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 144846 -> 144846 00002000:00010000:3.0:1780298645.819847:0:19928:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:3.0:1780298645.819849:0:19928:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298646 -> 1780298646 00002000:00010000:3.0:1780298645.819851:0:19928:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:3.0:1780298645.819852:0:19928:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 0 -> 16 00000100:00100000:3.0:1780298645.819873:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24380b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609728640:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 186us (326us total) trans 4294969332 rc 0/0 00000100:00100000:3.0:1780298645.819877:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6522 00000100:00100000:1.0:1780298645.820837:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.820872:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609728768 00000100:00100000:0.0:1780298645.820878:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609728768 00000100:00100000:3.0:1780298645.820902:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.820937:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609728896 00000100:00100000:0.0:1780298645.820941:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609728896 00000100:00100000:1.0:1780298645.820948:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10493 00000100:00100000:1.0:1780298645.820961:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f747100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2399:x1866777609728768:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298645.820970:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10494 00010000:00010000:1.0:1780298645.820971:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298645.820982:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609728896:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.820984:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e63036354 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d515a expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.820990:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.821000:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c703984a/0xaf35132e6303635b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5161 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.821031:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605114624:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.821040:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f745f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605114496:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.821069:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba40c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.821079:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.821751:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605114624:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.821771:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c703984a/0xaf35132e6303635b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5161 expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.821789:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c703984a/0xaf35132e6303635b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5161 expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.821801:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2000, rc 0) 00000100:00100000:1.0:1780298645.821814:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f745f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605114496:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.821828:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e63036354 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d515a expref: 15 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.821835:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609728896:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 854us (931us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.821844:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10494 00010000:00010000:1.0:1780298645.821847:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e63036354 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d515a expref: 15 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.821855:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278400, rc 0) 00000100:00100000:1.0:1780298645.821873:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f747100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2399:x1866777609728768:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 917us (1039us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.821877:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10493 00000100:00100000:1.0:1780298645.822964:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.822982:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.822991:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609729024 00000100:00100000:3.0:1780298645.822997:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609729024 00000100:00100000:3.0:1780298645.823030:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609729152 00000100:00100000:3.0:1780298645.823034:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609729152 00000100:00100000:2.0:1780298645.823040:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10495 00000100:00100000:2.0:1780298645.823052:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2398:x1866777609729024:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.823062:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298645.823063:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10496 00010000:00010000:2.0:1780298645.823074:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e63036362 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5168 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.823075:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609729152:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.823088:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.823099:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd0af3fe/0xaf35132e63036369 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d516f expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.823113:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b68700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605114752:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.823149:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605114880:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.823205:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.823206:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.823948:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b68700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605114752:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.823980:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e63036362 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5168 expref: 15 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.823983:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605114880:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.824002:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e63036362 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5168 expref: 15 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.824011:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9a200, rc 0) 00000100:00100000:1.0:1780298645.824044:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2398:x1866777609729024:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 996us (1079us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.824055:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10495 00010000:00010000:3.0:1780298645.824069:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000cd0af3fe/0xaf35132e63036369 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d516f expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.824084:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd0af3fe/0xaf35132e63036369 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d516f expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.824095:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93400, rc 0) 00000100:00100000:3.0:1780298645.824198:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f745500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609729152:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1080us (1168us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.824206:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10496 00000100:00100000:2.0:1780298645.825484:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.825515:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609729280 00000100:00100000:3.0:1780298645.825519:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609729280 00000100:00100000:3.0:1780298645.825555:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6523 00000100:00100000:3.0:1780298645.825560:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b6b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2400:x1866777609729280:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:3.0:1780298645.825709:0:19950:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:3.0:1780298645.825713:0:19950:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:3.0:1780298645.825715:0:19950:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298646 -> 1780298646 00002000:00010000:3.0:1780298645.825718:0:19950:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:3.0:1780298645.825720:0:19950:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 440 -> 440 00000100:00100000:3.0:1780298645.825754:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b6b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:2400:x1866777609729280:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 192us (270us total) trans 4294969799 rc 0/0 00000100:00100000:3.0:1780298645.825762:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6523 00000100:00100000:1.0:1780298645.826646:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.826693:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609729408 00000100:00100000:3.0:1780298645.826697:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609729408 00000100:00100000:3.0:1780298645.826731:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10497 00000100:00100000:3.0:1780298645.826739:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609729408:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.826749:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298645.826750:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:3.0:1780298645.826761:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002f8ed852/0xaf35132e63036370 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5176 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.826784:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609729536 00000100:00100000:1.0:1780298645.826789:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609729536 00000100:00100000:1.0:1780298645.826825:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10498 00000100:00100000:3.0:1780298645.826837:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605115008:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.826838:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b69880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609729536:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.826849:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.826859:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63036377 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d517d expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.826893:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255489500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605115136:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.826899:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.826940:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.827740:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605115008:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.827765:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255489500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605115136:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.827771:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002f8ed852/0xaf35132e63036370 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5176 expref: 15 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.827783:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63036377 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d517d expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.827792:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002f8ed852/0xaf35132e63036370 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5176 expref: 15 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.827797:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63036377 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d517d expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.827808:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92c00, rc 0) 00010000:00010000:0.0:1780298645.827820:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278a00, rc 0) 00000100:00100000:3.0:1780298645.827843:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609729408:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1105us (1197us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.827850:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b69880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609729536:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1015us (1099us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.827856:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10497 00000100:00100000:0.0:1780298645.827864:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10498 00000100:00100000:2.0:1780298645.828813:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.828839:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.828853:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609729664 00000100:00100000:0.0:1780298645.828859:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609729664 00000100:00100000:1.0:1780298645.828877:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609729792 00000100:00100000:1.0:1780298645.828882:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609729792 00000100:00100000:0.0:1780298645.828904:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10499 00000100:00100000:0.0:1780298645.828914:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b69f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2398:x1866777609729664:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298645.828917:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10500 00010000:00010000:0.0:1780298645.828924:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298645.828925:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b68700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609729792:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.828932:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.828933:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a5a492f0/0xaf35132e6303637e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5184 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.828938:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad103424/0xaf35132e63036385 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d518b expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.828963:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e56dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605115264:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.828963:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605115392:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.829003:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.829003:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba40c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.829762:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605115392:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.829779:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e56dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605115264:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.829780:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a5a492f0/0xaf35132e6303637e lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5184 expref: 15 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.829798:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a5a492f0/0xaf35132e6303637e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5184 expref: 15 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.829802:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ad103424/0xaf35132e63036385 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d518b expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.829823:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3600, rc 0) 00010000:00010000:3.0:1780298645.829824:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad103424/0xaf35132e63036385 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d518b expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.829836:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9aa00, rc 0) 00000100:00100000:0.0:1780298645.829862:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b69f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2398:x1866777609729664:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 950us (1048us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.829867:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b68700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609729792:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 942us (1025us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.829877:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10499 00000100:00100000:3.0:1780298645.829884:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10500 00000100:00100000:1.0:1780298645.831097:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.831171:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609729920 00000100:00100000:2.0:1780298645.831175:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609729920 00002000:00100000:2.0:1780298645.831205:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (3588096->3596287) req@ffff9cf25548ad80 x1866777609729920/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.831216:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25548ad80 with x1866777609729920 ext(3588096->3596287) 00002000:00010000:2.0:1780298645.831221:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf25548ad80 00000100:00100000:2.0:1780298645.831239:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6524 00000100:00100000:2.0:1780298645.831246:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:2399:x1866777609729920:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298645.831422:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:2399:x1866777609729920:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 176us (326us total) trans 0 rc 8192/8192 00000100:00100000:2.0:1780298645.831430:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6524 00002000:00100000:2.0:1780298645.831437:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (3588096->3596287) req@ffff9cf25548ad80 x1866777609729920/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/8632 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 8192/8192 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.831446:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25548ad80 with x1866777609729920 ext(3588096->3596287) 00002000:00010000:2.0:1780298645.831449:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf25548ad80 00000100:00100000:2.0:1780298645.832379:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.832400:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.832416:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609730048 00000100:00100000:1.0:1780298645.832422:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609730048 00000100:00100000:3.0:1780298645.832443:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609730176 00000100:00100000:3.0:1780298645.832449:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609730176 00000100:00100000:1.0:1780298645.832453:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10501 00000100:00100000:1.0:1780298645.832462:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e56dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2399:x1866777609730048:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.832469:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.832476:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e6303638c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5192 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.832486:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10502 00000100:00100000:3.0:1780298645.832495:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255491500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609730176:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.832503:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298645.832513:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255488380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605115520:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.832514:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b87f9a7e/0xaf35132e63036393 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5199 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.832548:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605115648:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.832564:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e540 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.832578:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.833309:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255488380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605115520:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.833316:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605115648:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.833325:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e6303638c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5192 expref: 15 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.833338:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e6303638c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5192 expref: 15 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.833339:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b87f9a7e/0xaf35132e63036393 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5199 expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.833366:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279a00, rc 0) 00010000:00010000:3.0:1780298645.833372:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b87f9a7e/0xaf35132e63036393 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5199 expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.833385:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93600, rc 0) 00000100:00100000:2.0:1780298645.833391:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e56dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2399:x1866777609730048:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 931us (1012us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.833396:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10501 00000100:00100000:3.0:1780298645.833412:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255491500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609730176:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 918us (1008us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.833420:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10502 00000100:00100000:2.0:1780298645.834357:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.834375:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.834391:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609730304 00000100:00100000:1.0:1780298645.834395:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609730304 00000100:00100000:0.0:1780298645.834410:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609730432 00000100:00100000:0.0:1780298645.834416:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609730432 00000100:00100000:1.0:1780298645.834426:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10503 00000100:00100000:1.0:1780298645.834437:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255493800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2398:x1866777609730304:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298645.834440:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10504 00010000:00010000:1.0:1780298645.834448:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298645.834454:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255493480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609730432:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.834458:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e6303639a lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d51a0 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.834464:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.834476:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd0af3fe/0xaf35132e630363a1 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d51a7 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.834486:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25548aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605115776:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.834513:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605115904:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.834523:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e540 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.834546:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.835202:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25548aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605115776:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.835218:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e6303639a lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d51a0 expref: 15 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.835233:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e6303639a lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d51a0 expref: 15 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.835243:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279e00, rc 0) 00000100:00100000:2.0:1780298645.835284:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255493800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2398:x1866777609730304:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 851us (926us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.835292:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10503 00000100:00100000:2.0:1780298645.835365:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605115904:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.835386:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000cd0af3fe/0xaf35132e630363a1 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d51a7 expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.835399:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd0af3fe/0xaf35132e630363a1 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d51a7 expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.835422:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93400, rc 0) 00000100:00100000:2.0:1780298645.835452:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255493480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609730432:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1002us (1074us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.835459:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10504 00000100:00100000:1.0:1780298645.836342:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.836367:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609730560 00000100:00100000:3.0:1780298645.836372:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609730560 00000100:00100000:3.0:1780298645.836410:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6525 00000100:00100000:3.0:1780298645.836419:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:2400:x1866777609730560:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:3.0:1780298645.836571:0:16972:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:3.0:1780298645.836575:0:16972:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:3.0:1780298645.836577:0:16972:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298646 -> 1780298646 00002000:00010000:3.0:1780298645.836579:0:16972:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:3.0:1780298645.836581:0:16972:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 440 -> 440 00000100:00100000:3.0:1780298645.836610:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2400:x1866777609730560:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 193us (269us total) trans 4294969800 rc 0/0 00000100:00100000:3.0:1780298645.836618:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6525 00000100:00100000:1.0:1780298645.837419:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.837466:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609730688 00000100:00100000:0.0:1780298645.837470:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609730688 00000100:00100000:1.0:1780298645.837475:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.837497:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609730816 00000100:00100000:0.0:1780298645.837500:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609730816 00000100:00100000:2.0:1780298645.837515:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10505 00000100:00100000:0.0:1780298645.837522:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10506 00000100:00100000:2.0:1780298645.837528:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255488380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2399:x1866777609730688:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298645.837534:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e426050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609730816:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.837539:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.837541:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.837550:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e630363a8 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d51ae expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.837551:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5a492f0/0xaf35132e630363af lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d51b5 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.837593:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605116032:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.837600:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255492300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605116160:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.837631:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba40c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.837634:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c840 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.838299:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255492300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605116160:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.838320:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e630363a8 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d51ae expref: 15 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.838337:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e630363a8 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d51ae expref: 15 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.838349:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9aa00, rc 0) 00000100:00100000:2.0:1780298645.838380:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255488380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2399:x1866777609730688:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 857us (961us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.838389:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10505 00000100:00100000:0.0:1780298645.838404:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605116032:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.838419:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a5a492f0/0xaf35132e630363af lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d51b5 expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.838432:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5a492f0/0xaf35132e630363af lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d51b5 expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.838439:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3600, rc 0) 00000100:00100000:0.0:1780298645.838461:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e426050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609730816:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 934us (988us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.838466:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10506 00000100:00100000:1.0:1780298645.839298:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.839330:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609730944 00000100:00100000:2.0:1780298645.839335:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609730944 00000100:00100000:2.0:1780298645.839368:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10507 00000100:00100000:2.0:1780298645.839377:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255489500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2398:x1866777609730944:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.839386:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.839394:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e630363b6 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d51bc expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.839422:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255492300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605116288:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.839441:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.839466:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609731072 00000100:00100000:2.0:1780298645.839476:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c840 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.839502:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609731072 00000100:00100000:1.0:1780298645.839539:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10508 00000100:00100000:1.0:1780298645.839547:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25548a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609731072:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.839553:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.839560:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017dab243/0xaf35132e630363bd lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d51c3 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.839580:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255488380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605116416:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.839605:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.840028:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255492300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605116288:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.840039:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e630363b6 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d51bc expref: 15 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.840049:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e630363b6 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d51bc expref: 15 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.840055:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9a600, rc 0) 00000100:00100000:2.0:1780298645.840079:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255489500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2398:x1866777609730944:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 705us (781us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.840084:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10507 00000100:00100000:1.0:1780298645.840171:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255488380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605116416:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.840186:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000017dab243/0xaf35132e630363bd lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d51c3 expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.840200:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017dab243/0xaf35132e630363bd lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d51c3 expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.840207:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279800, rc 0) 00000100:00100000:1.0:1780298645.840224:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25548a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609731072:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 679us (783us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.840231:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10508 00000100:00100000:1.0:1780298645.841209:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.841233:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609731200 00000100:00100000:3.0:1780298645.841237:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609731200 00000100:00100000:3.0:1780298645.841307:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10509 00000100:00100000:3.0:1780298645.841313:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255488380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2397:x1866777609731200:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.841320:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.841327:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e630363c4 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d51ca expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.841334:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.841364:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609731328 00000100:00100000:0.0:1780298645.841368:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609731328 00000100:00100000:3.0:1780298645.841385:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5dad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605116544:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.841393:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10510 00000100:00100000:0.0:1780298645.841402:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255090a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609731328:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.841408:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.841415:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0cae770/0xaf35132e630363cb lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d51d1 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.841426:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856000 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.841438:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605116672:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.841471:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba40c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.842194:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5dad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605116544:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.842211:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e630363c4 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d51ca expref: 15 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.842222:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e630363c4 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d51ca expref: 15 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.842226:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605116672:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.842233:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93600, rc 0) 00010000:00010000:0.0:1780298645.842245:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e0cae770/0xaf35132e630363cb lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d51d1 expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.842257:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0cae770/0xaf35132e630363cb lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d51d1 expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.842261:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255488380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2397:x1866777609731200:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 947us (1051us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.842283:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10509 00010000:00010000:0.0:1780298645.842294:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2600, rc 0) 00000100:00100000:0.0:1780298645.842314:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255090a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609731328:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 915us (984us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.842319:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10510 00000100:00100000:1.0:1780298645.843285:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.843304:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609731456 00000100:00100000:3.0:1780298645.843308:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609731456 00000100:00100000:3.0:1780298645.843328:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10511 00000100:00100000:3.0:1780298645.843333:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255093b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2398:x1866777609731456:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.843338:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.843344:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002f8ed852/0xaf35132e630363d2 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d51d8 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.843423:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5dad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605116800:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.843459:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.843462:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856000 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.843490:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609731584 00000100:00100000:0.0:1780298645.843494:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609731584 00000100:00100000:0.0:1780298645.843522:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10512 00000100:00100000:0.0:1780298645.843528:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255091f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609731584:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.843534:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.843540:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000101214ae/0xaf35132e630363d9 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d51df expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.843559:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605116928:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.843585:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba40c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.844150:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5dad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605116800:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.844170:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002f8ed852/0xaf35132e630363d2 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d51d8 expref: 15 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.844181:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002f8ed852/0xaf35132e630363d2 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d51d8 expref: 15 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.844188:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92c00, rc 0) 00000100:00100000:3.0:1780298645.844214:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255093b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2398:x1866777609731456:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 881us (929us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.844220:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10511 00000100:00100000:0.0:1780298645.844237:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605116928:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.844253:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000101214ae/0xaf35132e630363d9 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d51df expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.844264:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000101214ae/0xaf35132e630363d9 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d51df expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.844283:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3200, rc 0) 00000100:00100000:0.0:1780298645.844306:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255091f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609731584:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 779us (850us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.844312:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10512 00000100:00100000:2.0:1780298645.846198:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.846229:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609731712 00000100:00100000:1.0:1780298645.846234:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609731712 00000100:00100000:1.0:1780298645.846258:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10513 00000100:00100000:1.0:1780298645.846264:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255492300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:29360:x1866777609731712:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.846286:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.846294:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e630363e0 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d51e6 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.846308:0:19992:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e63036307 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5114 expref: 22 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.846315:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(68 169) 00010000:00010000:1.0:1780298645.846318:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(69 169) 00010000:00010000:1.0:1780298645.846322:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e630363e0 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [4403200->4415487] (req 4403200->4415487) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d51e6 expref: 15 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.846338:0:19992:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e63036307 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5114 expref: 22 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.846348:0:19992:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e63036307 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5114 expref: 22 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.846355:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255093100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605117056:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298645.846387:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.847004:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255093100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605117056:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298645.847015:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e630363e0 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [4403200->4415487] (req 4403200->4415487) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d51e6 expref: 15 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.847022:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279800, rc 0) 00000100:00100000:1.0:1780298645.847043:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255492300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:29360:x1866777609731712:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 780us (846us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298645.847048:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10513 00000100:00100000:3.0:1780298645.847238:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.847278:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609731840 00000100:00100000:0.0:1780298645.847282:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609731840 00010000:00010000:0.0:1780298645.847302:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e63036307 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5114 expref: 23 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.847325:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15328 00000100:00100000:0.0:1780298645.847331:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b1e850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2399:x1866777609731840:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298645.847338:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298645.847347:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e63036307 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5114 expref: 23 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.847357:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e63036307 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5114 expref: 23 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.847369:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e63036307 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5114 expref: 23 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.847377:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(69 169) 00010000:00010000:0.0:1780298645.847379:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(69 169) 00010000:00010000:0.0:1780298645.847385:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e630363e0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [4403200->37957631] (req 4403200->4415487) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d51e6 expref: 14 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.847390:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e630363e0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [4403200->37957631] (req 4403200->4415487) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d51e6 expref: 14 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.847408:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e630363e0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [4403200->37957631] (req 4403200->4415487) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d51e6 expref: 14 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.847414:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777605117184:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298645.847443:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba40c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.848015:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777605117184:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298645.848023:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298645.848041:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b1e850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:2399:x1866777609731840:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 711us (803us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.848047:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15328 00000100:00100000:2.0:1780298645.848417:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.848449:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609731968 00000100:00100000:0.0:1780298645.848452:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609731968 00000100:00100000:0.0:1780298645.848481:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62093 00000100:00100000:0.0:1780298645.848488:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255491f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+44:29360:x1866777609731968:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.848502:0:6477:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.848514:0:6477:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3600/0xaf35132e630363e7 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 23 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d51ed expref: -99 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.848602:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2e00/0xaf35132e630363ee lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 24 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.848614:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2e00/0xaf35132e630363ee lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 24 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.848621:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2e00/0xaf35132e630363ee lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 24 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1780298645.848641:0:6477:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 4339150 00000004:00200000:0.0:1780298645.848645:0:6477:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1780298645.848662:0:6477:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2e00/0xaf35132e630363ee lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 24 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.848678:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3600/0xaf35132e630363e7 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 24 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d51ed expref: 46 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.848688:0:6477:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2e00/0xaf35132e630363ee lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 23 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d51ed expref: 45 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.848693:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298645.848696:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1780298645.848699:0:6477:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2e00, rc 0) 00000100:00100000:0.0:1780298645.848730:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255491f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+45:29360:x1866777609731968:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 243us (313us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.848737:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62093 00000100:00100000:3.0:1780298645.849507:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.849536:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609732096 00000100:00100000:3.0:1780298645.849540:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.849541:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609732096 00000100:00100000:0.0:1780298645.849567:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609732224 00000100:00100000:0.0:1780298645.849569:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609732224 00000100:00100000:0.0:1780298645.849587:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10514 00000100:00100000:0.0:1780298645.849596:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5dad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:2399:x1866777609732096:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.849602:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298645.849604:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10515 00010000:00010000:0.0:1780298645.849609:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e630363f5 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d51f4 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.849621:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609732224:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.849634:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298645.849640:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605117312:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.849644:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e630363fc lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d51fb expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.849675:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4300 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.849678:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255492a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605117440:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.849711:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c840 going to sleep for 11 seconds 00002000:00010000:0.0:1780298645.850398:0:8356:0:(ofd_lvb.c:259:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size: 4194304 -> 4412060 00000100:00100000:0.0:1780298645.850407:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605117312:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.850421:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e630363f5 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d51f4 expref: 23 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.850432:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e630363f5 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d51f4 expref: 23 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.850436:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255492a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605117440:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.850442:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3c00, rc 0) 00010000:00010000:3.0:1780298645.850461:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e630363fc lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d51fb expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.850469:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5dad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:2399:x1866777609732096:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 876us (962us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.850480:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10514 00010000:00010000:3.0:1780298645.850487:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e630363fc lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d51fb expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.850504:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9ac00, rc 0) 00000100:00100000:3.0:1780298645.850529:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609732224:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 915us (988us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.850536:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10515 00000100:00100000:1.0:1780298645.851521:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.851554:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609732352 00000100:00100000:0.0:1780298645.851560:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609732352 00000100:00100000:0.0:1780298645.851587:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10516 00000100:00100000:0.0:1780298645.851594:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255093100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:2397:x1866777609732352:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.851602:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.851619:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e63036403 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5202 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.851647:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605117568:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.851671:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.851690:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4300 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.851702:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609732480 00000100:00100000:3.0:1780298645.851707:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609732480 00000100:00100000:0.0:1780298645.851755:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10517 00000100:00100000:0.0:1780298645.851761:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255490380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609732480:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.851767:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.851773:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0cae770/0xaf35132e6303640a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5209 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.851791:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605117696:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.851815:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba49c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.852344:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605117568:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.852356:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e63036403 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5202 expref: 23 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.852366:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e63036403 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5202 expref: 23 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.852373:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3200, rc 0) 00000100:00100000:0.0:1780298645.852397:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255093100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:2397:x1866777609732352:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 805us (876us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.852402:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10516 00000100:00100000:0.0:1780298645.852517:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605117696:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.852527:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e0cae770/0xaf35132e6303640a lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5209 expref: 12 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.852539:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0cae770/0xaf35132e6303640a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5209 expref: 12 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.852544:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2600, rc 0) 00000100:00100000:0.0:1780298645.852563:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255490380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609732480:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 803us (896us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.852567:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10517 00000100:00100000:1.0:1780298645.853388:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.853425:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609732608 00000100:00100000:2.0:1780298645.853429:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609732608 00000100:00100000:2.0:1780298645.853463:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6526 00000100:00100000:2.0:1780298645.853472:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255092300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609732608:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298645.853697:0:19949:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 144846 -> 1601008 00002000:00010000:2.0:1780298645.853701:0:19949:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:2.0:1780298645.853703:0:19949:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298646 -> 1780298646 00002000:00010000:2.0:1780298645.853705:0:19949:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:2.0:1780298645.853706:0:19949:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 16 -> 32 00000100:00100000:2.0:1780298645.853730:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255092300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609732608:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 260us (342us total) trans 4294969333 rc 0/0 00000100:00100000:2.0:1780298645.853734:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6526 00000100:00100000:1.0:1780298645.854574:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.854605:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.854619:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609732736 00000100:00100000:3.0:1780298645.854624:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609732736 00000100:00100000:0.0:1780298645.854632:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609732864 00000100:00100000:0.0:1780298645.854636:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609732864 00000100:00100000:3.0:1780298645.854660:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10518 00000100:00100000:3.0:1780298645.854673:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243062300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:2400:x1866777609732736:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298645.854673:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10519 00010000:00010000:3.0:1780298645.854684:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298645.854689:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243060000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609732864:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.854694:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002f8ed852/0xaf35132e63036411 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5210 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.854698:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.854707:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000488a7b88/0xaf35132e63036418 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5217 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.854729:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605117824:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.854739:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605117952:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.854780:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856000 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.854780:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba49c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.855426:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605117824:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.855434:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605117952:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.855449:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000488a7b88/0xaf35132e63036418 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5217 expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.855459:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002f8ed852/0xaf35132e63036411 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5210 expref: 23 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.855460:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000488a7b88/0xaf35132e63036418 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5217 expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.855483:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002f8ed852/0xaf35132e63036411 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5210 expref: 23 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.855490:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2200, rc 0) 00010000:00010000:3.0:1780298645.855493:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92c00, rc 0) 00000100:00100000:3.0:1780298645.855521:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243062300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:2400:x1866777609732736:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 851us (944us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.855521:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243060000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609732864:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 834us (911us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.855534:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10518 00000100:00100000:0.0:1780298645.855544:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10519 00000100:00100000:1.0:1780298645.856336:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.856356:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609732992 00000100:00100000:3.0:1780298645.856396:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609732992 00000100:00100000:3.0:1780298645.856418:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10520 00000100:00100000:3.0:1780298645.856425:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243063480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:2399:x1866777609732992:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.856431:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.856438:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e6303641f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d521e expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.856445:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.856474:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609733120 00000100:00100000:0.0:1780298645.856478:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609733120 00000100:00100000:0.0:1780298645.856505:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10521 00000100:00100000:3.0:1780298645.856509:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605118080:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.856514:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243061500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609733120:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.856523:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.856531:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a5a492f0/0xaf35132e63036426 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5225 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.856560:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605118208:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.856562:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856000 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.856589:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba49c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.857041:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605118080:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.857059:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e6303641f lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d521e expref: 23 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.857071:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e6303641f lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d521e expref: 23 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.857080:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93600, rc 0) 00000100:00100000:3.0:1780298645.857182:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243063480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:2399:x1866777609732992:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 758us (845us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.857188:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10520 00000100:00100000:0.0:1780298645.857252:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605118208:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.857277:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a5a492f0/0xaf35132e63036426 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5225 expref: 12 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.857290:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a5a492f0/0xaf35132e63036426 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5225 expref: 12 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.857300:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3600, rc 0) 00000100:00100000:0.0:1780298645.857323:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243061500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609733120:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 813us (882us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.857330:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10521 00000100:00100000:2.0:1780298645.858166:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.858186:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609733248 00000100:00100000:3.0:1780298645.858190:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609733248 00000100:00100000:3.0:1780298645.858211:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10522 00000100:00100000:3.0:1780298645.858215:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255492a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:29360:x1866777609733248:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.858221:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.858227:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000cd0af3fe/0xaf35132e6303642d lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d522c expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.858240:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(69 169) 00010000:00010000:3.0:1780298645.858243:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(69 169) 00010000:00010000:3.0:1780298645.858249:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000cd0af3fe/0xaf35132e6303642d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [454656->4403199] (req 454656->466943) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d522c expref: 23 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.858258:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000cd0af3fe/0xaf35132e6303642d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [454656->4403199] (req 454656->466943) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d522c expref: 23 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.858264:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93400, rc 0) 00000100:00100000:3.0:1780298645.858298:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255492a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:29360:x1866777609733248:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 82us (132us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298645.858303:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10522 00000100:00100000:1.0:1780298645.859118:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.859161:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609733376 00000100:00100000:3.0:1780298645.859168:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609733376 00002000:00100000:2.0:1780298645.859214:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (454656->458751) req@ffff9cf243061f80 x1866777609733376/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.859222:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243061f80 with x1866777609733376 ext(454656->458751) 00002000:00010000:2.0:1780298645.859227:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf243061f80 00000100:00100000:2.0:1780298645.859249:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6527 00000100:00100000:2.0:1780298645.859259:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243061f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2399:x1866777609733376:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298645.859399:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243061f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2399:x1866777609733376:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 144us (281us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298645.859404:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6527 00002000:00100000:2.0:1780298645.859408:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (454656->458751) req@ffff9cf243061f80 x1866777609733376/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/4536 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.859414:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243061f80 with x1866777609733376 ext(454656->458751) 00002000:00010000:2.0:1780298645.859417:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf243061f80 00000100:00100000:1.0:1780298645.860225:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.860245:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609733504 00000100:00100000:3.0:1780298645.860249:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609733504 00002000:00100000:3.0:1780298645.860264:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (462848->466943) req@ffff9cf24304fb80 x1866777609733504/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298645.860282:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24304fb80 with x1866777609733504 ext(462848->466943) 00002000:00010000:3.0:1780298645.860285:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24304fb80 00000100:00100000:3.0:1780298645.860301:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6528 00000100:00100000:3.0:1780298645.860307:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2397:x1866777609733504:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298645.860368:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2397:x1866777609733504:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 62us (145us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1780298645.860373:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6528 00002000:00100000:3.0:1780298645.860378:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (462848->466943) req@ffff9cf24304fb80 x1866777609733504/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/4536 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298645.860383:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24304fb80 with x1866777609733504 ext(462848->466943) 00002000:00010000:3.0:1780298645.860387:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24304fb80 00000100:00100000:1.0:1780298645.861296:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.861322:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609733632 00000100:00100000:0.0:1780298645.861327:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609733632 00000100:00100000:1.0:1780298645.861328:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.861351:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609733760 00000100:00100000:0.0:1780298645.861353:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609733760 00000100:00100000:0.0:1780298645.861371:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10523 00000100:00100000:0.0:1780298645.861377:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2398:x1866777609733632:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.861383:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298645.861384:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10524 00010000:00010000:0.0:1780298645.861391:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e63036434 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5233 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.861398:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609733760:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.861407:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.861416:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e6303643b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d523a expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.861420:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605118336:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.861453:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255491500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605118464:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.861462:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba49c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.861485:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6cf00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.862199:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605118336:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.862215:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e63036434 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5233 expref: 24 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.862225:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e63036434 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5233 expref: 24 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.862232:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3c00, rc 0) 00000100:00100000:2.0:1780298645.862251:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255491500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605118464:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.862256:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2398:x1866777609733632:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 878us (960us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.862262:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10523 00010000:00010000:2.0:1780298645.862281:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e6303643b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d523a expref: 12 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.862293:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e6303643b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d523a expref: 12 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.862301:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9ac00, rc 0) 00000100:00100000:2.0:1780298645.862321:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609733760:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 929us (994us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.862326:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10524 00000100:00100000:1.0:1780298645.863205:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.863232:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609733888 00000100:00100000:0.0:1780298645.863236:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609733888 00000100:00100000:1.0:1780298645.863237:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.863262:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609734016 00000100:00100000:0.0:1780298645.863265:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609734016 00000100:00100000:0.0:1780298645.863291:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10525 00000100:00100000:2.0:1780298645.863302:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10526 00000100:00100000:0.0:1780298645.863302:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2398:x1866777609733888:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.863310:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298645.863316:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609734016:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.863318:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e63036442 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5241 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.863325:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.863333:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0a0f245/0xaf35132e63036449 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5248 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.863340:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605118592:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.863362:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255491500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605118720:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.863378:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba49c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.863389:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6cf00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.864005:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605118592:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.864023:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255491500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605118720:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.864023:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e63036442 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5241 expref: 24 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.864039:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e63036442 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5241 expref: 24 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.864043:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e0a0f245/0xaf35132e63036449 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5248 expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.864050:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3800, rc 0) 00010000:00010000:2.0:1780298645.864065:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0a0f245/0xaf35132e63036449 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5248 expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.864076:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2398:x1866777609733888:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 778us (871us total) trans 0 rc 301/301 00010000:00010000:2.0:1780298645.864077:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b200, rc 0) 00000100:00100000:0.0:1780298645.864083:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10525 00000100:00100000:2.0:1780298645.864107:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609734016:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 795us (868us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.864112:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10526 00000100:00100000:1.0:1780298645.865084:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.865118:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609734144 00000100:00100000:3.0:1780298645.865148:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609734144 00000100:00100000:3.0:1780298645.865178:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10527 00000100:00100000:3.0:1780298645.865184:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:29360:x1866777609734144:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.865191:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.865198:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e63036450 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d524f expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.865211:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000cd0af3fe/0xaf35132e6303642d lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [454656->4403199] (req 454656->466943) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d522c expref: 22 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.865217:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(69 169) 00010000:00010000:3.0:1780298645.865220:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(70 169) 00010000:00010000:3.0:1780298645.865223:0:20252:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e63036450 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1851392->1863679] (req 1851392->1863679) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d524f expref: 16 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.865241:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000cd0af3fe/0xaf35132e6303642d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [454656->4403199] (req 454656->466943) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d522c expref: 22 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.865251:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000cd0af3fe/0xaf35132e6303642d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [454656->4403199] (req 454656->466943) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d522c expref: 22 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.865258:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605118848:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298645.865303:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856180 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.865855:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605118848:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298645.865872:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e63036450 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1851392->1863679] (req 1851392->1863679) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d524f expref: 16 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.865879:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93600, rc 0) 00000100:00100000:3.0:1780298645.865900:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:29360:x1866777609734144:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 718us (818us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298645.865905:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10527 00000100:00100000:1.0:1780298645.866170:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.866191:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609734272 00000100:00100000:2.0:1780298645.866196:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609734272 00002000:00100000:2.0:1780298645.866218:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (454656->466943) req@ffff9cf24304ce00 x1866777609734272/t0(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 12776/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.866228:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24304ce00 with x1866777609734272 ext(454656->466943) 00010000:00010000:2.0:1780298645.866234:0:8363:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000cd0af3fe/0xaf35132e6303642d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [454656->4403199] (req 454656->466943) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d522c expref: 23 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.866246:0:8363:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000cd0af3fe/0xaf35132e6303642d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [454656->4403199] (req 454656->466943) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d522c expref: 23 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1780298645.866253:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf24304ce00 00000100:00100000:2.0:1780298645.866283:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 222 00000100:00100000:2.0:1780298645.866290:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2399:x1866777609734272:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:2.0:1780298645.866722:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+25:2399:x1866777609734272:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 432us (554us total) trans 4294969801 rc 0/0 00000100:00100000:2.0:1780298645.866729:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 222 00002000:00100000:2.0:1780298645.866736:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (454656->466943) req@ffff9cf24304ce00 x1866777609734272/t4294969801(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 12776/448 e 0 to 0 dl 1780298656 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.866743:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24304ce00 with x1866777609734272 ext(454656->466943) 00010000:00010000:2.0:1780298645.866748:0:8363:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000cd0af3fe/0xaf35132e6303642d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [454656->4403199] (req 454656->466943) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d522c expref: 25 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.866759:0:8363:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000cd0af3fe/0xaf35132e6303642d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [454656->4403199] (req 454656->466943) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d522c expref: 25 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1780298645.866766:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf24304ce00 00000100:00100000:1.0:1780298645.867543:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.867587:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609734400 00000100:00100000:2.0:1780298645.867591:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609734400 00010000:00010000:2.0:1780298645.867605:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000cd0af3fe/0xaf35132e6303642d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [454656->4403199] (req 454656->466943) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d522c expref: 25 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.867626:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15329 00000100:00100000:2.0:1780298645.867631:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433de300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+25:2400:x1866777609734400:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298645.867638:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:2.0:1780298645.867644:0:8374:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 440 -> 464 00010000:00010000:2.0:1780298645.867648:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000cd0af3fe/0xaf35132e6303642d lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [454656->4403199] (req 454656->466943) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d522c expref: 25 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.867656:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000cd0af3fe/0xaf35132e6303642d lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [454656->4403199] (req 454656->466943) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d522c expref: 25 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.867666:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000cd0af3fe/0xaf35132e6303642d lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [454656->4403199] (req 454656->466943) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d522c expref: 25 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.867673:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(70 169) 00010000:00010000:2.0:1780298645.867675:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(70 169) 00010000:00010000:2.0:1780298645.867679:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e63036450 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1851392->4403199] (req 1851392->1863679) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d524f expref: 15 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.867685:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e63036450 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1851392->4403199] (req 1851392->1863679) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d524f expref: 15 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.867700:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e63036450 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1851392->4403199] (req 1851392->1863679) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d524f expref: 15 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.867712:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255491500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777605118976:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298645.867740:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6cf00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.868379:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255491500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777605118976:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298645.868388:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298645.868407:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433de300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+24:2400:x1866777609734400:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 776us (864us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.868414:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15329 00000100:00100000:1.0:1780298645.868474:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.868534:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609734528 00000100:00100000:3.0:1780298645.868539:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609734528 00000100:00100000:3.0:1780298645.868565:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6529 00000100:00100000:3.0:1780298645.868572:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609734528:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:3.0:1780298645.868681:0:16972:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4412060 -> 4194304 00002000:00010000:3.0:1780298645.868684:0:16972:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:3.0:1780298645.868686:0:16972:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298646 -> 1780298646 00002000:00010000:3.0:1780298645.868687:0:16972:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:3.0:1780298645.868689:0:16972:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 464 -> 464 00000100:00100000:3.0:1780298645.868715:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2400:x1866777609734528:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 144us (241us total) trans 4294969802 rc 0/0 00000100:00100000:3.0:1780298645.868720:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6529 00000100:00100000:1.0:1780298645.869562:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.869591:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609734656 00000100:00100000:2.0:1780298645.869595:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609734656 00000100:00100000:2.0:1780298645.869619:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10528 00000100:00100000:2.0:1780298645.869625:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+24:2397:x1866777609734656:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.869632:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.869639:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63036457 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5256 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.869663:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255491500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605119104:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.869675:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.869716:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609734784 00000100:00100000:2.0:1780298645.869719:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6cf00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.869721:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609734784 00000100:00100000:3.0:1780298645.869757:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10529 00000100:00100000:3.0:1780298645.869766:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609734784:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.869772:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.869780:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f8ed852/0xaf35132e6303645e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d525d expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.869800:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605119232:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.869870:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856180 going to sleep for 11 seconds 00002000:00010000:2.0:1780298645.870309:0:12101:0:(ofd_lvb.c:259:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size: 4194304 -> 4412060 00000100:00100000:2.0:1780298645.870318:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255491500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605119104:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.870333:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63036457 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5256 expref: 25 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.870345:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63036457 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5256 expref: 25 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.870353:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9ac00, rc 0) 00000100:00100000:2.0:1780298645.870378:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+24:2397:x1866777609734656:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 753us (816us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.870385:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10528 00000100:00100000:3.0:1780298645.870445:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605119232:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.870457:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002f8ed852/0xaf35132e6303645e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d525d expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.870467:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f8ed852/0xaf35132e6303645e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d525d expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.870473:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92c00, rc 0) 00000100:00100000:3.0:1780298645.870495:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609734784:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 731us (822us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.870503:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10529 00000100:00100000:0.0:1780298645.871187:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.871218:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609734912 00000100:00100000:2.0:1780298645.871223:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609734912 00000100:00100000:2.0:1780298645.871248:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10530 00000100:00100000:2.0:1780298645.871254:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f736680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+24:2398:x1866777609734912:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.871261:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.871279:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e0a0f245/0xaf35132e63036465 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5264 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.871305:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255491500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605119360:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.871318:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.871341:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6cf00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.871379:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609735040 00000100:00100000:3.0:1780298645.871383:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609735040 00000100:00100000:1.0:1780298645.871421:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10531 00000100:00100000:1.0:1780298645.871430:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609735040:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.871437:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.871443:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002ff2dd4d/0xaf35132e6303646c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d526b expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.871464:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433dd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605119488:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.871491:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.871843:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255491500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605119360:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.871854:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e0a0f245/0xaf35132e63036465 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5264 expref: 25 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.871864:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e0a0f245/0xaf35132e63036465 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5264 expref: 25 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.871870:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b200, rc 0) 00000100:00100000:2.0:1780298645.871892:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f736680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+24:2398:x1866777609734912:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 639us (706us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.871897:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10530 00000100:00100000:1.0:1780298645.872034:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433dd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605119488:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.872046:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002ff2dd4d/0xaf35132e6303646c lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d526b expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.872055:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002ff2dd4d/0xaf35132e6303646c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d526b expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.872065:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278800, rc 0) 00000100:00100000:1.0:1780298645.872087:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609735040:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 660us (773us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.872092:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10531 00000100:00100000:0.0:1780298645.872878:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.872907:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609735296 00000100:00100000:2.0:1780298645.872911:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609735296 00000100:00100000:2.0:1780298645.872933:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10532 00000100:00100000:2.0:1780298645.872940:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f736d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609735296:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.872946:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.872952:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e63036473 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5279 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.872974:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255491500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605119616:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.873006:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.873009:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6cf00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.873046:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609735168 00000100:00100000:0.0:1780298645.873051:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609735168 00000100:00100000:0.0:1780298645.873081:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10533 00000100:00100000:0.0:1780298645.873090:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2397:x1866777609735168:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.873097:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.873105:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000488a7b88/0xaf35132e6303647a lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5272 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.873152:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f734000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605119744:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.873193:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba49c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.873567:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255491500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605119616:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.873578:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e63036473 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5279 expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.873587:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e63036473 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5279 expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.873593:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b000, rc 0) 00000100:00100000:2.0:1780298645.873622:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f736d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609735296:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 683us (745us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.873627:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10532 00000100:00100000:0.0:1780298645.873711:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f734000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605119744:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.873724:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000488a7b88/0xaf35132e6303647a lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5272 expref: 18 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.873734:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000488a7b88/0xaf35132e6303647a lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5272 expref: 18 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.873742:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2200, rc 0) 00000100:00100000:0.0:1780298645.873764:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2397:x1866777609735168:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 678us (790us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.873770:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10533 00000100:00100000:1.0:1780298645.874417:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.874453:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609735424 00000100:00100000:3.0:1780298645.874457:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609735424 00000100:00100000:1.0:1780298645.874464:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.874492:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609735552 00000100:00100000:2.0:1780298645.874496:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609735552 00000100:00100000:0.0:1780298645.874505:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10534 00000100:00100000:0.0:1780298645.874514:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2398:x1866777609735424:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298645.874519:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10535 00010000:00010000:0.0:1780298645.874521:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298645.874529:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609735552:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.874529:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e63036481 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5280 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.874540:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.874548:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c1a1ebef/0xaf35132e63036488 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5287 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.874566:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f734000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605119872:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.874575:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605120000:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.874611:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856180 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.874613:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba49c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.875286:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f734000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605119872:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.875292:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605120000:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.875305:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e63036481 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5280 expref: 18 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.875310:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c1a1ebef/0xaf35132e63036488 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5287 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.875335:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e63036481 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5280 expref: 18 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.875336:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c1a1ebef/0xaf35132e63036488 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5287 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.875355:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93a00, rc 0) 00010000:00010000:0.0:1780298645.875360:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2600, rc 0) 00000100:00100000:3.0:1780298645.875379:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609735552:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 853us (916us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.875387:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10535 00000100:00100000:0.0:1780298645.875388:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2398:x1866777609735424:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 873us (969us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.875398:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10534 00000100:00100000:1.0:1780298645.876227:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.876245:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609735680 00000100:00100000:1.0:1780298645.876249:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.876251:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609735680 00000100:00100000:2.0:1780298645.876287:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609735808 00000100:00100000:3.0:1780298645.876292:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10536 00000100:00100000:2.0:1780298645.876292:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609735808 00000100:00100000:3.0:1780298645.876302:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2398:x1866777609735680:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.876308:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.876314:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002f8ed852/0xaf35132e6303648f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d528e expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.876327:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10537 00000100:00100000:3.0:1780298645.876337:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605120128:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.876338:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609735808:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.876347:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.876354:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0a0f245/0xaf35132e63036496 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5295 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.876373:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856180 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.876378:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255491500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605120256:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.876411:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6cf00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.876984:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605120128:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.876997:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002f8ed852/0xaf35132e6303648f lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d528e expref: 18 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.877007:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002f8ed852/0xaf35132e6303648f lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d528e expref: 18 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.877013:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92c00, rc 0) 00000100:00100000:3.0:1780298645.877036:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2398:x1866777609735680:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 737us (810us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.877041:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255491500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605120256:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.877044:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10536 00010000:00010000:2.0:1780298645.877067:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e0a0f245/0xaf35132e63036496 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5295 expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.877084:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0a0f245/0xaf35132e63036496 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5295 expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.877094:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b200, rc 0) 00000100:00100000:2.0:1780298645.877118:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609735808:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 785us (870us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.877147:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10537 00000100:00100000:1.0:1780298645.878042:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.878078:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609735936 00000100:00100000:2.0:1780298645.878083:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609735936 00002000:00100000:2.0:1780298645.878108:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x12:0x0] (3469312->3473407) req@ffff9cf34372e680 x1866777609735936/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.878115:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf34372e680 with x1866777609735936 ext(3469312->3473407) 00002000:00010000:2.0:1780298645.878119:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf34372e680 00000100:00100000:2.0:1780298645.878168:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6530 00000100:00100000:2.0:1780298645.878177:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609735936:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298645.878306:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609735936:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 131us (264us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298645.878311:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6530 00002000:00100000:2.0:1780298645.878316:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x12:0x0] (3469312->3473407) req@ffff9cf34372e680 x1866777609735936/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/440 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.878322:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf34372e680 with x1866777609735936 ext(3469312->3473407) 00002000:00010000:2.0:1780298645.878325:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf34372e680 00000100:00100000:1.0:1780298645.879408:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.879437:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609736064 00000100:00100000:2.0:1780298645.879440:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609736064 00002000:00100000:2.0:1780298645.879454:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x12:0x0] (3469312->3473407) req@ffff9cf34372e300 x1866777609736064/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 4584/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.879460:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf34372e300 with x1866777609736064 ext(3469312->3473407) 00002000:00010000:2.0:1780298645.879462:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf34372e300 00000100:00100000:2.0:1780298645.879476:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6531 00000100:00100000:2.0:1780298645.879482:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609736064:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:2.0:1780298645.879894:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609736064:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 412us (486us total) trans 4294969334 rc 0/0 00000100:00100000:2.0:1780298645.879899:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6531 00002000:00100000:2.0:1780298645.879903:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x12:0x0] (3469312->3473407) req@ffff9cf34372e300 x1866777609736064/t4294969334(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 4584/448 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.879908:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf34372e300 with x1866777609736064 ext(3469312->3473407) 00002000:00010000:2.0:1780298645.879912:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf34372e300 00000100:00100000:1.0:1780298645.880629:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.880664:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609736192 00000100:00100000:0.0:1780298645.880670:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609736192 00000100:00100000:0.0:1780298645.880711:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10538 00000100:00100000:0.0:1780298645.880718:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2397:x1866777609736192:12345-192.168.203.25@tcp:16:fsx.0 00000001:00080000:1.0:1780298645.881494:0:8351:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0000: transno 4294969334 is committed 00000001:00080000:1.0:1780298645.881505:0:8351:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0000: transno 4294969331 is committed 00000100:00100000:0.0:1780298645.881555:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609736192:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 838us (925us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.881560:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10538 00000100:00100000:1.0:1780298645.882588:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.882620:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609736448 00000100:00100000:0.0:1780298645.882625:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609736448 00000100:00100000:0.0:1780298645.882651:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10539 00000100:00100000:0.0:1780298645.882657:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609736448:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.882665:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.882675:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000488a7b88/0xaf35132e6303649d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d52a3 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.882704:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f734000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605120384:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.882737:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.882748:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba49c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.882776:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609736320 00000100:00100000:3.0:1780298645.882838:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609736320 00000100:00100000:3.0:1780298645.882863:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10540 00000100:00100000:3.0:1780298645.882869:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+24:2398:x1866777609736320:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.882878:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.882885:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e630364a4 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d529c expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.882911:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5d9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605120512:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.882938:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856180 going to sleep for 11 seconds 00002000:00010000:0.0:1780298645.883495:0:8358:0:(ofd_lvb.c:259:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size: 1601008 -> 3473408 00002000:00010000:0.0:1780298645.883499:0:8358:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks: 32 -> 40 00000100:00100000:0.0:1780298645.883505:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f734000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605120384:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.883521:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000488a7b88/0xaf35132e6303649d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d52a3 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.883535:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000488a7b88/0xaf35132e6303649d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d52a3 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.883547:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2200, rc 0) 00000100:00100000:0.0:1780298645.883578:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609736448:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 921us (990us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.883586:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10539 00000100:00100000:3.0:1780298645.883654:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5d9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605120512:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.883728:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e630364a4 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d529c expref: 25 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.883738:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e630364a4 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d529c expref: 25 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.883744:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92400, rc 0) 00000100:00100000:3.0:1780298645.883767:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+24:2398:x1866777609736320:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 898us (1031us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.883772:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10540 00000100:00100000:1.0:1780298645.884655:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.884686:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609736576 00000100:00100000:2.0:1780298645.884690:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609736576 00000100:00100000:2.0:1780298645.884718:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10541 00000100:00100000:2.0:1780298645.884726:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+24:2397:x1866777609736576:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.884732:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.884739:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009fd8d343/0xaf35132e630364ab lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d52aa expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.884755:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.884763:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255491500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605120640:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.884786:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609736704 00000100:00100000:0.0:1780298645.884791:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609736704 00000100:00100000:2.0:1780298645.884802:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6cf00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.884818:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10542 00000100:00100000:0.0:1780298645.884824:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609736704:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.884830:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.884837:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0cae770/0xaf35132e630364b2 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d52b1 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.884858:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f734000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605120768:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.884883:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba49c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.885425:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255491500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605120640:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.885437:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009fd8d343/0xaf35132e630364ab lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d52aa expref: 25 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.885446:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009fd8d343/0xaf35132e630364ab lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d52aa expref: 25 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.885452:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b000, rc 0) 00000100:00100000:2.0:1780298645.885474:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+24:2397:x1866777609736576:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 751us (822us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.885478:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10541 00000100:00100000:0.0:1780298645.885606:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f734000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605120768:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.885622:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e0cae770/0xaf35132e630364b2 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d52b1 expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.885637:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0cae770/0xaf35132e630364b2 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d52b1 expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.885647:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2600, rc 0) 00000100:00100000:0.0:1780298645.885667:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609736704:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 844us (915us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.885672:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10542 00000100:00100000:1.0:1780298645.886455:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.886488:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609736832 00000100:00100000:0.0:1780298645.886491:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609736832 00000100:00100000:0.0:1780298645.886513:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10543 00000100:00100000:0.0:1780298645.886520:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254889180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2397:x1866777609736832:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.886527:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.886534:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e630364b9 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d52b8 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.886554:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f734000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605120896:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.886579:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.886589:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba49c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.886619:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609736960 00000100:00100000:2.0:1780298645.886623:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609736960 00000100:00100000:2.0:1780298645.886649:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10544 00000100:00100000:2.0:1780298645.886657:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609736960:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.886666:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.886675:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e630364c0 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d52bf expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.886699:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255491500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605121024:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.886724:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6cf00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.887231:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f734000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605120896:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.887245:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e630364b9 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d52b8 expref: 18 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.887256:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e630364b9 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d52b8 expref: 18 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.887265:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3200, rc 0) 00000100:00100000:0.0:1780298645.887298:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254889180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2397:x1866777609736832:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 779us (843us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.887305:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10543 00000100:00100000:3.0:1780298645.887342:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255491500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605121024:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.887422:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e630364c0 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d52bf expref: 8 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.887439:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e630364c0 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d52bf expref: 8 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.887449:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9ac00, rc 0) 00000100:00100000:3.0:1780298645.887477:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609736960:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 822us (900us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.887485:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10544 00000100:00100000:1.0:1780298645.888212:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.888233:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.888234:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609737216 00000100:00100000:3.0:1780298645.888238:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609737216 00000100:00100000:2.0:1780298645.888263:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609737088 00000100:00100000:3.0:1780298645.888277:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10545 00000100:00100000:2.0:1780298645.888278:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609737088 00000100:00100000:3.0:1780298645.888285:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254888380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609737216:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.888292:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.888299:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f8ed852/0xaf35132e630364c7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d52cd expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.888306:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10546 00000100:00100000:2.0:1780298645.888313:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254888000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2398:x1866777609737088:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.888320:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298645.888327:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36e5da300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605121152:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.888327:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e630364ce lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d52c6 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.888355:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255493800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605121280:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.888366:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856180 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.888379:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c840 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.889083:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36e5da300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605121152:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.889097:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002f8ed852/0xaf35132e630364c7 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d52cd expref: 8 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.889107:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f8ed852/0xaf35132e630364c7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d52cd expref: 8 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.889115:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92c00, rc 0) 00000100:00100000:2.0:1780298645.889116:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255493800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605121280:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.889159:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254888380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609737216:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 874us (945us total) trans 0 rc 301/301 00010000:00010000:2.0:1780298645.889162:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e630364ce lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d52c6 expref: 18 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.889174:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10545 00010000:00010000:2.0:1780298645.889182:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e630364ce lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d52c6 expref: 18 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.889194:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9a200, rc 0) 00000100:00100000:2.0:1780298645.889226:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254888000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2398:x1866777609737088:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 914us (993us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.889232:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10546 00000100:00100000:1.0:1780298645.890354:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.890381:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609737344 00000100:00100000:3.0:1780298645.890387:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609737344 00002000:00100000:3.0:1780298645.890415:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (1900544->1912831) req@ffff9cf25488a680 x1866777609737344/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298645.890424:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25488a680 with x1866777609737344 ext(1900544->1912831) 00002000:00010000:3.0:1780298645.890429:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf25488a680 00000100:00100000:3.0:1780298645.890454:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6532 00000100:00100000:3.0:1780298645.890462:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25488a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609737344:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298645.890633:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25488a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609737344:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 173us (279us total) trans 0 rc 12288/12288 00000100:00100000:3.0:1780298645.890638:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6532 00002000:00100000:3.0:1780298645.890643:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (1900544->1912831) req@ffff9cf25488a680 x1866777609737344/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/12728 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 12288/12288 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298645.890649:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf25488a680 with x1866777609737344 ext(1900544->1912831) 00002000:00010000:3.0:1780298645.890652:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf25488a680 00000100:00100000:0.0:1780298645.891757:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.891808:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609737472 00000100:00100000:1.0:1780298645.891814:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609737472 00000100:00100000:1.0:1780298645.891850:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10547 00000100:00100000:1.0:1780298645.891858:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f734000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2398:x1866777609737472:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.891866:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298645.891872:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:1.0:1780298645.891876:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e630364d5 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d52d4 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.891905:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609737600 00000100:00100000:2.0:1780298645.891910:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609737600 00000100:00100000:1.0:1780298645.891915:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254889c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605121408:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.891934:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10548 00000100:00100000:2.0:1780298645.891941:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5da300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609737600:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.891948:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.891955:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c1d9addf/0xaf35132e630364dc lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d52db expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.891966:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.891979:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255493800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605121536:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.892004:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.892735:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254889c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605121408:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.892753:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e630364d5 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d52d4 expref: 18 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.892766:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e630364d5 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d52d4 expref: 18 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.892777:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278800, rc 0) 00000100:00100000:2.0:1780298645.892794:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255493800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605121536:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.892812:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f734000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2398:x1866777609737472:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 953us (1054us total) trans 0 rc 301/301 00010000:00010000:2.0:1780298645.892816:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c1d9addf/0xaf35132e630364dc lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d52db expref: 8 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.892827:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10547 00010000:00010000:2.0:1780298645.892833:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c1d9addf/0xaf35132e630364dc lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d52db expref: 8 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.892843:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9a600, rc 0) 00000100:00100000:2.0:1780298645.892874:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5da300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609737600:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 935us (1003us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.892879:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10548 00000100:00100000:3.0:1780298645.893957:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.893994:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609737728 00000100:00100000:1.0:1780298645.894000:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609737728 00000100:00100000:3.0:1780298645.894008:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.894042:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609737856 00000100:00100000:1.0:1780298645.894046:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609737856 00000100:00100000:0.0:1780298645.894053:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10549 00000100:00100000:0.0:1780298645.894064:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2399:x1866777609737728:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.894071:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298645.894075:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10550 00010000:00010000:0.0:1780298645.894081:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000488a7b88/0xaf35132e630364e3 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d52e2 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.894086:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609737856:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.894097:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.894106:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e630364ea lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d52e9 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.894117:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f737b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605121664:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.894160:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254889c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605121792:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.894189:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba49c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.894190:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.894986:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f737b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605121664:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.895009:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000488a7b88/0xaf35132e630364e3 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d52e2 expref: 18 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.895020:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254889c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605121792:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.895040:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000488a7b88/0xaf35132e630364e3 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d52e2 expref: 18 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.895044:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e630364ea lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d52e9 expref: 8 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.895063:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2200, rc 0) 00010000:00010000:1.0:1780298645.895070:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e630364ea lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d52e9 expref: 8 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.895080:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279a00, rc 0) 00000100:00100000:3.0:1780298645.895100:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2399:x1866777609737728:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1038us (1142us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.895107:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609737856:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1023us (1098us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.895113:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10549 00000100:00100000:1.0:1780298645.895121:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10550 00000100:00100000:2.0:1780298645.896226:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.896264:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609737984 00000100:00100000:0.0:1780298645.896280:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609737984 00000100:00100000:0.0:1780298645.896308:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10551 00000100:00100000:0.0:1780298645.896314:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255493800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+24:29360:x1866777609737984:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.896324:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.896334:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e630364f1 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d52f0 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.896356:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e63036450 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1851392->4403199] (req 1851392->1863679) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d524f expref: 16 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.896370:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(70 169) 00010000:00010000:0.0:1780298645.896373:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(71 169) 00010000:00010000:0.0:1780298645.896378:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e630364f1 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3866624->3878911] (req 3866624->3878911) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d52f0 expref: 25 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.896396:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e63036450 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1851392->4403199] (req 1851392->1863679) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d524f expref: 16 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.896411:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e63036450 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1851392->4403199] (req 1851392->1863679) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d524f expref: 16 pid: 20252 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.896420:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f734a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605121920:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298645.896452:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4300 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.897117:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f734a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605121920:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298645.897167:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e630364f1 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3866624->3878911] (req 3866624->3878911) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d52f0 expref: 25 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.897181:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3200, rc 0) 00000100:00100000:0.0:1780298645.897211:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255493800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+25:29360:x1866777609737984:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 896us (986us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.897221:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10551 00000100:00100000:2.0:1780298645.897432:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.897462:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609738112 00000100:00100000:3.0:1780298645.897466:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609738112 00010000:00010000:3.0:1780298645.897483:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e63036450 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1851392->4403199] (req 1851392->1863679) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d524f expref: 17 pid: 20252 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.897509:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15330 00000100:00100000:3.0:1780298645.897514:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255493480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2399:x1866777609738112:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298645.897522:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298645.897531:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e63036450 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1851392->4403199] (req 1851392->1863679) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d524f expref: 17 pid: 20252 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.897539:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e63036450 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1851392->4403199] (req 1851392->1863679) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d524f expref: 17 pid: 20252 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.897551:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e63036450 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1851392->4403199] (req 1851392->1863679) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d524f expref: 17 pid: 20252 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.897561:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(71 169) 00010000:00010000:3.0:1780298645.897563:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(71 169) 00010000:00010000:3.0:1780298645.897569:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e630364f1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3866624->4403199] (req 3866624->3878911) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d52f0 expref: 24 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.897574:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e630364f1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3866624->4403199] (req 3866624->3878911) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d52f0 expref: 24 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.897590:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e630364f1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3866624->4403199] (req 3866624->3878911) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d52f0 expref: 24 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.897598:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad9180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777605122048:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298645.897626:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856180 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.898260:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad9180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777605122048:192.168.203.25@tcp:105: 00010000:00010000:3.0:1780298645.898285:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298645.898312:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255493480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2399:x1866777609738112:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 797us (879us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298645.898321:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15330 00000100:00100000:2.0:1780298645.898533:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.898564:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609738240 00000100:00100000:3.0:1780298645.898568:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609738240 00000100:00100000:3.0:1780298645.898601:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6533 00000100:00100000:3.0:1780298645.898607:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255491500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+25:2399:x1866777609738240:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:3.0:1780298645.898862:0:8363:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4412060 -> 4194304 00002000:00010000:3.0:1780298645.898865:0:8363:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:3.0:1780298645.898867:0:8363:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298646 -> 1780298646 00002000:00010000:3.0:1780298645.898869:0:8363:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:3.0:1780298645.898870:0:8363:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 464 -> 488 00000100:00100000:3.0:1780298645.898898:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255491500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+26:2399:x1866777609738240:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 291us (366us total) trans 4294969803 rc 0/0 00000100:00100000:3.0:1780298645.898905:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6533 00000100:00100000:2.0:1780298645.899912:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.899941:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609738368 00000100:00100000:2.0:1780298645.899946:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.899947:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609738368 00000100:00100000:3.0:1780298645.899985:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609738496 00000100:00100000:3.0:1780298645.899989:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609738496 00000100:00100000:0.0:1780298645.900004:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10552 00000100:00100000:0.0:1780298645.900018:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2399:x1866777609738368:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298645.900023:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10553 00010000:00010000:0.0:1780298645.900030:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298645.900036:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e425850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609738496:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.900042:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e630364f8 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d52f7 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.900046:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.900059:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f8ed852/0xaf35132e630364ff lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d52fe expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.900088:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f734a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605122176:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.900096:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605122304:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.900169:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856180 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.900170:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4300 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.901032:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605122304:192.168.203.25@tcp:106: 00002000:00010000:0.0:1780298645.901033:0:9128:0:(ofd_lvb.c:259:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size: 4194304 -> 4412060 00000100:00100000:0.0:1780298645.901053:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f734a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605122176:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.901059:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002f8ed852/0xaf35132e630364ff lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d52fe expref: 8 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.901077:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e630364f8 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d52f7 expref: 17 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.901080:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f8ed852/0xaf35132e630364ff lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d52fe expref: 8 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.901103:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92c00, rc 0) 00010000:00010000:0.0:1780298645.901107:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e630364f8 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d52f7 expref: 17 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.901121:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3c00, rc 0) 00000100:00100000:3.0:1780298645.901209:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e425850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609738496:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1172us (1257us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.901209:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2399:x1866777609738368:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1191us (1293us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.901225:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10553 00000100:00100000:0.0:1780298645.901232:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10552 00000100:00100000:2.0:1780298645.902381:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.902408:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609738624 00000100:00100000:3.0:1780298645.902415:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609738624 00000100:00100000:2.0:1780298645.902424:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.902449:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609738752 00000100:00100000:3.0:1780298645.902452:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609738752 00000100:00100000:3.0:1780298645.902478:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10554 00000100:00100000:3.0:1780298645.902491:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2397:x1866777609738624:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298645.902491:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10555 00010000:00010000:3.0:1780298645.902505:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298645.902510:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609738752:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.902517:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e63036506 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5305 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.902520:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.902531:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e6303650d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d530c expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.902557:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605122432:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.902567:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254889c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605122560:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.902609:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856180 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.902610:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.903564:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605122432:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.903589:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e63036506 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5305 expref: 17 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.903594:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254889c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605122560:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.903618:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e63036506 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5305 expref: 17 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.903625:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e6303650d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d530c expref: 8 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.903634:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92600, rc 0) 00010000:00010000:1.0:1780298645.903644:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e6303650d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d530c expref: 8 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.903655:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279a00, rc 0) 00000100:00100000:3.0:1780298645.903670:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2397:x1866777609738624:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1183us (1289us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.903678:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10554 00000100:00100000:1.0:1780298645.903683:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609738752:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1178us (1257us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.903691:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10555 00000100:00100000:2.0:1780298645.904842:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.904872:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.904894:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609738880 00000100:00100000:1.0:1780298645.904901:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609738880 00000100:00100000:3.0:1780298645.904906:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609739008 00000100:00100000:3.0:1780298645.904913:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609739008 00000100:00100000:1.0:1780298645.904941:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10556 00000100:00100000:1.0:1780298645.904952:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2399:x1866777609738880:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298645.904964:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10557 00010000:00010000:1.0:1780298645.904964:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298645.904977:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609739008:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.904978:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e63036514 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5313 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.904991:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.905003:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c1a1ebef/0xaf35132e6303651b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d531a expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.905040:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605122816:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.905040:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254889c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605122688:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.905106:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856180 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.905107:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.905955:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254889c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605122688:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.905965:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605122816:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.905980:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e63036514 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5313 expref: 17 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.905988:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c1a1ebef/0xaf35132e6303651b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d531a expref: 8 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.906006:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e63036514 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5313 expref: 17 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.906008:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c1a1ebef/0xaf35132e6303651b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d531a expref: 8 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.906036:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93a00, rc 0) 00010000:00010000:1.0:1780298645.906036:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278800, rc 0) 00000100:00100000:3.0:1780298645.906071:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609739008:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1097us (1198us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.906073:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2399:x1866777609738880:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1122us (1232us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.906084:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10557 00000100:00100000:1.0:1780298645.906090:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10556 00000100:00100000:2.0:1780298645.907177:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.907211:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609739136 00000100:00100000:1.0:1780298645.907217:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609739136 00000100:00100000:1.0:1780298645.907252:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10558 00000100:00100000:1.0:1780298645.907260:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:29360:x1866777609739136:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.907280:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.907291:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63036522 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5321 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.907313:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e630364f1 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3866624->4403199] (req 3866624->3878911) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d52f0 expref: 25 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.907323:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(71 169) 00010000:00010000:1.0:1780298645.907328:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(72 169) 00010000:00010000:1.0:1780298645.907333:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63036522 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [4272128->4276223] (req 4272128->4276223) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5321 expref: 17 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.907357:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e630364f1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3866624->4403199] (req 3866624->3878911) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d52f0 expref: 25 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.907371:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e630364f1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3866624->4403199] (req 3866624->3878911) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d52f0 expref: 25 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.907383:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254889c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605122944:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298645.907423:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.908102:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254889c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605122944:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298645.908120:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63036522 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [4272128->4276223] (req 4272128->4276223) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5321 expref: 17 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.908152:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278c00, rc 0) 00000100:00100000:1.0:1780298645.908180:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:29360:x1866777609739136:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 921us (1006us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298645.908189:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10558 00000100:00100000:2.0:1780298645.908308:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.908348:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609739264 00000100:00100000:1.0:1780298645.908353:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609739264 00010000:00010000:1.0:1780298645.908372:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e630364f1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3866624->4403199] (req 3866624->3878911) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d52f0 expref: 26 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.908404:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15331 00000100:00100000:1.0:1780298645.908413:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+26:2399:x1866777609739264:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298645.908423:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298645.908438:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e630364f1 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3866624->4403199] (req 3866624->3878911) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d52f0 expref: 26 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.908452:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e630364f1 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3866624->4403199] (req 3866624->3878911) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d52f0 expref: 26 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.908477:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e630364f1 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [3866624->4403199] (req 3866624->3878911) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d52f0 expref: 26 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.908496:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(72 169) 00010000:00010000:1.0:1780298645.908500:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(72 169) 00010000:00010000:1.0:1780298645.908508:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63036522 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [4272128->4403199] (req 4272128->4276223) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5321 expref: 16 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.908518:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63036522 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [4272128->4403199] (req 4272128->4276223) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5321 expref: 16 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.908539:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63036522 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [4272128->4403199] (req 4272128->4276223) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5321 expref: 16 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.908546:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254889c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777605123072:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298645.908575:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.909319:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254889c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777605123072:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298645.909328:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298645.909350:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+25:2399:x1866777609739264:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 940us (1046us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298645.909356:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15331 00000100:00100000:2.0:1780298645.909596:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.909650:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609739392 00000100:00100000:3.0:1780298645.909661:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609739392 00002000:00100000:3.0:1780298645.909699:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (4272128->4276223) req@ffff9cf24310ed80 x1866777609739392/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298645.909717:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24310ed80 with x1866777609739392 ext(4272128->4276223) 00002000:00010000:3.0:1780298645.909723:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24310ed80 00000100:00100000:3.0:1780298645.909751:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6534 00000100:00100000:3.0:1780298645.909759:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2399:x1866777609739392:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298645.909927:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2399:x1866777609739392:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 168us (331us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298645.909933:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6534 00002000:00100000:3.0:1780298645.909938:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (4272128->4276223) req@ffff9cf24310ed80 x1866777609739392/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/440 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298645.909944:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24310ed80 with x1866777609739392 ext(4272128->4276223) 00002000:00010000:3.0:1780298645.909948:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24310ed80 00000100:00100000:2.0:1780298645.911257:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.911305:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609739520 00000100:00100000:1.0:1780298645.911312:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609739520 00002000:00100000:1.0:1780298645.911348:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (4276224->4280319) req@ffff9cf24310ea00 x1866777609739520/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.911359:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24310ea00 with x1866777609739520 ext(4276224->4280319) 00002000:00010000:1.0:1780298645.911364:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24310ea00 00000100:00100000:1.0:1780298645.911390:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6535 00000100:00100000:1.0:1780298645.911400:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2398:x1866777609739520:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298645.911544:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2398:x1866777609739520:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 146us (289us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298645.911552:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6535 00002000:00100000:1.0:1780298645.911559:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (4276224->4280319) req@ffff9cf24310ea00 x1866777609739520/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/440 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.911568:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24310ea00 with x1866777609739520 ext(4276224->4280319) 00002000:00010000:1.0:1780298645.911572:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24310ea00 00000100:00100000:2.0:1780298645.912822:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.912870:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609739648 00000100:00100000:0.0:1780298645.912874:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609739648 00002000:00100000:0.0:1780298645.912897:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (4280320->4284415) req@ffff9cf24310df80 x1866777609739648/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298645.912908:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24310df80 with x1866777609739648 ext(4280320->4284415) 00002000:00010000:0.0:1780298645.912912:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24310df80 00000100:00100000:0.0:1780298645.912937:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6536 00000100:00100000:0.0:1780298645.912944:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2400:x1866777609739648:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298645.913057:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2400:x1866777609739648:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 112us (235us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.913064:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6536 00002000:00100000:0.0:1780298645.913070:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (4280320->4284415) req@ffff9cf24310df80 x1866777609739648/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/440 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298645.913080:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24310df80 with x1866777609739648 ext(4280320->4284415) 00002000:00010000:0.0:1780298645.913084:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24310df80 00000100:00100000:2.0:1780298645.914290:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.914331:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609739776 00000100:00100000:3.0:1780298645.914341:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609739776 00002000:00100000:3.0:1780298645.914374:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (4272128->4412059) req@ffff9cf24310c000 x1866777609739776/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 504/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298645.914381:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24310c000 with x1866777609739776 ext(4272128->4412059) 00002000:00010000:3.0:1780298645.914392:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24310c000 00000100:00100000:3.0:1780298645.914413:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6537 00000100:00100000:3.0:1780298645.914419:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2398:x1866777609739776:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:0.0:1780298645.915914:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+19:2398:x1866777609739776:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 1490us (1620us total) trans 4294969804 rc 0/0 00000100:00100000:0.0:1780298645.915925:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6537 00002000:00100000:0.0:1780298645.915935:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (4272128->4412059) req@ffff9cf24310c000 x1866777609739776/t4294969804(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 504/448 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298645.915953:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24310c000 with x1866777609739776 ext(4272128->4412059) 00002000:00010000:0.0:1780298645.915964:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24310c000 00000100:00100000:2.0:1780298645.916681:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.916727:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609739904 00000100:00100000:1.0:1780298645.916734:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609739904 00000100:00100000:1.0:1780298645.916768:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10559 00000100:00100000:1.0:1780298645.916774:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+19:2400:x1866777609739904:12345-192.168.203.25@tcp:16:fsx.0 00000001:00080000:0.0:1780298645.917724:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969804 is committed 00000001:00080000:0.0:1780298645.917735:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969803 is committed 00000100:00100000:1.0:1780298645.917884:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609739904:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 1109us (1201us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298645.917893:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10559 00000100:00100000:2.0:1780298645.919036:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.919069:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.919089:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609740032 00000100:00100000:0.0:1780298645.919096:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609740032 00000100:00100000:3.0:1780298645.919113:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609740160 00000100:00100000:3.0:1780298645.919142:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609740160 00000100:00100000:0.0:1780298645.919182:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10560 00000100:00100000:1.0:1780298645.919192:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10561 00000100:00100000:0.0:1780298645.919200:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609740032:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298645.919210:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609740160:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.919215:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.919219:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.919232:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002ff2dd4d/0xaf35132e63036530 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d532f expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.919232:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e63036529 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5328 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.919314:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605123328:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.919316:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254889c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605123200:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.919392:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.919392:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4300 going to sleep for 11 seconds 00002000:00010000:0.0:1780298645.920299:0:12101:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks: 488 -> 536 00000100:00100000:0.0:1780298645.920317:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605123328:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.920319:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254889c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605123200:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.920353:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e63036529 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5328 expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.920354:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002ff2dd4d/0xaf35132e63036530 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d532f expref: 8 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.920382:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e63036529 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5328 expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.920384:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002ff2dd4d/0xaf35132e63036530 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d532f expref: 8 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.920397:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3c00, rc 0) 00010000:00010000:1.0:1780298645.920401:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278800, rc 0) 00000100:00100000:0.0:1780298645.920431:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609740032:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1239us (1394us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.920435:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609740160:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1234us (1364us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.920445:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10560 00000100:00100000:1.0:1780298645.920451:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10561 00000100:00100000:2.0:1780298645.921402:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.921424:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.921437:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609740288 00000100:00100000:1.0:1780298645.921441:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609740288 00000100:00100000:0.0:1780298645.921457:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609740416 00000100:00100000:0.0:1780298645.921463:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609740416 00000100:00100000:1.0:1780298645.921473:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10562 00000100:00100000:1.0:1780298645.921481:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609740288:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.921491:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298645.921502:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10563 00010000:00010000:1.0:1780298645.921503:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63036537 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5336 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.921515:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609740416:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.921524:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.921534:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e6303653e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d533d expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.921541:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254889c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605123456:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.921572:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605123584:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.921603:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.921605:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4300 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.922342:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254889c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605123456:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.922359:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63036537 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5336 expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.922371:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63036537 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5336 expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.922378:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279a00, rc 0) 00000100:00100000:0.0:1780298645.922406:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605123584:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.922416:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609740288:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 934us (1011us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.922425:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10562 00010000:00010000:0.0:1780298645.922431:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e6303653e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d533d expref: 8 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.922442:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e6303653e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d533d expref: 8 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.922449:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3800, rc 0) 00000100:00100000:0.0:1780298645.922466:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609740416:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 956us (1041us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.922470:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10563 00000100:00100000:3.0:1780298645.923453:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.923502:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609740544 00000100:00100000:0.0:1780298645.923508:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609740544 00000100:00100000:0.0:1780298645.923537:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10564 00000100:00100000:0.0:1780298645.923543:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:29360:x1866777609740544:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.923550:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.923558:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a5a492f0/0xaf35132e63036545 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5344 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.923572:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(72 169) 00010000:00010000:0.0:1780298645.923576:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(72 169) 00010000:00010000:0.0:1780298645.923583:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a5a492f0/0xaf35132e63036545 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2203648->4268031] (req 2203648->2211839) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5344 expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.923594:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a5a492f0/0xaf35132e63036545 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2203648->4268031] (req 2203648->2211839) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5344 expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.923601:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3600, rc 0) 00000100:00100000:0.0:1780298645.923625:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:29360:x1866777609740544:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 83us (181us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.923629:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10564 00000100:00100000:2.0:1780298645.924486:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.924541:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609740672 00000100:00100000:3.0:1780298645.924552:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609740672 00002000:00100000:3.0:1780298645.924596:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (2203648->2211839) req@ffff9cf2425f2300 x1866777609740672/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298645.924607:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2425f2300 with x1866777609740672 ext(2203648->2211839) 00002000:00010000:3.0:1780298645.924613:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2425f2300 00000100:00100000:3.0:1780298645.924643:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6538 00000100:00100000:3.0:1780298645.924653:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609740672:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298645.924869:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609740672:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 218us (384us total) trans 0 rc 8192/8192 00000100:00100000:3.0:1780298645.924878:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6538 00002000:00100000:3.0:1780298645.924885:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (2203648->2211839) req@ffff9cf2425f2300 x1866777609740672/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/8632 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 8192/8192 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298645.924895:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2425f2300 with x1866777609740672 ext(2203648->2211839) 00002000:00010000:3.0:1780298645.924898:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2425f2300 00000100:00100000:2.0:1780298645.925931:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.925993:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609740800 00000100:00100000:1.0:1780298645.926001:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609740800 00000100:00100000:1.0:1780298645.926052:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10565 00000100:00100000:1.0:1780298645.926069:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609740800:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298645.926073:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:1.0:1780298645.926085:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.926105:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e6303654c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d534b expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.926108:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609740928 00000100:00100000:3.0:1780298645.926116:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609740928 00000100:00100000:3.0:1780298645.926186:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10566 00000100:00100000:1.0:1780298645.926197:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254889c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605123712:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.926204:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609740928:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.926218:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.926234:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c1a1ebef/0xaf35132e63036553 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5352 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.926292:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244adbb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605123840:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.926297:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.926341:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856000 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.927058:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254889c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605123712:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.927064:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244adbb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605123840:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.927090:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e6303654c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d534b expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.927100:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c1a1ebef/0xaf35132e63036553 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5352 expref: 8 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.927158:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e6303654c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d534b expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.927170:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279200, rc 0) 00010000:00010000:3.0:1780298645.927174:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c1a1ebef/0xaf35132e63036553 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5352 expref: 8 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.927191:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93a00, rc 0) 00000100:00100000:1.0:1780298645.927202:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609740800:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1141us (1273us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.927208:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10565 00000100:00100000:3.0:1780298645.927224:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2397:x1866777609740928:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1026us (1157us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.927230:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10566 00000100:00100000:2.0:1780298645.928200:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.928245:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.928256:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609741056 00000100:00100000:0.0:1780298645.928263:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609741056 00000100:00100000:3.0:1780298645.928291:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609741184 00000100:00100000:3.0:1780298645.928298:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609741184 00000100:00100000:0.0:1780298645.928323:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10567 00000100:00100000:0.0:1780298645.928340:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609741056:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.928352:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298645.928357:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10568 00010000:00010000:0.0:1780298645.928364:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e6303655a lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5359 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.928374:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609741184:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.928385:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.928395:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002ff2dd4d/0xaf35132e63036561 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5360 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.928410:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605123968:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.928435:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254889c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605124096:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.928461:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4300 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.928472:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.929159:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605123968:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.929174:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254889c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605124096:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.929179:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e6303655a lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5359 expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.929192:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e6303655a lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5359 expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.929202:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002ff2dd4d/0xaf35132e63036561 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5360 expref: 8 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.929207:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3c00, rc 0) 00010000:00010000:1.0:1780298645.929221:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002ff2dd4d/0xaf35132e63036561 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5360 expref: 8 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.929234:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278800, rc 0) 00000100:00100000:0.0:1780298645.929234:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609741056:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 902us (1035us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.929243:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10567 00000100:00100000:1.0:1780298645.929264:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609741184:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 898us (1018us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.929284:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10568 00000100:00100000:2.0:1780298645.930278:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.930313:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609741312 00000100:00100000:1.0:1780298645.930318:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609741312 00000100:00100000:1.0:1780298645.930350:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10569 00000100:00100000:1.0:1780298645.930360:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:29360:x1866777609741312:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.930369:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.930379:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63036568 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5367 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.930395:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(72 169) 00010000:00010000:1.0:1780298645.930399:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(72 169) 00010000:00010000:1.0:1780298645.930409:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63036568 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [98304->2203647] (req 98304->102399) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5367 expref: 10 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.930423:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63036568 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [98304->2203647] (req 98304->102399) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5367 expref: 10 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.930432:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279a00, rc 0) 00000100:00100000:1.0:1780298645.930460:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:29360:x1866777609741312:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 103us (191us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298645.930467:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10569 00000100:00100000:2.0:1780298645.931381:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.931412:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609741440 00000100:00100000:3.0:1780298645.931421:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609741440 00002000:00100000:3.0:1780298645.931451:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (98304->102399) req@ffff9cf2425f1500 x1866777609741440/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298645.931459:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2425f1500 with x1866777609741440 ext(98304->102399) 00002000:00010000:3.0:1780298645.931463:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2425f1500 00000100:00100000:3.0:1780298645.931482:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6539 00000100:00100000:3.0:1780298645.931487:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609741440:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298645.931603:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609741440:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 114us (221us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1780298645.931610:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6539 00002000:00100000:3.0:1780298645.931616:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (98304->102399) req@ffff9cf2425f1500 x1866777609741440/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/4536 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298645.931632:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2425f1500 with x1866777609741440 ext(98304->102399) 00002000:00010000:3.0:1780298645.931636:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2425f1500 00000100:00100000:2.0:1780298645.932782:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.932879:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609741568 00000100:00100000:3.0:1780298645.932886:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609741568 00000100:00100000:2.0:1780298645.932911:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.932954:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609741696 00000100:00100000:3.0:1780298645.932959:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609741696 00000100:00100000:0.0:1780298645.932967:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10570 00000100:00100000:0.0:1780298645.932979:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609741568:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.932992:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298645.932999:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10571 00010000:00010000:0.0:1780298645.933008:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e6303656f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d536e expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.933017:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556ab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609741696:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.933038:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.933053:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f2c2b189/0xaf35132e63036576 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5375 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.933074:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605124224:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.933107:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244adbb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605124352:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.933184:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4300 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.933197:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2418563c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.934067:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605124224:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.934081:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244adbb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605124352:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.934097:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e6303656f lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d536e expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.934107:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f2c2b189/0xaf35132e63036576 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5375 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.934174:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e6303656f lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d536e expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.934175:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f2c2b189/0xaf35132e63036576 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5375 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.934215:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92600, rc 0) 00010000:00010000:0.0:1780298645.934222:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3800, rc 0) 00000100:00100000:0.0:1780298645.934265:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609741568:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1287us (1481us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.934281:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10570 00000100:00100000:2.0:1780298645.934307:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556ab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609741696:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1297us (1394us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.934318:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10571 00000100:00100000:2.0:1780298645.935459:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.935489:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.935501:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609741952 00000100:00100000:1.0:1780298645.935509:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609741952 00000100:00100000:3.0:1780298645.935524:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609741824 00000100:00100000:3.0:1780298645.935531:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609741824 00000100:00100000:1.0:1780298645.935561:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10572 00000100:00100000:1.0:1780298645.935576:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609741952:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298645.935585:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10573 00010000:00010000:1.0:1780298645.935590:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298645.935602:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556aad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609741824:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.935606:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e6303657d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5383 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.935615:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.935627:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e63036584 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d537c expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.935656:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254889c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605124480:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.935661:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244adbb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605124608:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.935732:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.935733:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2418563c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.936622:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244adbb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605124608:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.936650:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e63036584 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d537c expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.936662:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254889c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605124480:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.936670:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e63036584 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d537c expref: 9 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.936683:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93600, rc 0) 00010000:00010000:0.0:1780298645.936690:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e6303657d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5383 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.936716:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e6303657d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5383 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.936723:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556aad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609741824:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1126us (1231us total) trans 0 rc 301/301 00010000:00010000:0.0:1780298645.936732:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279200, rc 0) 00000100:00100000:1.0:1780298645.936736:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10573 00000100:00100000:0.0:1780298645.936762:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609741952:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1192us (1302us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.936770:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10572 00000100:00100000:2.0:1780298645.937861:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.937893:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.937915:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609742208 00000100:00100000:0.0:1780298645.937921:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609742208 00000100:00100000:1.0:1780298645.937941:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609742080 00000100:00100000:1.0:1780298645.937946:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609742080 00000100:00100000:0.0:1780298645.937957:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10574 00000100:00100000:0.0:1780298645.937965:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556aa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609742208:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.937974:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.937982:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008361787d/0xaf35132e6303658b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5391 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.937990:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10575 00000100:00100000:1.0:1780298645.938000:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2400:x1866777609742080:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.938007:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.938016:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63036592 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d538a expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.938020:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605124736:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.938053:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254888a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605124864:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.938088:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.938089:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4300 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.938995:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605124736:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.939008:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254888a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605124864:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.939017:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008361787d/0xaf35132e6303658b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5391 expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.939025:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63036592 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d538a expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.939053:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63036592 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d538a expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.939057:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008361787d/0xaf35132e6303658b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5391 expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.939074:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278a00, rc 0) 00010000:00010000:0.0:1780298645.939077:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2800, rc 0) 00000100:00100000:1.0:1780298645.939108:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2400:x1866777609742080:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1111us (1214us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.939115:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556aa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609742208:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1148us (1252us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.939119:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10575 00000100:00100000:0.0:1780298645.939153:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10574 00000100:00100000:2.0:1780298645.940726:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.940764:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609742336 00000100:00100000:1.0:1780298645.940770:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609742336 00000100:00100000:1.0:1780298645.940796:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10576 00000100:00100000:1.0:1780298645.940802:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609742336:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.940810:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.940818:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002ff2dd4d/0xaf35132e63036599 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5398 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.940833:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(61 122) 00010000:00010000:1.0:1780298645.940837:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(61 122) 00010000:00010000:1.0:1780298645.940843:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002ff2dd4d/0xaf35132e63036599 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [176128->1384447] (req 176128->180223) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5398 expref: 10 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.940854:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002ff2dd4d/0xaf35132e63036599 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [176128->1384447] (req 176128->180223) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5398 expref: 10 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.940860:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278800, rc 0) 00000100:00100000:1.0:1780298645.940884:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609742336:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 82us (159us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298645.940891:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10576 00000100:00100000:2.0:1780298645.941902:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.941947:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609742464 00000100:00100000:1.0:1780298645.941952:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609742464 00002000:00100000:1.0:1780298645.941984:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x12:0x0] (176128->180223) req@ffff9cf2556a8e00 x1866777609742464/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.941996:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2556a8e00 with x1866777609742464 ext(176128->180223) 00002000:00010000:1.0:1780298645.942000:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2556a8e00 00000100:00100000:1.0:1780298645.942024:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6540 00000100:00100000:1.0:1780298645.942033:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556a8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609742464:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298645.942221:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556a8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609742464:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 189us (319us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298645.942231:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6540 00002000:00100000:1.0:1780298645.942238:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x12:0x0] (176128->180223) req@ffff9cf2556a8e00 x1866777609742464/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/4536 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.942247:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2556a8e00 with x1866777609742464 ext(176128->180223) 00002000:00010000:1.0:1780298645.942250:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2556a8e00 00000100:00100000:2.0:1780298645.943322:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.943352:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609742592 00000100:00100000:0.0:1780298645.943356:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609742592 00002000:00100000:0.0:1780298645.943376:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x12:0x0] (180224->184319) req@ffff9cf2556ab800 x1866777609742592/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298645.943383:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2556ab800 with x1866777609742592 ext(180224->184319) 00002000:00010000:0.0:1780298645.943386:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2556ab800 00000100:00100000:0.0:1780298645.943402:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6541 00000100:00100000:0.0:1780298645.943408:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2556ab800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609742592:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298645.943488:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2556ab800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609742592:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 80us (168us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298645.943498:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6541 00002000:00100000:0.0:1780298645.943503:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x280000401:0x12:0x0] (180224->184319) req@ffff9cf2556ab800 x1866777609742592/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:351/0 lens 488/4536 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298645.943509:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2556ab800 with x1866777609742592 ext(180224->184319) 00002000:00010000:0.0:1780298645.943512:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2556ab800 00000100:00100000:2.0:1780298645.944510:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.944542:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609742720 00000100:00100000:2.0:1780298645.944544:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.944546:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609742720 00000100:00100000:0.0:1780298645.944573:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609742848 00000100:00100000:0.0:1780298645.944575:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609742848 00000100:00100000:0.0:1780298645.944607:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10577 00000100:00100000:1.0:1780298645.944618:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10578 00000100:00100000:0.0:1780298645.944619:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343966a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609742720:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.944631:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298645.944638:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343964000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609742848:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.944646:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.944646:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e630365a0 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d539f expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.944658:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b25cdf87/0xaf35132e630365a7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d53a6 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.944700:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254888a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605125120:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.944701:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605124992:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.944773:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e900 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.944773:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4300 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.945518:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254888a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605125120:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.945526:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605124992:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.945542:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b25cdf87/0xaf35132e630365a7 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d53a6 expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.945547:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e630365a0 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d539f expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.945565:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b25cdf87/0xaf35132e630365a7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d53a6 expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.945575:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278200, rc 0) 00010000:00010000:0.0:1780298645.945575:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e630365a0 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d539f expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.945588:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3c00, rc 0) 00000100:00100000:1.0:1780298645.945605:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343964000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609742848:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 973us (1061us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.945613:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10578 00000100:00100000:0.0:1780298645.945617:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343966a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609742720:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 997us (1100us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.945623:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10577 00000100:00100000:2.0:1780298645.946577:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.946619:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609742976 00000100:00100000:0.0:1780298645.946624:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609742976 00000100:00100000:0.0:1780298645.946654:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10579 00000100:00100000:0.0:1780298645.946661:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343964a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2397:x1866777609742976:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.946669:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.946679:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e630365ae lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d53ad expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.946704:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605125248:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.946715:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.946751:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4300 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.946761:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609743104 00000100:00100000:1.0:1780298645.946766:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609743104 00000100:00100000:1.0:1780298645.946801:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10580 00000100:00100000:1.0:1780298645.946809:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343965f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609743104:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.946816:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.946824:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e630365b5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d53b4 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.946854:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254888a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605125376:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.946883:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e900 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.947664:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605125248:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.947689:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e630365ae lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d53ad expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.947709:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e630365ae lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d53ad expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.947722:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2000, rc 0) 00000100:00100000:0.0:1780298645.947761:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343964a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2397:x1866777609742976:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1099us (1183us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.947770:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10579 00000100:00100000:1.0:1780298645.947835:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254888a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605125376:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.947856:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e630365b5 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d53b4 expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.947875:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e630365b5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d53b4 expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.947887:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278400, rc 0) 00000100:00100000:1.0:1780298645.947917:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343965f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609743104:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1108us (1203us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.947925:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10580 00000100:00100000:3.0:1780298645.948958:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.949010:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609743232 00000100:00100000:0.0:1780298645.949017:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609743232 00000100:00100000:0.0:1780298645.949066:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6542 00000100:00100000:0.0:1780298645.949075:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609743232:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298645.949398:0:19950:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4412060 -> 4412060 00002000:00010000:0.0:1780298645.949402:0:19950:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298645.949404:0:19950:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298645.949407:0:19950:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298645.949409:0:19950:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 536 -> 552 00000100:00100000:0.0:1780298645.949444:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609743232:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 371us (494us total) trans 4294969805 rc 0/0 00000100:00100000:0.0:1780298645.949451:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6542 00000100:00100000:2.0:1780298645.950507:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.950599:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609743360 00000100:00100000:3.0:1780298645.950609:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609743360 00000100:00100000:2.0:1780298645.950645:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.950670:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609743488 00000100:00100000:3.0:1780298645.950675:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609743488 00000100:00100000:1.0:1780298645.950723:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10581 00000100:00100000:0.0:1780298645.950733:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10582 00000100:00100000:1.0:1780298645.950739:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343966d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609743360:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298645.950749:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343966300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609743488:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.950750:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.950762:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.950767:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e630365bc lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d53bb expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.950775:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e630365c3 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d53c2 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.950822:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254888a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605125632:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.950822:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605125504:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.950892:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4300 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.950893:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e900 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.951743:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605125504:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.951762:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254888a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605125632:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.951764:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e630365c3 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d53c2 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.951777:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e630365bc lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d53bb expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.951780:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e630365c3 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d53c2 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.951789:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e630365bc lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d53bb expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.951792:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3800, rc 0) 00010000:00010000:1.0:1780298645.951796:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279c00, rc 0) 00000100:00100000:1.0:1780298645.951821:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343966d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609743360:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1089us (1315us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.951821:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343966300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609743488:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1079us (1177us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.951829:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10582 00000100:00100000:1.0:1780298645.951833:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10581 00000100:00100000:2.0:1780298645.952980:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.953008:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.953032:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609743616 00000100:00100000:0.0:1780298645.953040:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609743616 00000100:00100000:1.0:1780298645.953061:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609743744 00000100:00100000:1.0:1780298645.953068:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609743744 00000100:00100000:0.0:1780298645.953082:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10583 00000100:00100000:0.0:1780298645.953091:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343964380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2397:x1866777609743616:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.953099:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.953107:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008361787d/0xaf35132e630365ca lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d53c9 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.953116:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10584 00000100:00100000:2.0:1780298645.953162:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf343965880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609743744:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.953171:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298645.953179:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605125760:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.953181:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b4561703/0xaf35132e630365d1 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d53d0 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.953224:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481ff480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605125888:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.953234:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4300 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.953256:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c840 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.954033:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605125760:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.954053:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481ff480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605125888:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.954055:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008361787d/0xaf35132e630365ca lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d53c9 expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.954070:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008361787d/0xaf35132e630365ca lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d53c9 expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.954079:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b4561703/0xaf35132e630365d1 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d53d0 expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.954079:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2800, rc 0) 00010000:00010000:1.0:1780298645.954096:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b4561703/0xaf35132e630365d1 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d53d0 expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.954107:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b600, rc 0) 00000100:00100000:0.0:1780298645.954117:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343964380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2397:x1866777609743616:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1026us (1136us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.954147:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10583 00000100:00100000:1.0:1780298645.954158:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf343965880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609743744:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1031us (1147us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.954166:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10584 00000100:00100000:0.0:1780298645.955248:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.955288:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609743872 00000100:00100000:3.0:1780298645.955299:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609743872 00000100:00100000:0.0:1780298645.955321:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.955354:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609744000 00000100:00100000:3.0:1780298645.955357:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609744000 00000100:00100000:1.0:1780298645.955379:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10585 00000100:00100000:1.0:1780298645.955407:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f736680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609743872:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.955420:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.955431:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e630365d8 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d53d7 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.955437:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10586 00000100:00100000:3.0:1780298645.955449:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f736d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609744000:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.955464:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298645.955478:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254888a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605126016:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.955482:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f2c2b189/0xaf35132e630365df lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d53de expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.955539:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244adb100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605126144:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.955541:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e900 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.955608:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.956425:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254888a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605126016:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.956448:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e630365d8 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d53d7 expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.956451:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244adb100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605126144:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.956483:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e630365d8 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d53d7 expref: 9 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.956492:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f2c2b189/0xaf35132e630365df lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d53de expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.956515:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279e00, rc 0) 00010000:00010000:3.0:1780298645.956539:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f2c2b189/0xaf35132e630365df lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d53de expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.956552:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92600, rc 0) 00000100:00100000:1.0:1780298645.956561:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f736680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2399:x1866777609743872:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1169us (1312us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.956571:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10585 00000100:00100000:3.0:1780298645.956595:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f736d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609744000:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1144us (1268us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.956606:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10586 00000100:00100000:2.0:1780298645.957646:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.957691:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609744128 00000100:00100000:1.0:1780298645.957698:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609744128 00000100:00100000:1.0:1780298645.957740:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10587 00000100:00100000:1.0:1780298645.957751:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2400:x1866777609744128:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298645.957757:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:1.0:1780298645.957764:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.957776:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e630365e6 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d53e5 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.957820:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254888a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605126272:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.957836:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609744256 00000100:00100000:3.0:1780298645.957843:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609744256 00000100:00100000:1.0:1780298645.957881:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e900 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.957896:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10588 00000100:00100000:3.0:1780298645.957910:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609744256:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.957921:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.957933:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f8ed852/0xaf35132e630365ed lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d53ec expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.958003:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244adb100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605126400:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.958040:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.958645:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254888a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605126272:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.958668:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e630365e6 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d53e5 expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.958685:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e630365e6 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d53e5 expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.958696:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278400, rc 0) 00000100:00100000:1.0:1780298645.958733:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2400:x1866777609744128:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 985us (1089us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.958740:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10587 00000100:00100000:3.0:1780298645.958932:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244adb100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605126400:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298645.958956:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002f8ed852/0xaf35132e630365ed lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d53ec expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.958972:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f8ed852/0xaf35132e630365ed lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d53ec expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.958987:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92c00, rc 0) 00000100:00100000:3.0:1780298645.959019:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609744256:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1115us (1266us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.959027:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10588 00000100:00100000:2.0:1780298645.960163:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.960201:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609744384 00000100:00100000:0.0:1780298645.960207:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609744384 00000100:00100000:0.0:1780298645.960250:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10589 00000100:00100000:3.0:1780298645.960251:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.960274:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609744384:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.960286:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298645.960289:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609744512 00000100:00100000:1.0:1780298645.960295:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609744512 00010000:00010000:0.0:1780298645.960297:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e630365f4 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d53f3 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.960327:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10590 00000100:00100000:1.0:1780298645.960336:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609744512:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298645.960338:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f734000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605126528:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.960347:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.960358:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b25cdf87/0xaf35132e630365fb lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d53fa expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.960390:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254888a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605126656:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.960391:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4300 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.960426:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e900 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.961090:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f734000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605126528:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.961106:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e630365f4 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d53f3 expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.961118:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e630365f4 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d53f3 expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.961148:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3c00, rc 0) 00000100:00100000:0.0:1780298645.961173:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609744384:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 912us (1013us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.961179:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10589 00000100:00100000:1.0:1780298645.961227:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254888a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605126656:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.961243:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b25cdf87/0xaf35132e630365fb lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d53fa expref: 8 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.961257:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b25cdf87/0xaf35132e630365fb lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d53fa expref: 8 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.961281:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278200, rc 0) 00000100:00100000:1.0:1780298645.961310:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609744512:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 977us (1063us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.961318:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10590 00000100:00100000:2.0:1780298645.962091:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.962149:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609744640 00000100:00100000:0.0:1780298645.962154:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609744640 00000100:00100000:0.0:1780298645.962182:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10591 00000100:00100000:0.0:1780298645.962191:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609744640:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.962200:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.962209:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e63036602 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5401 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.962216:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.962247:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f734000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605126784:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.962248:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609744768 00000100:00100000:1.0:1780298645.962254:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609744768 00000100:00100000:0.0:1780298645.962294:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4300 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.962306:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10592 00000100:00100000:1.0:1780298645.962317:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609744768:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.962326:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.962335:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005b92bd2b/0xaf35132e63036609 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5408 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.962366:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254888a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605126912:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.962402:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e900 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.962926:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f734000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605126784:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.962942:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e63036602 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5401 expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.962957:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e63036602 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5401 expref: 12 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.962978:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3800, rc 0) 00000100:00100000:0.0:1780298645.963010:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609744640:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 821us (920us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.963018:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10591 00000100:00100000:1.0:1780298645.963087:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254888a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605126912:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.963103:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005b92bd2b/0xaf35132e63036609 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5408 expref: 8 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.963144:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005b92bd2b/0xaf35132e63036609 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5408 expref: 8 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.963156:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279c00, rc 0) 00000100:00100000:1.0:1780298645.963186:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609744768:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 872us (972us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.963194:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10592 00000100:00100000:2.0:1780298645.963986:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.964024:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609744896 00000100:00100000:3.0:1780298645.964028:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609744896 00000100:00100000:3.0:1780298645.964068:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6543 00000100:00100000:3.0:1780298645.964077:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609744896:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298645.964359:0:8363:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4412060 -> 4412060 00002000:00010000:2.0:1780298645.964362:0:8363:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:2.0:1780298645.964364:0:8363:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298646 -> 1780298646 00002000:00010000:2.0:1780298645.964366:0:8363:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:2.0:1780298645.964367:0:8363:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 552 -> 568 00000100:00100000:2.0:1780298645.964401:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609744896:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 325us (413us total) trans 4294969806 rc 0/0 00000100:00100000:2.0:1780298645.964408:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6543 00000100:00100000:3.0:1780298645.965254:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.965325:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609745024 00000100:00100000:2.0:1780298645.965329:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609745024 00000100:00100000:2.0:1780298645.965358:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10593 00000100:00100000:2.0:1780298645.965365:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2398:x1866777609745024:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.965372:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298645.965378:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:2.0:1780298645.965381:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63036610 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d540f expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.965413:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609745152 00000100:00100000:1.0:1780298645.965418:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609745152 00000100:00100000:2.0:1780298645.965430:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481ff800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605127040:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.965449:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10594 00000100:00100000:1.0:1780298645.965456:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ada300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609745152:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.965464:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.965472:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e63036617 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5416 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.965486:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.965497:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254888a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605127168:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.965526:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e900 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.966355:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481ff800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605127040:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.966359:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254888a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605127168:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.966381:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63036610 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d540f expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.966383:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e63036617 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5416 expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.966413:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63036610 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d540f expref: 9 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.966414:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e5b3111/0xaf35132e63036617 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5416 expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.966435:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9ac00, rc 0) 00010000:00010000:1.0:1780298645.966445:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278400, rc 0) 00000100:00100000:1.0:1780298645.966471:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ada300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609745152:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1015us (1094us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.966475:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2398:x1866777609745024:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1104us (1215us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.966484:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10594 00000100:00100000:2.0:1780298645.966494:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10593 00000100:00100000:3.0:1780298645.967624:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.967660:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.967682:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609745280 00000100:00100000:2.0:1780298645.967688:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609745280 00000100:00100000:0.0:1780298645.967692:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609745408 00000100:00100000:0.0:1780298645.967695:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609745408 00000100:00100000:0.0:1780298645.967728:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10595 00000100:00100000:2.0:1780298645.967734:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10596 00000100:00100000:0.0:1780298645.967738:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609745408:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298645.967740:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2397:x1866777609745280:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.967749:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.967749:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.967759:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d71a6aba/0xaf35132e6303661e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d541d expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.967759:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008361787d/0xaf35132e63036625 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5424 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.967793:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481ff800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605127424:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.967793:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f734000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605127296:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.967837:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.967837:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4300 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.969152:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481ff800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605127424:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.969157:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f734000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605127296:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298645.969176:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d71a6aba/0xaf35132e6303661e lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d541d expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.969188:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008361787d/0xaf35132e63036625 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5424 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.969191:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d71a6aba/0xaf35132e6303661e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d541d expref: 9 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298645.969209:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9bc00, rc 0) 00010000:00010000:0.0:1780298645.969219:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008361787d/0xaf35132e63036625 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5424 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.969231:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2800, rc 0) 00000100:00100000:2.0:1780298645.969236:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:2397:x1866777609745280:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1496us (1614us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298645.969246:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10596 00000100:00100000:0.0:1780298645.969257:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609745408:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1521us (1596us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.969263:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10595 00000100:00100000:3.0:1780298645.970684:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.970728:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609745536 00000100:00100000:2.0:1780298645.970735:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609745536 00002000:00100000:2.0:1780298645.970774:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x12:0x0] (2772992->2777087) req@ffff9cf244ad9880 x1866777609745536/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.970785:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244ad9880 with x1866777609745536 ext(2772992->2777087) 00002000:00010000:2.0:1780298645.970790:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf244ad9880 00000100:00100000:2.0:1780298645.970815:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6544 00000100:00100000:2.0:1780298645.970824:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609745536:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298645.971030:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609745536:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 207us (348us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298645.971037:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6544 00002000:00100000:2.0:1780298645.971043:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x12:0x0] (2772992->2777087) req@ffff9cf244ad9880 x1866777609745536/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/4536 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298645.971052:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244ad9880 with x1866777609745536 ext(2772992->2777087) 00002000:00010000:2.0:1780298645.971056:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf244ad9880 00000100:00100000:3.0:1780298645.972490:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.972548:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609745664 00000100:00100000:1.0:1780298645.972555:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609745664 00002000:00100000:1.0:1780298645.972585:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x12:0x0] (2781184->2785279) req@ffff9cf244adb480 x1866777609745664/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.972596:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244adb480 with x1866777609745664 ext(2781184->2785279) 00002000:00010000:1.0:1780298645.972600:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf244adb480 00000100:00100000:1.0:1780298645.972625:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6545 00000100:00100000:1.0:1780298645.972639:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609745664:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298645.972755:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609745664:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 118us (267us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298645.972762:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6545 00002000:00100000:1.0:1780298645.972768:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x12:0x0] (2781184->2785279) req@ffff9cf244adb480 x1866777609745664/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/4536 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298645.972778:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244adb480 with x1866777609745664 ext(2781184->2785279) 00002000:00010000:1.0:1780298645.972781:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf244adb480 00000100:00100000:3.0:1780298645.973877:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.973899:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.973926:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609745792 00000100:00100000:0.0:1780298645.973932:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609745792 00000100:00100000:1.0:1780298645.973941:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609745920 00000100:00100000:1.0:1780298645.973946:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609745920 00000100:00100000:0.0:1780298645.973976:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10597 00000100:00100000:1.0:1780298645.973988:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10598 00000100:00100000:1.0:1780298645.973999:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609745920:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298645.974005:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609745792:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.974010:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.974016:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.974028:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005b92bd2b/0xaf35132e6303662c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5432 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.974030:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e63036633 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d542b expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.974095:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254888a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605127552:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.974097:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f734000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605127680:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.974195:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e900 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.974195:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4300 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.975174:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254888a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605127552:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.975189:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f734000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605127680:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.975205:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005b92bd2b/0xaf35132e6303662c lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5432 expref: 8 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.975214:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e63036633 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d542b expref: 13 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.975224:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005b92bd2b/0xaf35132e6303662c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5432 expref: 8 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.975248:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279c00, rc 0) 00010000:00010000:0.0:1780298645.975254:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e63036633 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d542b expref: 13 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.975277:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3c00, rc 0) 00000100:00100000:1.0:1780298645.975294:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609745920:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1298us (1393us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.975303:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10598 00000100:00100000:0.0:1780298645.975304:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609745792:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1317us (1425us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.975314:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10597 00000100:00100000:3.0:1780298645.976484:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.976546:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609746048 00000100:00100000:1.0:1780298645.976554:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609746048 00000100:00100000:3.0:1780298645.976594:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.976600:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10599 00000100:00100000:1.0:1780298645.976612:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609746048:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298645.976621:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609746176 00000100:00100000:2.0:1780298645.976628:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609746176 00010000:00010000:1.0:1780298645.976628:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.976640:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e6303663a lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5439 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298645.976672:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10600 00000100:00100000:1.0:1780298645.976675:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254888a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605127808:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.976685:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e427050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609746176:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.976696:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.976707:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e63036641 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5440 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.976733:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e900 going to sleep for 11 seconds 00000100:00100000:2.0:1780298645.976753:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481ff800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605127936:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.976797:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.977379:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254888a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605127808:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.977398:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e6303663a lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5439 expref: 13 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.977414:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e6303663a lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5439 expref: 13 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.977423:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278400, rc 0) 00000100:00100000:1.0:1780298645.977478:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609746048:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 865us (999us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.977485:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10599 00000100:00100000:0.0:1780298645.977546:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481ff800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605127936:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.977568:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e63036641 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5440 expref: 8 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.977584:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e63036641 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5440 expref: 8 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.977598:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9ac00, rc 0) 00000100:00100000:0.0:1780298645.977638:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e427050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609746176:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 958us (1045us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.977646:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10600 00000100:00100000:3.0:1780298645.978552:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.978593:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609746304 00000100:00100000:0.0:1780298645.978598:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609746304 00000100:00100000:0.0:1780298645.978633:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10601 00000100:00100000:0.0:1780298645.978642:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+8:29360:x1866777609746304:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.978651:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.978661:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e63036648 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5447 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.978680:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63036568 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [98304->2203647] (req 98304->102399) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5367 expref: 11 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.978690:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(72 169) 00010000:00010000:0.0:1780298645.978695:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(73 169) 00010000:00010000:0.0:1780298645.978700:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e63036648 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [2170880->2179071] (req 2170880->2179071) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5447 expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.978721:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63036568 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [98304->2203647] (req 98304->102399) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5367 expref: 11 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.978735:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63036568 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [98304->2203647] (req 98304->102399) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5367 expref: 11 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.978747:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605128064:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298645.978791:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4300 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.979551:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605128064:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298645.979567:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e63036648 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [2170880->2179071] (req 2170880->2179071) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5447 expref: 9 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.979578:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3800, rc 0) 00000100:00100000:0.0:1780298645.979613:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609746304:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 973us (1063us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298645.979624:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10601 00000100:00100000:2.0:1780298645.979875:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.979914:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609746432 00000100:00100000:0.0:1780298645.979919:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609746432 00002000:00100000:0.0:1780298645.979948:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (98304->102399) req@ffff9cf3481ffb80 x1866777609746432/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 4584/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298645.979959:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481ffb80 with x1866777609746432 ext(98304->102399) 00010000:00010000:0.0:1780298645.979964:0:19928:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63036568 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [98304->2203647] (req 98304->102399) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5367 expref: 12 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.979976:0:19928:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63036568 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [98304->2203647] (req 98304->102399) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5367 expref: 12 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1780298645.979983:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf3481ffb80 00000100:00100000:0.0:1780298645.980008:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 223 00000100:00100000:0.0:1780298645.980016:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481ffb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609746432:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:0.0:1780298645.980523:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481ffb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609746432:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 507us (648us total) trans 4294969807 rc 0/0 00000100:00100000:0.0:1780298645.980531:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 223 00002000:00100000:0.0:1780298645.980537:0:19928:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x2c0000401:0x11:0x0] (98304->102399) req@ffff9cf3481ffb80 x1866777609746432/t4294969807(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 4584/448 e 0 to 0 dl 1780298656 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298645.980546:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481ffb80 with x1866777609746432 ext(98304->102399) 00010000:00010000:0.0:1780298645.980552:0:19928:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63036568 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [98304->2203647] (req 98304->102399) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5367 expref: 14 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.980570:0:19928:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63036568 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [98304->2203647] (req 98304->102399) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5367 expref: 14 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1780298645.980576:0:19928:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf3481ffb80 00000100:00100000:3.0:1780298645.981319:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298645.981367:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609746560 00000100:00100000:1.0:1780298645.981373:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609746560 00010000:00010000:1.0:1780298645.981395:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63036568 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [98304->2203647] (req 98304->102399) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5367 expref: 14 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.981428:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15332 00000100:00100000:1.0:1780298645.981438:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adb100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2397:x1866777609746560:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298645.981449:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298645.981461:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63036568 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [98304->2203647] (req 98304->102399) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5367 expref: 14 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.981479:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63036568 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [98304->2203647] (req 98304->102399) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5367 expref: 14 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.981505:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63036568 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [98304->2203647] (req 98304->102399) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5367 expref: 14 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.981518:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(73 169) 00010000:00010000:1.0:1780298645.981522:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(73 169) 00010000:00010000:1.0:1780298645.981530:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e63036648 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [2170880->2203647] (req 2170880->2179071) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5447 expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.981539:0:6463:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e63036648 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [2170880->2203647] (req 2170880->2179071) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5447 expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.981573:0:6463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e63036648 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [2170880->2203647] (req 2170880->2179071) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5447 expref: 8 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.981585:0:6463:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254888a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6463:1866777605128192:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298645.981640:0:6463:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e900 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.982489:0:6463:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254888a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6463:1866777605128192:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298645.982502:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298645.982537:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adb100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2397:x1866777609746560:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1101us (1216us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298645.982545:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15332 00000100:00100000:3.0:1780298645.982667:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.982722:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609746688 00000100:00100000:0.0:1780298645.982728:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609746688 00000100:00100000:0.0:1780298645.982774:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6546 00000100:00100000:0.0:1780298645.982783:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609746688:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298645.982902:0:19950:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4412060 -> 4412060 00002000:00010000:0.0:1780298645.982905:0:19950:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298645.982907:0:19950:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298645.982908:0:19950:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298645.982910:0:19950:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 568 -> 568 00000100:00100000:0.0:1780298645.982939:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609746688:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 159us (318us total) trans 4294969808 rc 0/0 00000100:00100000:0.0:1780298645.982944:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6546 00000100:00100000:2.0:1780298645.984108:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.984184:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609746816 00000100:00100000:0.0:1780298645.984189:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609746816 00000100:00100000:0.0:1780298645.984221:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10602 00000100:00100000:0.0:1780298645.984231:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2399:x1866777609746816:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.984241:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.984253:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008361787d/0xaf35132e6303664f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d544e expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.984283:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.984306:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605128320:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.984323:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609746944 00000100:00100000:1.0:1780298645.984327:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609746944 00000100:00100000:0.0:1780298645.984366:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4300 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.984367:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10603 00000100:00100000:1.0:1780298645.984379:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ada300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609746944:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.984386:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.984394:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b25cdf87/0xaf35132e63036656 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5455 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.984418:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254888a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605128448:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.984440:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e900 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.985280:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254888a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605128448:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.985295:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b25cdf87/0xaf35132e63036656 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5455 expref: 8 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.985308:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b25cdf87/0xaf35132e63036656 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5455 expref: 8 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.985316:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278200, rc 0) 00000100:00100000:1.0:1780298645.985341:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ada300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609746944:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 967us (1059us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.985345:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10603 00000100:00100000:0.0:1780298645.985878:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605128320:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.985893:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008361787d/0xaf35132e6303664f lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d544e expref: 14 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.985903:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008361787d/0xaf35132e6303664f lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d544e expref: 14 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.985910:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2800, rc 0) 00000100:00100000:0.0:1780298645.985937:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2399:x1866777609746816:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1710us (1831us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.985942:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10602 00000100:00100000:3.0:1780298645.986886:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.986919:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.986946:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609747072 00000100:00100000:0.0:1780298645.986951:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609747072 00000100:00100000:1.0:1780298645.986973:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609747200 00000100:00100000:1.0:1780298645.986979:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609747200 00000100:00100000:0.0:1780298645.986992:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10604 00000100:00100000:0.0:1780298645.987005:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609747072:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.987014:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.987026:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000488a7b88/0xaf35132e6303665d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d545c expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.987030:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10605 00000100:00100000:1.0:1780298645.987045:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609747200:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298645.987058:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298645.987068:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605128576:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.987069:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e63036664 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5463 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.987107:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254888a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605128704:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.987163:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4300 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.987192:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e900 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.988291:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605128576:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.988308:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254888a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605128704:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.988315:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000488a7b88/0xaf35132e6303665d lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d545c expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.988330:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e63036664 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5463 expref: 8 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.988340:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000488a7b88/0xaf35132e6303665d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d545c expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.988350:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e63036664 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5463 expref: 8 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.988354:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2200, rc 0) 00010000:00010000:1.0:1780298645.988366:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279e00, rc 0) 00000100:00100000:0.0:1780298645.988390:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609747072:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1390us (1510us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.988395:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609747200:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1352us (1474us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.988409:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10604 00000100:00100000:1.0:1780298645.988417:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10605 00000100:00100000:2.0:1780298645.989887:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.989935:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609747328 00000100:00100000:0.0:1780298645.989941:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609747328 00002000:00100000:0.0:1780298645.989967:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x12:0x0] (2940928->2949119) req@ffff9cf3481fed80 x1866777609747328/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/0 e 0 to 0 dl 1780298656 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298645.989975:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fed80 with x1866777609747328 ext(2940928->2949119) 00002000:00010000:0.0:1780298645.989978:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf3481fed80 00000100:00100000:0.0:1780298645.990005:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6547 00000100:00100000:0.0:1780298645.990011:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609747328:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298645.990199:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2398:x1866777609747328:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 187us (314us total) trans 0 rc 8192/8192 00000100:00100000:0.0:1780298645.990206:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6547 00002000:00100000:0.0:1780298645.990213:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x12:0x0] (2940928->2949119) req@ffff9cf3481fed80 x1866777609747328/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:351/0 lens 488/8632 e 0 to 0 dl 1780298656 ref 1 fl Complete:/600/0 rc 8192/8192 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298645.990221:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fed80 with x1866777609747328 ext(2940928->2949119) 00002000:00010000:0.0:1780298645.990225:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf3481fed80 00000100:00100000:3.0:1780298645.991452:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.991490:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.991507:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609747584 00000100:00100000:2.0:1780298645.991512:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609747584 00000100:00100000:1.0:1780298645.991540:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609747456 00000100:00100000:2.0:1780298645.991546:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10606 00000100:00100000:1.0:1780298645.991547:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609747456 00000100:00100000:2.0:1780298645.991557:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609747584:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298645.991567:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298645.991579:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e6303666b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5471 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.991594:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10607 00000100:00100000:1.0:1780298645.991603:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609747456:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298645.991618:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605128832:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.991618:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.991634:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63036672 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d546a expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.991672:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254888a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605128960:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298645.991674:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.991708:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e900 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.992611:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605128832:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.992638:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e6303666b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5471 expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.992656:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e6303666b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5471 expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.992669:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9ac00, rc 0) 00000100:00100000:3.0:1780298645.992676:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254888a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605128960:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.992716:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609747584:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1158us (1328us total) trans 0 rc 301/301 00010000:00010000:3.0:1780298645.992721:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63036672 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d546a expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298645.992730:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10606 00010000:00010000:3.0:1780298645.992760:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63036672 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d546a expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.992775:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278200, rc 0) 00000100:00100000:3.0:1780298645.992901:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609747456:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1226us (1341us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.992909:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10607 00000100:00100000:2.0:1780298645.993883:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298645.993915:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298645.993928:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609747712 00000100:00100000:0.0:1780298645.993936:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609747712 00000100:00100000:1.0:1780298645.993952:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609747840 00000100:00100000:1.0:1780298645.993958:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609747840 00000100:00100000:0.0:1780298645.993986:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10608 00000100:00100000:0.0:1780298645.993998:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609747712:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298645.994000:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10609 00010000:00010000:0.0:1780298645.994014:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298645.994020:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609747840:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.994029:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e63036679 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5478 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.994030:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298645.994041:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e63036680 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d547f expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.994075:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605129088:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298645.994078:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25488aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605129216:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.994175:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba49c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.994180:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298645.994999:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25488aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605129216:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.995016:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605129088:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298645.995022:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e63036680 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d547f expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.995036:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e63036680 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d547f expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.995040:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e63036679 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5478 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298645.995049:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279a00, rc 0) 00010000:00010000:0.0:1780298645.995062:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e63036679 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5478 expref: 11 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.995074:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3c00, rc 0) 00000100:00100000:1.0:1780298645.995079:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609747840:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1068us (1163us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298645.995085:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10609 00000100:00100000:0.0:1780298645.995102:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2397:x1866777609747712:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1106us (1222us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.995110:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10608 00000100:00100000:2.0:1780298645.996289:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.996334:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609747968 00000100:00100000:3.0:1780298645.996345:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609747968 00000100:00100000:3.0:1780298645.996473:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10610 00000100:00100000:2.0:1780298645.996483:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.996488:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609747968:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.996510:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.996528:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e63036687 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5486 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298645.996529:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609748096 00000100:00100000:0.0:1780298645.996537:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609748096 00000100:00100000:0.0:1780298645.996574:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10611 00000100:00100000:0.0:1780298645.996587:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609748096:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298645.996602:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298645.996612:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0cae770/0xaf35132e6303668e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d548d expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298645.996637:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244adbb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605129344:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.996653:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605129472:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298645.996734:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba49c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298645.996735:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:0.0:1780298645.997702:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605129472:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298645.997732:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244adbb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605129344:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298645.997739:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e0cae770/0xaf35132e6303668e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d548d expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.997755:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e63036687 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5486 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.997770:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e63036687 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5486 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.997779:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92600, rc 0) 00010000:00010000:0.0:1780298645.997779:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0cae770/0xaf35132e6303668e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d548d expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298645.997798:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2600, rc 0) 00000100:00100000:3.0:1780298645.997818:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609747968:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1337us (1529us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298645.997824:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10610 00000100:00100000:0.0:1780298645.997834:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609748096:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1250us (1350us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298645.997845:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10611 00000100:00100000:2.0:1780298645.998951:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.998980:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609748224 00000100:00100000:3.0:1780298645.998986:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609748224 00000100:00100000:3.0:1780298645.999014:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10612 00000100:00100000:3.0:1780298645.999020:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609748224:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298645.999028:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298645.999036:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002f8ed852/0xaf35132e63036695 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5494 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.999052:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(73 169) 00010000:00010000:3.0:1780298645.999056:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(73 169) 00010000:00010000:3.0:1780298645.999065:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002f8ed852/0xaf35132e63036695 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [520192->2170879] (req 520192->528383) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5494 expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.999081:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002f8ed852/0xaf35132e63036695 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [520192->2170879] (req 520192->528383) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5494 expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298645.999096:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92c00, rc 0) 00000100:00100000:3.0:1780298645.999165:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:29360:x1866777609748224:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 144us (217us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298645.999171:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10612 00000100:00100000:2.0:1780298645.999889:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298645.999918:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609748352 00000100:00100000:3.0:1780298645.999923:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609748352 00000100:00100000:3.0:1780298645.999962:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6548 00000100:00100000:3.0:1780298645.999970:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609748352:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:3.0:1780298646.000182:0:16972:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4412060 -> 4412060 00002000:00010000:3.0:1780298646.000186:0:16972:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:3.0:1780298646.000189:0:16972:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298646 -> 1780298646 00002000:00010000:3.0:1780298646.000191:0:16972:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:3.0:1780298646.000193:0:16972:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 568 -> 584 00000100:00100000:2.0:1780298646.000254:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2397:x1866777609748352:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 285us (365us total) trans 4294969809 rc 0/0 00000100:00100000:2.0:1780298646.000264:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6548 00000100:00100000:2.0:1780298646.001347:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.001388:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609748480 00000100:00100000:2.0:1780298646.001395:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.001400:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609748480 00000100:00100000:0.0:1780298646.001439:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609748608 00000100:00100000:0.0:1780298646.001441:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609748608 00000100:00100000:1.0:1780298646.001458:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10613 00000100:00100000:1.0:1780298646.001478:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609748480:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298646.001482:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10614 00000100:00100000:2.0:1780298646.001496:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481ff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609748608:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.001498:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00080000:3.0:1780298646.001505:0:6473:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1780298646.001505:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.001512:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e6303669c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d549b expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.001516:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0a0f245/0xaf35132e630366a3 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d54a2 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.001567:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481ff480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605129600:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.001577:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25488aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605129728:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.001628:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6cf00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.001628:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.002409:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481ff480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605129600:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.002409:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25488aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605129728:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298646.002430:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e0a0f245/0xaf35132e630366a3 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54a2 expref: 8 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.002431:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e6303669c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d549b expref: 14 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.002447:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e6303669c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d549b expref: 14 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.002451:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0a0f245/0xaf35132e630366a3 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54a2 expref: 8 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.002460:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b200, rc 0) 00010000:00010000:1.0:1780298646.002460:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279e00, rc 0) 00000100:00100000:2.0:1780298646.002489:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481ff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609748608:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 997us (1098us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.002489:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609748480:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1021us (1142us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.002496:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10613 00000100:00100000:2.0:1780298646.002501:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10614 00000100:00100000:3.0:1780298646.003736:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.003756:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.003788:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609748736 00000100:00100000:1.0:1780298646.003793:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609748864 00000100:00100000:0.0:1780298646.003797:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609748736 00000100:00100000:1.0:1780298646.003800:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609748864 00000100:00100000:0.0:1780298646.003857:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10615 00000100:00100000:3.0:1780298646.003866:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10616 00000100:00100000:0.0:1780298646.003873:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2399:x1866777609748736:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298646.003884:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609748864:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298646.003902:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.003903:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298646.003911:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb0d13ab/0xaf35132e630366aa lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d54b0 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.003921:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008361787d/0xaf35132e630366b1 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d54a9 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.003943:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605129856:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.003979:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605129984:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298646.003989:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.004041:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba49c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.004724:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605129856:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.004754:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d13ab/0xaf35132e630366aa lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54b0 expref: 8 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.004780:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb0d13ab/0xaf35132e630366aa lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54b0 expref: 8 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.004799:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92400, rc 0) 00000100:00100000:0.0:1780298646.004846:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609748864:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 970us (1087us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.004855:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10616 00000100:00100000:1.0:1780298646.004906:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605129984:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298646.004931:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008361787d/0xaf35132e630366b1 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54a9 expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.004957:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008361787d/0xaf35132e630366b1 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54a9 expref: 14 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.004970:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2800, rc 0) 00000100:00100000:1.0:1780298646.005006:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2399:x1866777609748736:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1140us (1269us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.005014:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10615 00000100:00100000:3.0:1780298646.005929:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.005969:0:17567:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609749120 00000100:00100000:2.0:1780298646.005974:0:17567:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609749120 00000100:00080000:3.0:1780298646.006014:0:6473:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1780298646.006029:0:17567:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62094 00000100:00100000:2.0:1780298646.006038:0:17567:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+50:29360:x1866777609749120:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:2.0:1780298646.006081:0:17567:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298646.006098:0:17567:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93c00/0xaf35132e6303631c lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 24 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5122 expref: 50 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.006107:0:17567:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298646.006110:0:17567:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298646.006305:0:17567:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b200/0xaf35132e630366b8 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 23 type: IBT flags: 0x40000000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.006317:0:17567:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2e00/0xaf35132e630363ee lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 23 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d51ed expref: 44 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.006325:0:17567:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b200/0xaf35132e630366b8 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 23 type: IBT flags: 0x50210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.006344:0:17567:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2e00/0xaf35132e630363ee lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 23 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d51ed expref: 44 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.006355:0:17567:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2e00/0xaf35132e630363ee lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 23 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d51ed expref: 44 pid: 6477 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.006363:0:17567:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481ff480 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:17567:1866777605130112:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298646.006402:0:17567:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c840 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.007056:0:17567:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481ff480 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:17567:1866777605130112:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298646.007067:0:17567:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b200/0xaf35132e630366b8 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 23 type: IBT flags: 0x40210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.007275:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.007315:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609749248 00000100:00100000:2.0:1780298646.007320:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609749248 00010000:00010000:2.0:1780298646.007342:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2e00/0xaf35132e630363ee lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 23 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d51ed expref: 45 pid: 6477 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.007368:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15333 00000100:00100000:2.0:1780298646.007373:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fb800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+45:2400:x1866777609749248:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298646.007381:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298646.007385:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2e00/0xaf35132e630363ee lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 24 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d51ed expref: 45 pid: 6477 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.007393:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2e00/0xaf35132e630363ee lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 24 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d51ed expref: 45 pid: 6477 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1780298646.007396:0:6473:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1780298646.007407:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2e00/0xaf35132e630363ee lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 24 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d51ed expref: 45 pid: 6477 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.007415:0:6462:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298646.007419:0:6462:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b200/0xaf35132e630366b8 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 23 type: IBT flags: 0x40210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.007425:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b200/0xaf35132e630366b8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 23 type: IBT flags: 0x40290000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.007430:0:6462:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b200/0xaf35132e630366b8 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 23 type: IBT flags: 0x40290000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.007444:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1780298646.007459:0:17567:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b200/0xaf35132e630366b8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 22 type: IBT flags: 0x40210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.007467:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fb800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+44:2400:x1866777609749248:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 93us (202us total) trans 0 rc 0/0 00010000:00010000:0.0:1780298646.007468:0:17567:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b200/0xaf35132e630366b8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 22 type: IBT flags: 0x40210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.007481:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15333 00000001:00080000:0.0:1780298646.007521:0:17567:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991132 00000004:00010000:0.0:1780298646.007547:0:17567:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000aa18d273 reply state 000000009c78a0ab transno 4294991133 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b200/0xaf35132e630366b8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 22 type: IBT flags: 0x40210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.007562:0:17567:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9b200/0xaf35132e630366b8 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 22 type: IBT flags: 0x50210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.007570:0:17567:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298646.007572:0:17567:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1780298646.007575:0:17567:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1780298646.007636:0:17567:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+52:29360:x1866777609749120:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1598us (1707us total) trans 4294991133 rc 0/0 00000100:00100000:0.0:1780298646.007644:0:17567:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62094 00000020:00080000:0.0:1780298646.007648:0:17567:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1780298646.008338:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.008377:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609749376 00000100:00100000:0.0:1780298646.008381:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609749376 00000100:00100000:0.0:1780298646.008409:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10617 00000100:00100000:0.0:1780298646.008414:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:29360:x1866777609749376:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.008422:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.008435:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e630366bf lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d54b7 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.008452:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000002f8ed852/0xaf35132e63036695 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [520192->2170879] (req 520192->528383) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5494 expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.008458:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e63036648 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [2170880->2203647] (req 2170880->2179071) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5447 expref: 11 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.008463:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63036522 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [4272128->4403199] (req 4272128->4276223) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5321 expref: 14 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.008467:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e630363e0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [4403200->37957631] (req 4403200->4415487) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d51e6 expref: 14 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.008479:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000a5a492f0/0xaf35132e63036545 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [2203648->4268031] (req 2203648->2211839) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5344 expref: 14 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.008484:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 5 to [0x2c0000401:0x11:0x0].0x0 time=1780298645 v=(73 169) 00010000:00010000:0.0:1780298646.008487:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(78 169) 00010000:00010000:0.0:1780298646.008490:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e630366bf lrc: 8/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54b7 expref: 14 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.008504:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000a5a492f0/0xaf35132e63036545 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [2203648->4268031] (req 2203648->2211839) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5344 expref: 14 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.008511:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000a5a492f0/0xaf35132e63036545 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [2203648->4268031] (req 2203648->2211839) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5344 expref: 14 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.008518:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605130240:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298646.008555:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e630363e0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [4403200->37957631] (req 4403200->4415487) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d51e6 expref: 14 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.008563:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e630363e0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [4403200->37957631] (req 4403200->4415487) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d51e6 expref: 14 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.008572:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f734000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605130368:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298646.008606:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63036522 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [4272128->4403199] (req 4272128->4276223) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5321 expref: 14 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.008617:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63036522 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [4272128->4403199] (req 4272128->4276223) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5321 expref: 14 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.008628:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f734a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605130496:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298646.008659:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e63036648 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [2170880->2203647] (req 2170880->2179071) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5447 expref: 11 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.008670:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e63036648 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [2170880->2203647] (req 2170880->2179071) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5447 expref: 11 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.008680:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f736a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605130624:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298646.008766:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000002f8ed852/0xaf35132e63036695 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [520192->2170879] (req 520192->528383) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5494 expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.008774:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000002f8ed852/0xaf35132e63036695 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [520192->2170879] (req 520192->528383) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5494 expref: 11 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.008783:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f734e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605130752:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298646.008807:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4300 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.009591:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605130240:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298646.009613:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f734000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605130368:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298646.009621:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f734a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605130496:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298646.010197:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.010241:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.010343:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609749632 00000100:00100000:0.0:1780298646.010345:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609749760 00000100:00100000:1.0:1780298646.010349:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609749632 00000100:00100000:0.0:1780298646.010359:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609749760 00000100:00100000:3.0:1780298646.010367:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:1.0:1780298646.010378:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000a5a492f0/0xaf35132e63036545 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [2203648->4268031] (req 2203648->2211839) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5344 expref: 16 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.010401:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63036522 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [4272128->4403199] (req 4272128->4276223) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5321 expref: 16 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.010409:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609749504 00000100:00100000:1.0:1780298646.010415:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609749504 00000100:00100000:2.0:1780298646.010417:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15334 00000100:00100000:2.0:1780298646.010435:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2400:x1866777609749632:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298646.010443:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e630363e0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [4403200->37957631] (req 4403200->4415487) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d51e6 expref: 17 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.010447:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:2.0:1780298646.010449:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:0.0:1780298646.010461:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15335 00010000:00010000:2.0:1780298646.010470:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000a5a492f0/0xaf35132e63036545 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 8 type: EXT [2203648->4268031] (req 2203648->2211839) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5344 expref: 17 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.010478:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609749888 00000100:00100000:0.0:1780298646.010480:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b1f050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2399:x1866777609749760:12345-192.168.203.25@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1780298646.010483:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609749888 00010000:00010000:2.0:1780298646.010495:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000a5a492f0/0xaf35132e63036545 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 8 type: EXT [2203648->4268031] (req 2203648->2211839) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5344 expref: 17 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.010500:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:0.0:1780298646.010512:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298646.010524:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a5a492f0/0xaf35132e63036545 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 10 type: EXT [2203648->4268031] (req 2203648->2211839) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5344 expref: 17 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.010524:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e63036648 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 10 type: EXT [2170880->2203647] (req 2170880->2179071) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5447 expref: 12 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.010534:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63036522 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 11 type: EXT [4272128->4403199] (req 4272128->4276223) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5321 expref: 17 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.010540:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609750016 00010000:00010000:2.0:1780298646.010543:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298646.010545:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609750016 00010000:00010000:0.0:1780298646.010560:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63036522 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [4272128->4403199] (req 4272128->4276223) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5321 expref: 16 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.010566:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000002f8ed852/0xaf35132e63036695 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 8 type: EXT [520192->2170879] (req 520192->528383) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5494 expref: 13 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.010583:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609749632:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 152us (385us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298646.010607:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15334 00010000:00010000:0.0:1780298646.010621:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63036522 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [4272128->4403199] (req 4272128->4276223) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5321 expref: 15 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.010622:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15336 00000100:00100000:2.0:1780298646.010637:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15337 00000100:00100000:1.0:1780298646.010640:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2400:x1866777609749504:12345-192.168.203.25@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1780298646.010651:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fa680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609749888:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298646.010664:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298646.010664:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298646.010668:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298646.010676:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e63036648 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 8 type: EXT [2170880->2203647] (req 2170880->2179071) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5447 expref: 13 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.010676:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e630363e0 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 8 type: EXT [4403200->37957631] (req 4403200->4415487) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d51e6 expref: 14 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.010689:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e63036648 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [2170880->2203647] (req 2170880->2179071) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5447 expref: 13 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.010699:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e630363e0 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [4403200->37957631] (req 4403200->4415487) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d51e6 expref: 14 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.010719:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1780298646.010719:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e630363e0 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [4403200->37957631] (req 4403200->4415487) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d51e6 expref: 14 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.010729:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b1f050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2399:x1866777609749760:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 256us (483us total) trans 0 rc 0/0 00010000:00010000:1.0:1780298646.010732:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298646.010742:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15335 00000100:00100000:2.0:1780298646.010749:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fa680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609749888:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 98us (306us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298646.010754:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609749504:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 118us (390us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298646.010760:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15337 00000100:00100000:1.0:1780298646.010765:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15336 00000100:00100000:1.0:1780298646.010786:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15338 00000100:00100000:1.0:1780298646.010799:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609750016:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298646.010808:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298646.010817:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000002f8ed852/0xaf35132e63036695 lrc: 5/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [520192->2170879] (req 520192->528383) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5494 expref: 12 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.010835:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000002f8ed852/0xaf35132e63036695 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [520192->2170879] (req 520192->528383) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5494 expref: 12 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.010853:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(78 169) 00010000:00010000:1.0:1780298646.010857:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(78 169) 00010000:00010000:1.0:1780298646.010862:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e630366bf lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54b7 expref: 11 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.010870:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e630366bf lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50080000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54b7 expref: 11 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.010874:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e63036648 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [2170880->2203647] (req 2170880->2179071) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5447 expref: 12 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.010894:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f736a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605130624:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298646.010901:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e630366bf lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54b7 expref: 11 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.010913:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25488aa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777605130880:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298646.010919:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002f8ed852/0xaf35132e63036695 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [520192->2170879] (req 520192->528383) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5494 expref: 11 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.010934:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f734e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605130752:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298646.010954:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e3c0 going to sleep for 11 seconds 00000100:00080000:1.0:1780298646.010976:0:6473:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:0.0:1780298646.011005:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e630366bf lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54b7 expref: 11 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.011019:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2600, rc 0) 00000100:00100000:0.0:1780298646.011058:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:29360:x1866777609749376:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 2642us (2717us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298646.011067:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10617 00000100:00100000:1.0:1780298646.011713:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25488aa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777605130880:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298646.011729:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298646.011767:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609750016:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 972us (1277us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298646.011773:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15338 00000100:00100000:3.0:1780298646.011999:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.012036:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609750144 00000100:00100000:1.0:1780298646.012041:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609750144 00000100:00100000:1.0:1780298646.012076:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10618 00000100:00100000:1.0:1780298646.012086:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:29360:x1866777609750144:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.012099:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.012113:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e630366c6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d54be expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.012159:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e6303634d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1384448->34938879] (req 1384448->1392639) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5153 expref: 8 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.012170:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000002ff2dd4d/0xaf35132e63036599 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [176128->1384447] (req 176128->180223) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5398 expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.012178:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 2 to [0x280000401:0x12:0x0].0x0 time=1780298645 v=(61 122) 00010000:00010000:1.0:1780298646.012182:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298646 v=(63 122) 00010000:00010000:1.0:1780298646.012187:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e630366c6 lrc: 5/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54be expref: 8 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.012218:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000002ff2dd4d/0xaf35132e63036599 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [176128->1384447] (req 176128->180223) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5398 expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.012242:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000002ff2dd4d/0xaf35132e63036599 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [176128->1384447] (req 176128->180223) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5398 expref: 9 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.012251:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25488aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605131008:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298646.012311:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e6303634d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1384448->34938879] (req 1384448->1392639) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5153 expref: 8 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.012322:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e6303634d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1384448->34938879] (req 1384448->1392639) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5153 expref: 8 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.012333:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25488b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605131136:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298646.012359:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.013177:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25488b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605131136:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298646.013194:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25488aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605131008:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298646.013216:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e630366c6 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54be expref: 8 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.013229:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279e00, rc 0) 00000100:00100000:1.0:1780298646.013257:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:29360:x1866777609750144:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1174us (1260us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298646.013264:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10618 00000100:00100000:2.0:1780298646.013567:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.013607:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609750400 00000100:00100000:3.0:1780298646.013608:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.013614:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609750400 00000100:00100000:2.0:1780298646.013645:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609750272 00000100:00100000:2.0:1780298646.013649:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609750272 00010000:00010000:1.0:1780298646.013651:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000002ff2dd4d/0xaf35132e63036599 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [176128->1384447] (req 176128->180223) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5398 expref: 10 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1780298646.013666:0:6473:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00100000:2.0:1780298646.013679:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x12:0x0] (2772992->2785279) req@ffff9cf3481fb480 x1866777609750272/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:352/0 lens 12776/0 e 0 to 0 dl 1780298657 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298646.013689:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fb480 with x1866777609750272 ext(2772992->2785279) 00000100:00100000:1.0:1780298646.013691:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15339 00010000:00010000:2.0:1780298646.013695:0:19949:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e6303634d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1384448->34938879] (req 1384448->1392639) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5153 expref: 8 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.013698:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481ff480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609750400:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298646.013707:0:19949:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e6303634d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1384448->34938879] (req 1384448->1392639) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5153 expref: 8 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.013711:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:2.0:1780298646.013715:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9cf3481fb480 00010000:00010000:1.0:1780298646.013726:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000002ff2dd4d/0xaf35132e63036599 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [176128->1384447] (req 176128->180223) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5398 expref: 10 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.013738:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000002ff2dd4d/0xaf35132e63036599 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [176128->1384447] (req 176128->180223) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5398 expref: 10 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.013744:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 224 00000100:00100000:2.0:1780298646.013751:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609750272:12345-192.168.203.25@tcp:4:fsx.0 00010000:00010000:1.0:1780298646.013760:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002ff2dd4d/0xaf35132e63036599 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [176128->1384447] (req 176128->180223) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5398 expref: 10 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.013771:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298646.013797:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481ff480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609750400:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 99us (231us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298646.013807:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15339 00000100:00100000:2.0:1780298646.014332:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609750272:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 580us (777us total) trans 4294969335 rc 0/0 00000100:00100000:2.0:1780298646.014340:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 224 00002000:00100000:2.0:1780298646.014348:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x12:0x0] (2772992->2785279) req@ffff9cf3481fb480 x1866777609750272/t4294969335(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:352/0 lens 12776/448 e 0 to 0 dl 1780298657 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298646.014365:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fb480 with x1866777609750272 ext(2772992->2785279) 00010000:00010000:2.0:1780298646.014371:0:19949:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e6303634d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1384448->34938879] (req 1384448->1392639) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5153 expref: 10 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.014384:0:19949:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e6303634d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1384448->34938879] (req 1384448->1392639) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5153 expref: 10 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1780298646.014392:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff9cf3481fb480 00000100:00100000:3.0:1780298646.015226:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.015260:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609750528 00000100:00100000:0.0:1780298646.015278:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609750528 00010000:00010000:0.0:1780298646.015305:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e6303634d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [1384448->34938879] (req 1384448->1392639) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5153 expref: 10 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.015336:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15340 00000100:00100000:0.0:1780298646.015343:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609750528:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298646.015353:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:0.0:1780298646.015364:0:7400:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 40 -> 64 00010000:00010000:0.0:1780298646.015369:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e6303634d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1384448->34938879] (req 1384448->1392639) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5153 expref: 10 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.015378:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e6303634d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1384448->34938879] (req 1384448->1392639) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5153 expref: 10 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.015395:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e6303634d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [1384448->34938879] (req 1384448->1392639) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5153 expref: 10 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.015406:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298646 v=(63 122) 00010000:00010000:0.0:1780298646.015409:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298646 v=(63 122) 00010000:00010000:0.0:1780298646.015413:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e630366c6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54be expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.015420:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e630366c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54be expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.015444:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e630366c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54be expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.015456:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f734e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:7400:1866777605131264:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298646.015526:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1900 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.016438:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f734e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:7400:1866777605131264:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298646.016460:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298646.016496:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609750528:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1153us (1273us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298646.016506:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15340 00000100:00100000:3.0:1780298646.016692:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.016739:0:19928:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609750656 00000100:00100000:0.0:1780298646.016747:0:19928:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609750656 00000100:00100000:3.0:1780298646.016766:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00002000:00010000:0.0:1780298646.016777:0:19928:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (1903093->18446744073709551615) 00002000:00100000:0.0:1780298646.016783:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481f8380 with x1866777609750656 ext(1903093->18446744073709551615) 00002000:00010000:0.0:1780298646.016794:0:19928:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481f8380. 00000100:00100000:1.0:1780298646.016833:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609750784 00000100:00100000:0.0:1780298646.016842:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6549 00000100:00100000:1.0:1780298646.016845:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609750784 00000100:00100000:0.0:1780298646.016857:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609750656:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:1.0:1780298646.016876:0:19943:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00002000:00100000:1.0:1780298646.016881:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fb800 with x1866777609750784 ext(0->18446744073709551615) 00002000:00010000:1.0:1780298646.016890:0:19943:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf3481fb800. 00000100:00100000:1.0:1780298646.016915:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6550 00000100:00100000:1.0:1780298646.016931:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609750784:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:0.0:1780298646.017393:0:19928:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4412060 -> 1903093 00002000:00010000:0.0:1780298646.017397:0:19928:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298646.017399:0:19928:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298646.017401:0:19928:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298646.017403:0:19928:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 584 -> 488 00002000:00010000:1.0:1780298646.017439:0:19943:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 3473408 -> 0 00000100:00100000:0.0:1780298646.017439:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609750656:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 589us (748us total) trans 4294969810 rc 0/0 00002000:00010000:1.0:1780298646.017455:0:19943:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:1.0:1780298646.017460:0:19943:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298646 -> 1780298646 00000100:00100000:0.0:1780298646.017461:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6549 00002000:00010000:0.0:1780298646.017465:0:19928:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (1903093->18446744073709551615) 00002000:00010000:1.0:1780298646.017467:0:19943:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00100000:0.0:1780298646.017469:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481f8380 with x1866777609750656 ext(1903093->18446744073709551615) 00002000:00010000:1.0:1780298646.017480:0:19943:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 64 -> 0 00002000:00010000:0.0:1780298646.017482:0:19928:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481f8380. 00000100:00100000:1.0:1780298646.017519:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609750784:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 592us (751us total) trans 4294969336 rc 0/0 00000100:00100000:1.0:1780298646.017528:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6550 00002000:00010000:1.0:1780298646.017532:0:19943:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00002000:00100000:1.0:1780298646.017536:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fb800 with x1866777609750784 ext(0->18446744073709551615) 00002000:00010000:1.0:1780298646.017541:0:19943:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf3481fb800. 00000100:00100000:0.0:1780298646.018581:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.018642:0:9976:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609750912 00000100:00100000:2.0:1780298646.018649:0:9976:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609750912 00000100:00100000:2.0:1780298646.018693:0:9976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62095 00000100:00100000:2.0:1780298646.018702:0:9976:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f736a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+44:29360:x1866777609750912:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.018717:0:9976:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298646.018732:0:9976:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9bc00/0xaf35132e630366cd lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 23 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d54c5 expref: -99 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.018811:0:9976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9be00/0xaf35132e630366d4 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 24 type: IBT flags: 0x40000000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.018823:0:9976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9be00/0xaf35132e630366d4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 24 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.018831:0:9976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9be00/0xaf35132e630366d4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 24 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298646.018852:0:9976:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 1903093 00000004:00200000:2.0:1780298646.018855:0:9976:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298646.018871:0:9976:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9be00/0xaf35132e630366d4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 24 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.018888:0:9976:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9bc00/0xaf35132e630366cd lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 24 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54c5 expref: 46 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.018900:0:9976:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9be00/0xaf35132e630366d4 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 23 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54c5 expref: 45 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.018906:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298646.018908:0:9976:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298646.018910:0:9976:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9be00, rc 0) 00000100:00100000:2.0:1780298646.018937:0:9976:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f736a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+45:29360:x1866777609750912:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 239us (358us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298646.018941:0:9976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62095 00000100:00100000:2.0:1780298646.020006:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.020034:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.020054:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609751168 00000100:00100000:0.0:1780298646.020060:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609751168 00000100:00100000:3.0:1780298646.020083:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609751040 00000100:00100000:3.0:1780298646.020091:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609751040 00000100:00100000:0.0:1780298646.020107:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10619 00000100:00100000:0.0:1780298646.020119:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609751168:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.020156:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298646.020166:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10620 00010000:00010000:0.0:1780298646.020174:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e630366db lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d54d3 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.020176:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609751040:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298646.020195:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298646.020209:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e630366e2 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d54cc expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.020235:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f734a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605131392:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298646.020260:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605131520:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.020313:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1900 going to sleep for 11 seconds 00000100:00100000:3.0:1780298646.020339:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.021092:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f734a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605131392:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.021117:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605131520:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.021118:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e630366db lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54d3 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.021194:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e630366db lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54d3 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.021203:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e630366e2 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54cc expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.021207:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3800, rc 0) 00010000:00010000:1.0:1780298646.021233:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f2c2b189/0xaf35132e630366e2 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54cc expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.021249:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92600, rc 0) 00000100:00100000:0.0:1780298646.021250:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609751168:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1133us (1241us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.021263:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10619 00000100:00100000:1.0:1780298646.021306:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609751040:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1130us (1267us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.021315:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10620 00000100:00100000:1.0:1780298646.022483:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.022538:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.022569:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609751296 00000100:00100000:0.0:1780298646.022577:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609751296 00000100:00100000:2.0:1780298646.022593:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609751424 00000100:00100000:2.0:1780298646.022607:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609751424 00000100:00100000:0.0:1780298646.022622:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10621 00000100:00100000:0.0:1780298646.022632:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25488aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609751296:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.022646:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298646.022662:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10622 00010000:00010000:0.0:1780298646.022665:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001d4afe0b/0xaf35132e630366e9 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d54da expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.022680:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25488b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609751424:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.022695:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.022709:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002ff2dd4d/0xaf35132e630366f0 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d54e1 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.022721:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f734a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605131648:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.022760:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf25488ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605131776:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.022784:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1900 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.022805:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.023634:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f734a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605131648:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.023661:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001d4afe0b/0xaf35132e630366e9 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54da expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.023679:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001d4afe0b/0xaf35132e630366e9 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54da expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.023707:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3000, rc 0) 00000100:00100000:0.0:1780298646.023756:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25488aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609751296:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1125us (1274us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.023766:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10621 00000100:00100000:2.0:1780298646.023835:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf25488ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605131776:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298646.023861:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002ff2dd4d/0xaf35132e630366f0 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54e1 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.023893:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002ff2dd4d/0xaf35132e630366f0 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54e1 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.023906:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278800, rc 0) 00000100:00100000:2.0:1780298646.023941:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25488b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609751424:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1269us (1402us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298646.023949:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10622 00000100:00100000:1.0:1780298646.025000:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.025048:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609751552 00000100:00100000:0.0:1780298646.025056:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609751552 00000100:00100000:0.0:1780298646.025110:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6551 00000100:00100000:0.0:1780298646.025121:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25488ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609751552:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298646.025361:0:8363:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 1903093 -> 1903093 00002000:00010000:0.0:1780298646.025365:0:8363:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298646.025368:0:8363:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298646.025371:0:8363:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298646.025373:0:8363:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 488 -> 488 00000100:00100000:0.0:1780298646.025414:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25488ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609751552:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 295us (418us total) trans 4294969811 rc 0/0 00000100:00100000:0.0:1780298646.025422:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6551 00000100:00100000:0.0:1780298646.026368:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.026406:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609751680 00000100:00100000:2.0:1780298646.026412:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609751680 00000100:00100000:2.0:1780298646.026452:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10623 00000100:00100000:2.0:1780298646.026462:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f734a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609751680:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.026472:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298646.026480:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:2.0:1780298646.026484:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d71a6aba/0xaf35132e630366f7 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d54e8 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.026532:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7be680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605131904:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298646.026564:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609751808 00000100:00100000:3.0:1780298646.026574:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609751808 00000100:00100000:2.0:1780298646.026600:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c840 going to sleep for 11 seconds 00000100:00100000:3.0:1780298646.026630:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10624 00000100:00100000:3.0:1780298646.026641:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254888a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609751808:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298646.026655:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298646.026667:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e630366fe lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d54ef expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.026703:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605132032:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298646.026776:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2418563c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.027287:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7be680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605131904:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298646.027307:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d71a6aba/0xaf35132e630366f7 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54e8 expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.027321:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d71a6aba/0xaf35132e630366f7 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54e8 expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.027334:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9bc00, rc 0) 00000100:00100000:2.0:1780298646.027364:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f734a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609751680:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 904us (997us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298646.027370:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10623 00000100:00100000:3.0:1780298646.027541:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605132032:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298646.027561:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e630366fe lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54ef expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.027576:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e630366fe lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54ef expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.027586:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93000, rc 0) 00000100:00100000:3.0:1780298646.027623:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254888a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609751808:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 985us (1147us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298646.027631:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10624 00000100:00100000:1.0:1780298646.028486:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.028529:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609751936 00000100:00100000:2.0:1780298646.028537:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609751936 00000100:00100000:2.0:1780298646.028589:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10625 00000100:00100000:2.0:1780298646.028600:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254889c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609751936:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.028611:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298646.028620:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:2.0:1780298646.028623:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b4561703/0xaf35132e63036705 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d54f6 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.028646:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609752064 00000100:00100000:1.0:1780298646.028653:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609752064 00000100:00100000:2.0:1780298646.028670:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7be680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605132160:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.028693:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10626 00000100:00100000:1.0:1780298646.028706:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f734000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609752064:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.028718:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.028733:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e6303670c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d54fd expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.028741:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c840 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.028776:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254889180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605132288:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.028819:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.029486:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7be680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605132160:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298646.029508:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b4561703/0xaf35132e63036705 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54f6 expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.029526:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b4561703/0xaf35132e63036705 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54f6 expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.029546:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b600, rc 0) 00000100:00100000:2.0:1780298646.029590:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254889c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2400:x1866777609751936:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 991us (1103us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298646.029609:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10625 00000100:00100000:1.0:1780298646.029667:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254889180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605132288:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298646.029687:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e6303670c lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54fd expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.029706:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e6303670c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54fd expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.029718:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278a00, rc 0) 00000100:00100000:1.0:1780298646.029750:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f734000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609752064:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1049us (1134us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.029757:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10626 00000100:00100000:0.0:1780298646.030715:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.030751:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609752192 00000100:00100000:1.0:1780298646.030756:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609752192 00000100:00100000:1.0:1780298646.030791:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10627 00000100:00100000:1.0:1780298646.030799:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f736d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609752192:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.030809:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.030821:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e63036713 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5504 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.030842:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e630366bf lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54b7 expref: 12 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.030852:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(78 169) 00010000:00010000:1.0:1780298646.030857:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(79 169) 00010000:00010000:1.0:1780298646.030862:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e63036713 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1073152->1081343] (req 1073152->1081343) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5504 expref: 11 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.030886:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e630366bf lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54b7 expref: 12 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.030901:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e630366bf lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54b7 expref: 12 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.030916:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254889180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605132416:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298646.030968:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.031691:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254889180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605132416:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298646.031708:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e63036713 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1073152->1081343] (req 1073152->1081343) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5504 expref: 11 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.031721:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278800, rc 0) 00000100:00100000:1.0:1780298646.031782:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f736d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:29360:x1866777609752192:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 984us (1067us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298646.031789:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10627 00000100:00100000:0.0:1780298646.031918:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.031958:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609752320 00000100:00100000:1.0:1780298646.031963:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609752320 00010000:00010000:1.0:1780298646.031983:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e630366bf lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54b7 expref: 13 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.032009:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15341 00000100:00100000:1.0:1780298646.032015:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f737800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609752320:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298646.032027:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298646.032046:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e630366bf lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54b7 expref: 13 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.032059:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e630366bf lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54b7 expref: 13 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.032082:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e630366bf lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54b7 expref: 13 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.032093:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(79 169) 00010000:00010000:1.0:1780298646.032096:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(79 169) 00010000:00010000:1.0:1780298646.032104:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e63036713 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1073152->34627583] (req 1073152->1081343) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5504 expref: 10 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.032116:0:6463:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e63036713 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1073152->34627583] (req 1073152->1081343) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5504 expref: 10 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.032156:0:6463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e63036713 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1073152->34627583] (req 1073152->1081343) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5504 expref: 10 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.032166:0:6463:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254889180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6463:1866777605132544:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298646.032200:0:6463:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.032940:0:6463:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254889180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6463:1866777605132544:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298646.032951:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298646.032981:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f737800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609752320:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 966us (1063us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298646.032993:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15341 00000100:00100000:0.0:1780298646.033230:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.033279:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609752448 00000100:00100000:2.0:1780298646.033288:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609752448 00000100:00100000:2.0:1780298646.033337:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6552 00000100:00100000:2.0:1780298646.033351:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f734e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609752448:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298646.033600:0:16972:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 1903093 -> 1903093 00002000:00010000:2.0:1780298646.033605:0:16972:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:2.0:1780298646.033608:0:16972:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298646 -> 1780298646 00002000:00010000:2.0:1780298646.033610:0:16972:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:2.0:1780298646.033613:0:16972:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 488 -> 504 00000100:00100000:2.0:1780298646.033660:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f734e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609752448:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 314us (430us total) trans 4294969812 rc 0/0 00000100:00100000:2.0:1780298646.033668:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6552 00000100:00100000:0.0:1780298646.034693:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.034739:0:17578:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609752576 00000100:00100000:3.0:1780298646.034747:0:17578:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609752576 00000100:00100000:3.0:1780298646.034791:0:17578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62096 00000100:00100000:3.0:1780298646.034801:0:17578:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f736680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+52:29360:x1866777609752576:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298646.034819:0:17578:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298646.034835:0:17578:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93a00/0xaf35132e6303671a lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 24 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d550b expref: -99 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.034921:0:17578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93000/0xaf35132e63036721 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 25 type: IBT flags: 0x40000000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.034937:0:17578:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93000/0xaf35132e63036721 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 25 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.034948:0:17578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93000/0xaf35132e63036721 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 25 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1780298646.034973:0:17578:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 1903093 00000004:00200000:3.0:1780298646.034978:0:17578:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1780298646.034996:0:17578:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93000/0xaf35132e63036721 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 25 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.035022:0:17578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93a00/0xaf35132e6303671a lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 25 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d550b expref: 54 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.035037:0:17578:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93000/0xaf35132e63036721 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 24 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d550b expref: 53 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.035046:0:17578:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298646.035049:0:17578:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1780298646.035053:0:17578:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93000, rc 0) 00000100:00100000:3.0:1780298646.035089:0:17578:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f736680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+53:29360:x1866777609752576:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 291us (399us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298646.035098:0:17578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62096 00000100:00100000:3.0:1780298646.036094:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.036156:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609752704 00000100:00100000:1.0:1780298646.036161:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609752704 00000100:00100000:1.0:1780298646.036195:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10628 00000100:00100000:1.0:1780298646.036202:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609752704:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.036210:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.036225:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63036728 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5512 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.036281:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254889180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605132672:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.036324:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.036994:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254889180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605132672:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298646.037009:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63036728 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5512 expref: 13 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.037026:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63036728 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5512 expref: 13 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.037036:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278a00, rc 0) 00000100:00100000:1.0:1780298646.037066:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609752704:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 864us (973us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.037072:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10628 00000100:00100000:0.0:1780298646.039153:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.039219:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609752832 00000100:00100000:2.0:1780298646.039230:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609752832 00000100:00100000:2.0:1780298646.039289:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10629 00000100:00100000:2.0:1780298646.039301:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f737b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609752832:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.039314:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298646.039326:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b4561703/0xaf35132e6303672f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5519 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.039369:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7be680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605132800:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.039440:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c840 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.040335:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7be680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605132800:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298646.040358:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b4561703/0xaf35132e6303672f lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5519 expref: 13 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.040372:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b4561703/0xaf35132e6303672f lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5519 expref: 13 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.040381:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b600, rc 0) 00000100:00100000:2.0:1780298646.040412:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f737b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609752832:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1117us (1265us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298646.040418:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10629 00000100:00100000:0.0:1780298646.041556:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.041610:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609752960 00000100:00100000:1.0:1780298646.041620:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609752960 00000100:00100000:1.0:1780298646.041665:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10630 00000100:00100000:1.0:1780298646.041674:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609752960:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.041689:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.041702:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63036736 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5520 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.041751:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254889180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605132928:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.041810:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.042565:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254889180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605132928:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298646.042587:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63036736 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5520 expref: 13 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.042604:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63036736 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5520 expref: 13 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.042615:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279c00, rc 0) 00000100:00100000:1.0:1780298646.042655:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609752960:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 983us (1099us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.042663:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10630 00000100:00100000:3.0:1780298646.043650:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.043696:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609753088 00000100:00100000:1.0:1780298646.043704:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609753088 00000100:00100000:1.0:1780298646.043747:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10631 00000100:00100000:1.0:1780298646.043756:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:29360:x1866777609753088:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.043768:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.043780:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e6303673d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5527 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.043801:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e63036713 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1073152->34627583] (req 1073152->1081343) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5504 expref: 11 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.043816:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(79 169) 00010000:00010000:1.0:1780298646.043820:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(80 169) 00010000:00010000:1.0:1780298646.043825:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e6303673d lrc: 4/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1839104->1843199] (req 1839104->1843199) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5527 expref: 13 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.043847:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e63036713 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1073152->34627583] (req 1073152->1081343) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5504 expref: 11 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.043859:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e63036713 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1073152->34627583] (req 1073152->1081343) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5504 expref: 11 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.043870:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254889180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605133056:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298646.043918:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.044720:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254889180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605133056:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298646.044737:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e6303673d lrc: 3/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1839104->1843199] (req 1839104->1843199) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5527 expref: 13 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.044748:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278400, rc 0) 00000100:00100000:1.0:1780298646.044776:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:29360:x1866777609753088:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1022us (1130us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298646.044783:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10631 00000100:00100000:0.0:1780298646.044943:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.045024:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609753216 00000100:00100000:3.0:1780298646.045030:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609753216 00010000:00010000:3.0:1780298646.045053:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e63036713 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1073152->34627583] (req 1073152->1081343) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5504 expref: 12 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.045084:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15342 00000100:00100000:3.0:1780298646.045092:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b1d850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2399:x1866777609753216:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298646.045103:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1780298646.045118:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e63036713 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1073152->34627583] (req 1073152->1081343) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5504 expref: 12 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.045162:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e63036713 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1073152->34627583] (req 1073152->1081343) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5504 expref: 12 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.045189:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e63036713 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [1073152->34627583] (req 1073152->1081343) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5504 expref: 12 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.045201:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(80 169) 00010000:00010000:3.0:1780298646.045205:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(80 169) 00010000:00010000:3.0:1780298646.045212:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e6303673d lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1839104->1843199) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5527 expref: 12 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.045222:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e6303673d lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1839104->1843199) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5527 expref: 12 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.045304:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e6303673d lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1839104->1843199) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5527 expref: 12 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.045366:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7be680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777605133184:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298646.045410:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c840 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.046117:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7be680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777605133184:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298646.046152:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298646.046191:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b1d850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609753216:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1099us (1247us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298646.046200:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15342 00000100:00100000:0.0:1780298646.046501:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.046536:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609753344 00000100:00100000:2.0:1780298646.046540:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609753344 00002000:00100000:2.0:1780298646.046568:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (1839104->1843199) req@ffff9cf24519f100 x1866777609753344/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:352/0 lens 488/0 e 0 to 0 dl 1780298657 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298646.046576:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24519f100 with x1866777609753344 ext(1839104->1843199) 00002000:00010000:2.0:1780298646.046580:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24519f100 00000100:00100000:2.0:1780298646.046601:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6553 00000100:00100000:2.0:1780298646.046605:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609753344:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298646.046776:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609753344:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 170us (276us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298646.046781:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6553 00002000:00100000:2.0:1780298646.046785:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (1839104->1843199) req@ffff9cf24519f100 x1866777609753344/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:352/0 lens 488/4536 e 0 to 0 dl 1780298657 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298646.046791:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24519f100 with x1866777609753344 ext(1839104->1843199) 00002000:00010000:2.0:1780298646.046794:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24519f100 00000100:00100000:0.0:1780298646.047920:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.048001:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609753472 00000100:00100000:3.0:1780298646.048007:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609753472 00002000:00100000:3.0:1780298646.048038:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (1843200->1847295) req@ffff9cf24519c700 x1866777609753472/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:352/0 lens 488/0 e 0 to 0 dl 1780298657 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298646.048048:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24519c700 with x1866777609753472 ext(1843200->1847295) 00002000:00010000:3.0:1780298646.048053:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24519c700 00000100:00100000:3.0:1780298646.048106:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6554 00000100:00100000:3.0:1780298646.048112:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2398:x1866777609753472:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298646.048241:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2398:x1866777609753472:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 129us (323us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1780298646.048246:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6554 00002000:00100000:3.0:1780298646.048251:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (1843200->1847295) req@ffff9cf24519c700 x1866777609753472/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:352/0 lens 488/4536 e 0 to 0 dl 1780298657 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298646.048257:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24519c700 with x1866777609753472 ext(1843200->1847295) 00002000:00010000:3.0:1780298646.048260:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24519c700 00000100:00100000:1.0:1780298646.049506:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.049569:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609753600 00000100:00100000:0.0:1780298646.049576:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609753600 00002000:00100000:0.0:1780298646.049603:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (1818624->1884159) req@ffff9cf254889180 x1866777609753600/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:352/0 lens 504/0 e 0 to 0 dl 1780298657 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298646.049613:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf254889180 with x1866777609753600 ext(1818624->1884159) 00002000:00010000:0.0:1780298646.049616:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf254889180 00000100:00100000:0.0:1780298646.049646:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6555 00000100:00100000:0.0:1780298646.049655:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254889180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2399:x1866777609753600:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298646.050706:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254889180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2399:x1866777609753600:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 1050us (1200us total) trans 0 rc 57344/57344 00000100:00100000:0.0:1780298646.050714:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6555 00002000:00100000:0.0:1780298646.050722:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (1818624->1884159) req@ffff9cf254889180 x1866777609753600/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:352/0 lens 504/440 e 0 to 0 dl 1780298657 ref 1 fl Complete:/600/0 rc 57344/57344 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298646.050730:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf254889180 with x1866777609753600 ext(1818624->1884159) 00002000:00010000:0.0:1780298646.050733:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf254889180 00000100:00100000:2.0:1780298646.053887:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.053937:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609753728 00000100:00100000:1.0:1780298646.053945:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609753728 00000100:00100000:1.0:1780298646.053998:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10632 00000100:00100000:1.0:1780298646.054009:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7be680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:29360:x1866777609753728:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.054024:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.054043:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63036744 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d552e expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.054074:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e6303673d lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1839104->1843199) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5527 expref: 12 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.054084:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(80 169) 00010000:00010000:1.0:1780298646.054088:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(80 169) 00010000:00010000:1.0:1780298646.054093:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63036744 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [5750784->5758975] (req 5750784->5758975) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d552e expref: 12 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.054151:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e6303673d lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1839104->1843199) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5527 expref: 12 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.054166:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e6303673d lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1839104->1843199) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5527 expref: 12 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.054181:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254888380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605133312:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298646.054255:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298646.055171:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254888380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605133312:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298646.055195:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63036744 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [5750784->5758975] (req 5750784->5758975) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d552e expref: 12 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.055212:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278200, rc 0) 00000100:00100000:3.0:1780298646.055343:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7be680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:29360:x1866777609753728:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1335us (1459us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298646.055352:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10632 00000100:00100000:2.0:1780298646.055370:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.055424:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609753856 00000100:00100000:0.0:1780298646.055430:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609753856 00010000:00010000:0.0:1780298646.055454:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e6303673d lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1839104->1843199) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5527 expref: 13 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.055486:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15343 00000100:00100000:0.0:1780298646.055496:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bc700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2400:x1866777609753856:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298646.055510:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298646.055529:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e6303673d lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1839104->1843199) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5527 expref: 13 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.055542:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e6303673d lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1839104->1843199) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5527 expref: 13 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.055573:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e6303673d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 1839104->1843199) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5527 expref: 13 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.055587:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(80 169) 00010000:00010000:0.0:1780298646.055591:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(80 169) 00010000:00010000:0.0:1780298646.055598:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63036744 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [5750784->39305215] (req 5750784->5758975) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d552e expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.055607:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63036744 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [5750784->39305215] (req 5750784->5758975) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d552e expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.055641:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63036744 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [5750784->39305215] (req 5750784->5758975) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d552e expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.055654:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519f480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777605133440:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298646.055698:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1900 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.056399:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519f480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777605133440:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298646.056410:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298646.056434:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bc700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609753856:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 941us (1067us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298646.056445:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15343 00000100:00100000:1.0:1780298646.056992:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.057048:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609753984 00000100:00100000:2.0:1780298646.057052:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609753984 00000100:00100000:2.0:1780298646.057085:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10633 00000100:00100000:2.0:1780298646.057094:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254888000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609753984:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.057105:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298646.057116:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e6303674b lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5535 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.057177:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605133568:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.057210:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00002000:00010000:0.0:1780298646.057928:0:19992:0:(ofd_lvb.c:259:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size: 1903093 -> 5755015 00000100:00100000:0.0:1780298646.057945:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605133568:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.057966:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e6303674b lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5535 expref: 13 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.057981:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e6303674b lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5535 expref: 13 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.057992:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9ac00, rc 0) 00000100:00100000:0.0:1780298646.058031:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254888000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609753984:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 933us (1034us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.058041:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10633 00000100:00100000:1.0:1780298646.058942:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.058997:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609754112 00000100:00100000:2.0:1780298646.059004:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609754112 00000100:00100000:2.0:1780298646.059049:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10634 00000100:00100000:2.0:1780298646.059058:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254888380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609754112:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.059067:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298646.059076:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63036752 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d553c expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.059112:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605133696:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.059175:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6cf00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.059862:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605133696:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298646.059891:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63036752 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d553c expref: 13 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.059905:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63036752 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d553c expref: 13 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.059914:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9aa00, rc 0) 00000100:00100000:2.0:1780298646.059944:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254888380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609754112:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 888us (1004us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298646.059950:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10634 00000100:00100000:0.0:1780298646.060873:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.060922:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609754240 00000100:00100000:1.0:1780298646.060927:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609754240 00000100:00100000:1.0:1780298646.060958:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10635 00000100:00100000:1.0:1780298646.060966:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:29360:x1866777609754240:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.060976:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.060985:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63036759 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5543 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.061004:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(80 169) 00010000:00010000:1.0:1780298646.061008:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(80 169) 00010000:00010000:1.0:1780298646.061014:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63036759 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [4964352->5750783] (req 4964352->4972543) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5543 expref: 13 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.061026:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63036759 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [4964352->5750783] (req 4964352->4972543) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5543 expref: 13 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.061032:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279c00, rc 0) 00000100:00100000:1.0:1780298646.061057:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:29360:x1866777609754240:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 94us (186us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298646.061061:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10635 00000100:00100000:1.0:1780298646.061905:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.061942:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609754368 00000100:00100000:0.0:1780298646.061947:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609754368 00002000:00100000:0.0:1780298646.061979:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (4964352->4968447) req@ffff9cf2558f1500 x1866777609754368/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:352/0 lens 488/0 e 0 to 0 dl 1780298657 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298646.061992:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2558f1500 with x1866777609754368 ext(4964352->4968447) 00002000:00010000:0.0:1780298646.061996:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2558f1500 00000100:00100000:0.0:1780298646.062019:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6556 00000100:00100000:0.0:1780298646.062026:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609754368:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298646.062198:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609754368:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 172us (292us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298646.062205:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6556 00002000:00100000:0.0:1780298646.062211:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (4964352->4968447) req@ffff9cf2558f1500 x1866777609754368/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:352/0 lens 488/440 e 0 to 0 dl 1780298657 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298646.062219:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2558f1500 with x1866777609754368 ext(4964352->4968447) 00002000:00010000:0.0:1780298646.062223:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2558f1500 00000100:00100000:3.0:1780298646.063224:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.063257:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609754496 00000100:00100000:2.0:1780298646.063262:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609754496 00002000:00100000:2.0:1780298646.063291:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (4968448->4972543) req@ffff9cf3481f9180 x1866777609754496/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:352/0 lens 488/0 e 0 to 0 dl 1780298657 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298646.063298:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481f9180 with x1866777609754496 ext(4968448->4972543) 00002000:00010000:2.0:1780298646.063301:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481f9180 00000100:00100000:2.0:1780298646.063320:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6557 00000100:00100000:2.0:1780298646.063326:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609754496:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298646.063406:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609754496:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 80us (184us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298646.063411:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6557 00002000:00100000:2.0:1780298646.063415:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (4968448->4972543) req@ffff9cf3481f9180 x1866777609754496/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:352/0 lens 488/440 e 0 to 0 dl 1780298657 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298646.063420:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481f9180 with x1866777609754496 ext(4968448->4972543) 00002000:00010000:2.0:1780298646.063423:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf3481f9180 00000100:00100000:2.0:1780298646.064606:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.064638:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.064663:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609754624 00000100:00100000:3.0:1780298646.064669:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609754624 00000100:00100000:1.0:1780298646.064694:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609754752 00000100:00100000:1.0:1780298646.064699:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609754752 00000100:00100000:3.0:1780298646.064725:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10636 00000100:00100000:3.0:1780298646.064737:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609754624:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298646.064747:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10637 00010000:00010000:3.0:1780298646.064749:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298646.064760:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7be300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609754752:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298646.064765:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c1a1ebef/0xaf35132e63036760 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d554a expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.064770:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.064783:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63036767 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5551 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.064820:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605133824:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.064829:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605133952:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.064895:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e540 going to sleep for 11 seconds 00000100:00100000:3.0:1780298646.064896:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856000 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.067488:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605133952:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298646.067494:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605133824:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298646.067526:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c1a1ebef/0xaf35132e63036760 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d554a expref: 14 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.067527:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63036767 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5551 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.067552:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63036767 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5551 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.067560:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c1a1ebef/0xaf35132e63036760 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d554a expref: 14 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.067570:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278a00, rc 0) 00010000:00010000:3.0:1780298646.067573:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93a00, rc 0) 00000100:00100000:3.0:1780298646.067617:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609754624:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 2879us (3008us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298646.067617:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7be300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609754752:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 2857us (2973us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298646.067634:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10637 00000100:00100000:3.0:1780298646.067647:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10636 00000100:00100000:2.0:1780298646.068787:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.068814:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.068843:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609754880 00000100:00100000:1.0:1780298646.068853:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609754880 00000100:00100000:0.0:1780298646.068859:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609755008 00000100:00100000:0.0:1780298646.068863:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609755008 00000100:00100000:1.0:1780298646.068901:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10638 00000100:00100000:1.0:1780298646.068911:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609754880:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.068921:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.068932:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e6303676e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5558 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.068948:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10639 00000100:00100000:3.0:1780298646.068960:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609755008:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298646.068970:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298646.068975:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f2680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605134080:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298646.068980:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004cdc8249/0xaf35132e63036775 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d555f expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.069027:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605134208:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.069032:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e540 going to sleep for 11 seconds 00000100:00100000:3.0:1780298646.069060:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298646.070510:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605134208:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.070542:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f2680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605134080:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298646.070545:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004cdc8249/0xaf35132e63036775 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d555f expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.070562:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e6303676e lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5558 expref: 14 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.070566:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004cdc8249/0xaf35132e63036775 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d555f expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.070590:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92200, rc 0) 00010000:00010000:1.0:1780298646.070591:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e6303676e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5558 expref: 14 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.070603:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279a00, rc 0) 00000100:00100000:3.0:1780298646.070630:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609755008:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1671us (1813us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298646.070639:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10639 00000100:00100000:1.0:1780298646.070660:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609754880:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1721us (1842us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.070669:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10638 00000100:00100000:2.0:1780298646.071719:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.071758:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609755136 00000100:00100000:0.0:1780298646.071765:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609755136 00000100:00100000:0.0:1780298646.071800:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10640 00000100:00100000:0.0:1780298646.071821:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609755136:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.071834:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.071844:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a5a492f0/0xaf35132e6303677c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5566 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.071880:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605134336:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.071924:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1900 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.072694:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605134336:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.072713:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a5a492f0/0xaf35132e6303677c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5566 expref: 13 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.072728:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a5a492f0/0xaf35132e6303677c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5566 expref: 13 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.072745:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3600, rc 0) 00000100:00100000:0.0:1780298646.072776:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2398:x1866777609755136:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 961us (1058us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.072783:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10640 00000100:00100000:2.0:1780298646.073634:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.073664:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609755264 00000100:00100000:3.0:1780298646.073668:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609755264 00000100:00100000:3.0:1780298646.073701:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10641 00000100:00100000:3.0:1780298646.073710:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481ff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:29360:x1866777609755264:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298646.073718:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298646.073728:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000056a806f/0xaf35132e63036783 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d556d expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.073746:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(80 169) 00010000:00010000:3.0:1780298646.073750:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(80 169) 00010000:00010000:3.0:1780298646.073759:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000056a806f/0xaf35132e63036783 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2088960->4964351] (req 2088960->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d556d expref: 13 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.073775:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000056a806f/0xaf35132e63036783 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [2088960->4964351] (req 2088960->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d556d expref: 13 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.073784:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92800, rc 0) 00000100:00100000:3.0:1780298646.073807:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481ff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:29360:x1866777609755264:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 101us (175us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298646.073812:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10641 00000100:00100000:2.0:1780298646.074511:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.074540:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609755392 00000100:00100000:3.0:1780298646.074544:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609755392 00000100:00100000:3.0:1780298646.074576:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6558 00000100:00100000:3.0:1780298646.074580:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f737800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2399:x1866777609755392:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:3.0:1780298646.074746:0:19950:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 5755015 -> 2093435 00002000:00010000:3.0:1780298646.074749:0:19950:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:3.0:1780298646.074752:0:19950:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298646 -> 1780298646 00002000:00010000:3.0:1780298646.074754:0:19950:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:3.0:1780298646.074757:0:19950:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 504 -> 520 00000100:00100000:3.0:1780298646.074787:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f737800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2399:x1866777609755392:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 205us (274us total) trans 4294969813 rc 0/0 00000100:00100000:3.0:1780298646.074792:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6558 00000100:00100000:2.0:1780298646.075767:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.075822:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609755520 00000100:00100000:0.0:1780298646.075828:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609755520 00000100:00100000:0.0:1780298646.075863:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10642 00000100:00100000:0.0:1780298646.075882:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b01880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609755520:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.075891:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.075902:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e6303678a lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5574 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.075911:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.075947:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605134464:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.075980:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609755648 00000100:00100000:1.0:1780298646.075985:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609755648 00000100:00100000:0.0:1780298646.076001:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1900 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.076033:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10643 00000100:00100000:1.0:1780298646.076043:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b03100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609755648:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.076051:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.076061:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017dab243/0xaf35132e63036791 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d557b expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.076094:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f2680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605134592:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.076172:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e540 going to sleep for 11 seconds 00002000:00010000:1.0:1780298646.076836:0:12101:0:(ofd_lvb.c:259:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size: 2093435 -> 5755015 00000100:00100000:1.0:1780298646.076846:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605134464:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298646.076871:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e6303678a lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5574 expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.076891:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e6303678a lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5574 expref: 14 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.076904:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2e00, rc 0) 00000100:00100000:1.0:1780298646.076946:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b01880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609755520:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1069us (1179us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.076954:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10642 00000100:00100000:3.0:1780298646.077096:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f2680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605134592:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298646.077113:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000017dab243/0xaf35132e63036791 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d557b expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.077147:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000017dab243/0xaf35132e63036791 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d557b expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.077157:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279800, rc 0) 00000100:00100000:3.0:1780298646.077193:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b03100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609755648:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1153us (1287us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298646.077200:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10643 00000100:00100000:2.0:1780298646.078181:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.078220:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.078244:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609755776 00000100:00100000:3.0:1780298646.078249:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609755904 00000100:00100000:0.0:1780298646.078251:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609755776 00000100:00100000:3.0:1780298646.078253:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609755904 00000100:00100000:0.0:1780298646.078299:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10644 00000100:00100000:3.0:1780298646.078305:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10645 00000100:00100000:0.0:1780298646.078308:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b01f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609755776:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298646.078313:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609755904:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.078318:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298646.078319:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298646.078326:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004cdc8249/0xaf35132e63036798 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5589 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.078326:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000488a7b88/0xaf35132e6303679f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5582 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.078358:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605134720:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.078358:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605134848:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298646.078403:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856000 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.078403:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1900 going to sleep for 11 seconds 00000100:00100000:3.0:1780298646.079291:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605134720:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298646.079314:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004cdc8249/0xaf35132e63036798 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5589 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.079330:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004cdc8249/0xaf35132e63036798 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5589 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.079346:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92200, rc 0) 00000100:00100000:2.0:1780298646.079376:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605134848:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298646.079386:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609755904:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1072us (1164us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298646.079396:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10645 00010000:00010000:2.0:1780298646.079422:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000488a7b88/0xaf35132e6303679f lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5582 expref: 14 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.079446:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000488a7b88/0xaf35132e6303679f lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5582 expref: 14 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.079460:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2200, rc 0) 00000100:00100000:2.0:1780298646.079513:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b01f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609755776:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1206us (1340us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298646.079523:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10644 00000100:00100000:0.0:1780298646.080559:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.080633:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609756032 00000100:00100000:3.0:1780298646.080638:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609756032 00000100:00100000:3.0:1780298646.080673:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10646 00000100:00100000:3.0:1780298646.080683:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609756032:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298646.080693:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298646.080699:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:3.0:1780298646.080702:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c1a1ebef/0xaf35132e630367a6 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5590 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.080730:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609756160 00000100:00100000:0.0:1780298646.080735:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609756160 00000100:00100000:0.0:1780298646.080761:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10647 00000100:00100000:0.0:1780298646.080767:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e426850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609756160:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.080774:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298646.080778:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481fa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605134976:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.080783:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008361787d/0xaf35132e630367ad lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5597 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.080815:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605135104:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298646.080829:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856000 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.080849:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1900 going to sleep for 11 seconds 00000100:00100000:3.0:1780298646.081618:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481fa680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605134976:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298646.081645:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c1a1ebef/0xaf35132e630367a6 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5590 expref: 14 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.081652:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605135104:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298646.081669:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c1a1ebef/0xaf35132e630367a6 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5590 expref: 14 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.081671:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008361787d/0xaf35132e630367ad lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5597 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.081695:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008361787d/0xaf35132e630367ad lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5597 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.081696:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93a00, rc 0) 00010000:00010000:1.0:1780298646.081706:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2800, rc 0) 00000100:00100000:1.0:1780298646.081739:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e426850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609756160:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 971us (1041us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298646.081742:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2398:x1866777609756032:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1057us (1180us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.081747:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10647 00000100:00100000:3.0:1780298646.081757:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10646 00000100:00100000:2.0:1780298646.082792:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.082825:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609756288 00000100:00100000:3.0:1780298646.082830:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609756288 00000100:00100000:3.0:1780298646.082860:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10648 00000100:00100000:3.0:1780298646.082869:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b01c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:29360:x1866777609756288:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298646.082875:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298646.082882:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000038a88423/0xaf35132e630367b4 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d559e expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.082896:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(80 169) 00010000:00010000:3.0:1780298646.082900:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(80 169) 00010000:00010000:3.0:1780298646.082909:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000038a88423/0xaf35132e630367b4 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1265664->2088959] (req 1265664->1269759) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d559e expref: 14 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.082923:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000038a88423/0xaf35132e630367b4 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1265664->2088959] (req 1265664->1269759) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d559e expref: 14 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.082932:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93e00, rc 0) 00000100:00100000:3.0:1780298646.082960:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b01c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:29360:x1866777609756288:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 93us (169us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298646.082966:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10648 00000100:00100000:2.0:1780298646.083853:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.083880:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609756416 00000100:00100000:3.0:1780298646.083885:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609756416 00002000:00100000:3.0:1780298646.083914:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (1265664->1269759) req@ffff9cf241b00000 x1866777609756416/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:352/0 lens 488/0 e 0 to 0 dl 1780298657 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298646.083931:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf241b00000 with x1866777609756416 ext(1265664->1269759) 00002000:00010000:3.0:1780298646.083935:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf241b00000 00000100:00100000:3.0:1780298646.083956:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6559 00000100:00100000:3.0:1780298646.083965:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b00000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609756416:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298646.084150:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b00000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609756416:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 185us (295us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1780298646.084158:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6559 00002000:00100000:3.0:1780298646.084165:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (1265664->1269759) req@ffff9cf241b00000 x1866777609756416/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:352/0 lens 488/4536 e 0 to 0 dl 1780298657 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298646.084173:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf241b00000 with x1866777609756416 ext(1265664->1269759) 00002000:00010000:3.0:1780298646.084177:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf241b00000 00000100:00100000:0.0:1780298646.085300:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.085332:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.085333:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609756672 00000100:00100000:2.0:1780298646.085337:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609756672 00000100:00100000:1.0:1780298646.085368:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609756544 00000100:00100000:2.0:1780298646.085372:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10649 00000100:00100000:1.0:1780298646.085374:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609756544 00000100:00100000:2.0:1780298646.085381:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609756672:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.085389:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298646.085399:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000019780e8d/0xaf35132e630367bb lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d55ac expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.085406:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10650 00000100:00100000:1.0:1780298646.085416:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609756544:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.085423:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.085431:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e630367c2 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d55a5 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.085444:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b01880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605135232:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.085468:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605135360:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.085506:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6cf00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.085506:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.086293:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605135360:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298646.086313:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e630367c2 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d55a5 expref: 15 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.086326:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e630367c2 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d55a5 expref: 15 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.086337:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b01880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605135232:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298646.086340:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278c00, rc 0) 00010000:00010000:0.0:1780298646.086359:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000019780e8d/0xaf35132e630367bb lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d55ac expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.086370:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609756544:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 955us (1038us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.086377:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10650 00010000:00010000:0.0:1780298646.086378:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000019780e8d/0xaf35132e630367bb lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d55ac expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.086392:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9a200, rc 0) 00000100:00100000:0.0:1780298646.086430:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609756672:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1049us (1130us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.086437:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10649 00000100:00100000:3.0:1780298646.087389:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.087414:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.087442:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609756800 00000100:00100000:2.0:1780298646.087451:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609756800 00000100:00100000:0.0:1780298646.087472:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609756928 00000100:00100000:0.0:1780298646.087478:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609756928 00000100:00100000:2.0:1780298646.087493:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10651 00000100:00100000:2.0:1780298646.087506:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2397:x1866777609756800:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298646.087507:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10652 00010000:00010000:2.0:1780298646.087521:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298646.087527:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609756928:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.087535:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e630367c9 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d55b3 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.087542:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298646.087560:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f2c2b189/0xaf35132e630367d0 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d55ba expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.087582:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241b03100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605135488:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298646.087604:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605135616:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.087642:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6cf00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298646.087651:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856180 going to sleep for 11 seconds 00000100:00100000:3.0:1780298646.088356:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241b03100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605135488:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.088371:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605135616:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298646.088384:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e630367c9 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d55b3 expref: 15 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.088400:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e630367c9 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d55b3 expref: 15 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.088400:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f2c2b189/0xaf35132e630367d0 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d55ba expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.088439:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9aa00, rc 0) 00010000:00010000:0.0:1780298646.088441:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f2c2b189/0xaf35132e630367d0 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d55ba expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.088456:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92600, rc 0) 00000100:00100000:3.0:1780298646.088478:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2397:x1866777609756800:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 975us (1093us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298646.088484:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10651 00000100:00100000:0.0:1780298646.088494:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609756928:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 971us (1073us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.088506:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10652 00000100:00100000:0.0:1780298646.089956:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.089997:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609757056 00000100:00100000:1.0:1780298646.090012:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609757056 00002000:00100000:1.0:1780298646.090041:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (2199552->2265087) req@ffff9cf24519c380 x1866777609757056/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:352/0 lens 488/0 e 0 to 0 dl 1780298657 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298646.090049:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24519c380 with x1866777609757056 ext(2199552->2265087) 00002000:00010000:1.0:1780298646.090052:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24519c380 00000100:00100000:1.0:1780298646.090074:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6560 00000100:00100000:1.0:1780298646.090082:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609757056:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298646.090788:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609757056:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 706us (831us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298646.090796:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6560 00002000:00100000:1.0:1780298646.090802:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (2199552->2265087) req@ffff9cf24519c380 x1866777609757056/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:352/0 lens 488/440 e 0 to 0 dl 1780298657 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298646.090818:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24519c380 with x1866777609757056 ext(2199552->2265087) 00002000:00010000:1.0:1780298646.090821:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24519c380 00000100:00100000:2.0:1780298646.091971:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.092008:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609757184 00000100:00100000:0.0:1780298646.092014:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609757184 00000100:00100000:0.0:1780298646.092055:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10653 00000100:00100000:0.0:1780298646.092064:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609757184:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298646.092072:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:0.0:1780298646.092078:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.092093:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e630367d7 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d55c1 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.092167:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605135744:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298646.092188:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609757312 00000100:00100000:3.0:1780298646.092197:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609757312 00000100:00100000:0.0:1780298646.092238:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1900 going to sleep for 11 seconds 00000100:00100000:3.0:1780298646.092276:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10654 00000100:00100000:3.0:1780298646.092292:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b01f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609757312:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298646.092304:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298646.092315:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c1a1ebef/0xaf35132e630367de lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d55c8 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.092348:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605135872:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298646.092488:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856180 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.093067:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605135744:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298646.093089:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605135872:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.093102:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e630367d7 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d55c1 expref: 15 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.093106:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c1a1ebef/0xaf35132e630367de lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d55c8 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.093151:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e630367d7 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d55c1 expref: 15 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.093174:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3200, rc 0) 00010000:00010000:3.0:1780298646.093176:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c1a1ebef/0xaf35132e630367de lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d55c8 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.093185:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93a00, rc 0) 00000100:00100000:0.0:1780298646.093205:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609757184:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1143us (1235us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298646.093207:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b01f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609757312:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 919us (1140us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.093215:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10653 00000100:00100000:3.0:1780298646.093223:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10654 00000100:00100000:2.0:1780298646.094137:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.094161:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.094187:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609757568 00000100:00100000:1.0:1780298646.094189:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609757440 00000100:00100000:3.0:1780298646.094193:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609757568 00000100:00100000:1.0:1780298646.094194:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609757440 00000100:00100000:3.0:1780298646.094233:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10655 00000100:00100000:1.0:1780298646.094240:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10656 00000100:00100000:3.0:1780298646.094246:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b03100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609757568:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298646.094251:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241b01880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2397:x1866777609757440:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298646.094257:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.094260:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298646.094278:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004cdc8249/0xaf35132e630367e5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d55d6 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.094280:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e630367ec lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d55cf expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.094316:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605136000:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.094335:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605136128:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298646.094357:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856180 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.094372:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298646.094965:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605136000:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298646.094979:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004cdc8249/0xaf35132e630367e5 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d55d6 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.094989:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004cdc8249/0xaf35132e630367e5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d55d6 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.094996:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92200, rc 0) 00000100:00100000:1.0:1780298646.095013:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605136128:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298646.095023:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b03100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609757568:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 781us (898us total) trans 0 rc 301/301 00010000:00010000:1.0:1780298646.095030:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e630367ec lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d55cf expref: 15 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.095036:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10655 00010000:00010000:1.0:1780298646.095047:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e630367ec lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d55cf expref: 15 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.095055:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278c00, rc 0) 00000100:00100000:1.0:1780298646.095079:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241b01880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2397:x1866777609757440:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 834us (918us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.095085:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10656 00000100:00100000:2.0:1780298646.095968:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.095998:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609757696 00000100:00100000:1.0:1780298646.096002:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609757696 00000100:00100000:1.0:1780298646.096029:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10657 00000100:00100000:1.0:1780298646.096037:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348318000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609757696:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.096046:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.096054:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002ff2dd4d/0xaf35132e630367f3 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d55dd expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.096077:0:19992:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e630366c6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54be expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.096086:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298646 v=(63 122) 00010000:00010000:1.0:1780298646.096089:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298646 v=(64 122) 00010000:00010000:1.0:1780298646.096094:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002ff2dd4d/0xaf35132e630367f3 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3076096->3088383] (req 3076096->3088383) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d55dd expref: 10 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.096114:0:19992:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e630366c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54be expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.096150:0:19992:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e630366c6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54be expref: 9 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.096159:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605136256:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298646.096190:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.096717:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605136256:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298646.096733:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002ff2dd4d/0xaf35132e630367f3 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3076096->3088383] (req 3076096->3088383) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d55dd expref: 10 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.096744:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278800, rc 0) 00000100:00100000:1.0:1780298646.096768:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348318000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609757696:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 733us (801us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298646.096775:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10657 00000100:00100000:2.0:1780298646.096916:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.096948:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609757824 00000100:00100000:0.0:1780298646.096952:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609757824 00010000:00010000:0.0:1780298646.096969:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e630366c6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54be expref: 10 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.096992:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15344 00000100:00100000:0.0:1780298646.096996:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34831b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609757824:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298646.097004:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298646.097012:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e630366c6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54be expref: 10 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.097020:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e630366c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54be expref: 10 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.097032:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e630366c6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54be expref: 10 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.097043:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298646 v=(64 122) 00010000:00010000:0.0:1780298646.097045:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298646 v=(64 122) 00010000:00010000:0.0:1780298646.097051:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000002ff2dd4d/0xaf35132e630367f3 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3076096->3088383) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d55dd expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.097057:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002ff2dd4d/0xaf35132e630367f3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3076096->3088383) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d55dd expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.097072:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000002ff2dd4d/0xaf35132e630367f3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 3076096->3088383) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d55dd expref: 9 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.097079:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519ca80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:20253:1866777605136384:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298646.097104:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1900 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.097730:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519ca80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:20253:1866777605136384:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298646.097738:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298646.097759:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34831b480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609757824:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 763us (844us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298646.097765:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15344 00000100:00100000:2.0:1780298646.098002:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.098032:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609757952 00000100:00100000:0.0:1780298646.098036:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609757952 00002000:00100000:0.0:1780298646.098057:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x12:0x0] (3076096->3088383) req@ffff9cf348318380 x1866777609757952/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:352/0 lens 488/0 e 0 to 0 dl 1780298657 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298646.098064:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf348318380 with x1866777609757952 ext(3076096->3088383) 00002000:00010000:0.0:1780298646.098067:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf348318380 00000100:00100000:0.0:1780298646.098085:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6561 00000100:00100000:0.0:1780298646.098089:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348318380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609757952:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298646.098249:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348318380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609757952:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 159us (246us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298646.098255:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6561 00002000:00100000:0.0:1780298646.098260:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x12:0x0] (3076096->3088383) req@ffff9cf348318380 x1866777609757952/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:352/0 lens 488/440 e 0 to 0 dl 1780298657 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298646.098278:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf348318380 with x1866777609757952 ext(3076096->3088383) 00002000:00010000:0.0:1780298646.098282:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf348318380 00000100:00100000:2.0:1780298646.099174:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.099199:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609758080 00000100:00100000:3.0:1780298646.099204:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609758080 00000100:00100000:3.0:1780298646.099239:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10658 00000100:00100000:3.0:1780298646.099248:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348319180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2397:x1866777609758080:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298646.099257:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298646.099278:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e630367fa lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d55e4 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.099313:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605136512:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298646.099359:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856180 going to sleep for 11 seconds 00000100:00100000:3.0:1780298646.099957:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605136512:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298646.099971:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e630367fa lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d55e4 expref: 15 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.099981:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e630367fa lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d55e4 expref: 15 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.099988:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92a00, rc 0) 00000100:00100000:3.0:1780298646.100010:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348319180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2397:x1866777609758080:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 764us (836us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298646.100015:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10658 00000100:00100000:2.0:1780298646.100918:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.100959:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609758208 00000100:00100000:0.0:1780298646.100965:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609758208 00000100:00100000:0.0:1780298646.101004:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10659 00000100:00100000:0.0:1780298646.101012:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34831a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609758208:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.101022:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.101031:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e63036801 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d55eb expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.101062:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605136640:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.101107:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1900 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.101854:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605136640:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.101871:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e63036801 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d55eb expref: 15 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.101891:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e63036801 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d55eb expref: 15 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.101901:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2600, rc 0) 00000100:00100000:0.0:1780298646.101933:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34831a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609758208:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 922us (1016us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.101939:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10659 00000100:00100000:2.0:1780298646.102842:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.102890:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609758336 00000100:00100000:1.0:1780298646.102896:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609758336 00000100:00100000:1.0:1780298646.102932:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10660 00000100:00100000:1.0:1780298646.102941:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34831bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609758336:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298646.102950:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:1.0:1780298646.102953:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.102964:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63036808 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d55f2 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.102991:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609758464 00000100:00100000:0.0:1780298646.102996:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609758464 00000100:00100000:1.0:1780298646.102997:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605136768:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.103030:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10661 00000100:00100000:0.0:1780298646.103041:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348318a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609758464:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.103051:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298646.103057:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00010000:00010000:0.0:1780298646.103062:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c703984a/0xaf35132e6303680f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d55f9 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.103080:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298646 v=(64 122) 00010000:00010000:0.0:1780298646.103085:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298646 v=(64 122) 00010000:00010000:0.0:1780298646.103093:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c703984a/0xaf35132e6303680f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d55f9 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.103107:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c703984a/0xaf35132e6303680f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d55f9 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.103116:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2000, rc 0) 00000100:00100000:0.0:1780298646.103188:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348318a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609758464:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 149us (240us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298646.103195:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10661 00000100:00100000:1.0:1780298646.103636:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605136768:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298646.103648:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63036808 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d55f2 expref: 15 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.103657:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63036808 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d55f2 expref: 15 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.103663:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278c00, rc 0) 00000100:00100000:1.0:1780298646.103685:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34831bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609758336:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 747us (844us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.103690:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10660 00000100:00100000:2.0:1780298646.104664:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.104708:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609758592 00000100:00100000:0.0:1780298646.104714:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609758592 00002000:00100000:0.0:1780298646.104741:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x12:0x0] (1642496->1646591) req@ffff9cf348318700 x1866777609758592/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:352/0 lens 488/0 e 0 to 0 dl 1780298657 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298646.104751:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf348318700 with x1866777609758592 ext(1642496->1646591) 00002000:00010000:0.0:1780298646.104756:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf348318700 00000100:00100000:0.0:1780298646.104780:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6562 00000100:00100000:0.0:1780298646.104788:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348318700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609758592:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298646.104924:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348318700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609758592:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 136us (260us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298646.104932:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6562 00002000:00100000:0.0:1780298646.104940:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x12:0x0] (1642496->1646591) req@ffff9cf348318700 x1866777609758592/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:352/0 lens 488/440 e 0 to 0 dl 1780298657 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298646.104949:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf348318700 with x1866777609758592 ext(1642496->1646591) 00002000:00010000:0.0:1780298646.104952:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf348318700 00000100:00100000:3.0:1780298646.105855:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.105892:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609758720 00000100:00100000:0.0:1780298646.105897:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609758720 00002000:00100000:0.0:1780298646.105917:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (1646592->1650687) req@ffff9cf3481f8a80 x1866777609758720/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:352/0 lens 488/0 e 0 to 0 dl 1780298657 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298646.105926:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481f8a80 with x1866777609758720 ext(1646592->1650687) 00002000:00010000:0.0:1780298646.105929:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf3481f8a80 00000100:00100000:0.0:1780298646.105951:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6563 00000100:00100000:0.0:1780298646.105958:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609758720:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298646.106029:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609758720:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 71us (175us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298646.106036:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6563 00002000:00100000:0.0:1780298646.106042:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (1646592->1650687) req@ffff9cf3481f8a80 x1866777609758720/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:352/0 lens 488/440 e 0 to 0 dl 1780298657 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298646.106051:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481f8a80 with x1866777609758720 ext(1646592->1650687) 00002000:00010000:0.0:1780298646.106055:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf3481f8a80 00000100:00100000:2.0:1780298646.107066:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.107105:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609758848 00000100:00100000:3.0:1780298646.107110:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609758848 00000100:00100000:2.0:1780298646.107196:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10662 00000100:00100000:2.0:1780298646.107205:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34831aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609758848:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.107214:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298646.107225:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e63036816 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5600 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.107277:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf36d4b6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605136896:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.107331:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:3.0:1780298646.108002:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf36d4b6300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605136896:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298646.108026:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e63036816 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5600 expref: 15 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.108044:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e63036816 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5600 expref: 15 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.108055:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9a600, rc 0) 00000100:00100000:3.0:1780298646.108085:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34831aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609758848:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 883us (1022us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298646.108093:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10662 00000100:00100000:3.0:1780298646.108916:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.108958:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609758976 00000100:00100000:0.0:1780298646.108962:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609758976 00000100:00100000:0.0:1780298646.108986:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10663 00000100:00100000:0.0:1780298646.108991:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609758976:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.108998:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.109008:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e6303681d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5607 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.109030:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24519ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605137024:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.109061:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1900 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.109735:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24519ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605137024:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.109747:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e6303681d lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5607 expref: 15 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.109757:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e6303681d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5607 expref: 15 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.109767:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3200, rc 0) 00000100:00100000:0.0:1780298646.109790:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609758976:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 799us (876us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.109794:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10663 00000100:00100000:2.0:1780298646.110793:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.110835:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609759104 00000100:00100000:1.0:1780298646.110840:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609759104 00000100:00100000:1.0:1780298646.110865:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10664 00000100:00100000:1.0:1780298646.110874:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2398:x1866777609759104:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.110880:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.110888:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63036824 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d560e expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.110919:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605137152:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.110962:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.111617:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605137152:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298646.111635:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63036824 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d560e expref: 15 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.111651:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63036824 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d560e expref: 15 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.111661:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279200, rc 0) 00000100:00100000:1.0:1780298646.111690:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2398:x1866777609759104:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 818us (899us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.111698:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10664 00000100:00100000:2.0:1780298646.112453:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.112481:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609759232 00000100:00100000:1.0:1780298646.112484:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609759232 00000100:00100000:1.0:1780298646.112511:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10665 00000100:00100000:1.0:1780298646.112516:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609759232:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.112523:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.112532:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e6303682b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5615 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.112550:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000002ff2dd4d/0xaf35132e630367f3 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3076096->3088383) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d55dd expref: 11 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.112569:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000c703984a/0xaf35132e6303680f lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d55f9 expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.112577:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298646 v=(64 122) 00010000:00010000:1.0:1780298646.112581:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298646 v=(64 122) 00010000:00010000:1.0:1780298646.112586:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e6303682b lrc: 5/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2142208->2150399] (req 2142208->2150399) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5615 expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.112607:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000c703984a/0xaf35132e6303680f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d55f9 expref: 9 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.112622:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000c703984a/0xaf35132e6303680f lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d55f9 expref: 9 pid: 19992 timeout: 935 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.112634:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605137280:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298646.112671:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000002ff2dd4d/0xaf35132e630367f3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3076096->3088383) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d55dd expref: 11 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.112679:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000002ff2dd4d/0xaf35132e630367f3 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 3076096->3088383) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d55dd expref: 11 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.112696:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605137408:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298646.112726:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.113356:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605137280:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298646.113367:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605137408:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298646.113383:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e6303682b lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2142208->2150399] (req 2142208->2150399) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5615 expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.113396:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279e00, rc 0) 00000100:00100000:1.0:1780298646.113427:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:29360:x1866777609759232:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 909us (973us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298646.113435:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10665 00000100:00100000:2.0:1780298646.113593:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.113618:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.113638:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609759360 00000100:00100000:1.0:1780298646.113643:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609759360 00000100:00100000:3.0:1780298646.113646:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609759488 00000100:00100000:3.0:1780298646.113652:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609759488 00010000:00010000:1.0:1780298646.113664:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000c703984a/0xaf35132e6303680f lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d55f9 expref: 10 pid: 19992 timeout: 935 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.113672:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000002ff2dd4d/0xaf35132e630367f3 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 3076096->3088383) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d55dd expref: 11 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.113694:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15345 00000100:00100000:1.0:1780298646.113705:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b6680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609759360:12345-192.168.203.25@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1780298646.113706:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15346 00010000:00010000:1.0:1780298646.113719:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1780298646.113721:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609759488:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1780298646.113732:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298646.113736:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000c703984a/0xaf35132e6303680f lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d55f9 expref: 10 pid: 19992 timeout: 935 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.113744:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000002ff2dd4d/0xaf35132e630367f3 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 3076096->3088383) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d55dd expref: 11 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.113752:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000c703984a/0xaf35132e6303680f lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d55f9 expref: 10 pid: 19992 timeout: 935 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.113765:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000002ff2dd4d/0xaf35132e630367f3 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 3076096->3088383) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d55dd expref: 11 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.113787:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c703984a/0xaf35132e6303680f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d55f9 expref: 10 pid: 19992 timeout: 935 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.113792:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002ff2dd4d/0xaf35132e630367f3 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 3076096->3088383) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d55dd expref: 11 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.113810:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298646 v=(64 122) 00010000:00010000:3.0:1780298646.113813:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298646 v=(64 122) 00010000:00010000:3.0:1780298646.113819:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e6303682b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2142208->35696639] (req 2142208->2150399) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5615 expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.113825:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e6303682b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2142208->35696639] (req 2142208->2150399) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5615 expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.113832:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1780298646.113844:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e6303682b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2142208->35696639] (req 2142208->2150399) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5615 expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.113853:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f9c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777605137536:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298646.113865:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b6680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609759360:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 162us (270us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298646.113873:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15345 00000100:00100000:3.0:1780298646.113890:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856180 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.114611:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f9c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777605137536:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298646.114622:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298646.114651:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609759488:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 935us (1032us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298646.114659:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15346 00000100:00100000:2.0:1780298646.114740:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.114791:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609759616 00000100:00100000:3.0:1780298646.114794:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609759616 00000100:00100000:3.0:1780298646.114824:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6564 00000100:00100000:3.0:1780298646.114831:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609759616:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:3.0:1780298646.114973:0:8362:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:3.0:1780298646.114975:0:8362:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:3.0:1780298646.114977:0:8362:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298646 -> 1780298646 00002000:00010000:3.0:1780298646.114978:0:8362:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:3.0:1780298646.114980:0:8362:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 0 -> 16 00000100:00100000:3.0:1780298646.115004:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609759616:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 173us (264us total) trans 4294969337 rc 0/0 00000100:00100000:3.0:1780298646.115011:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6564 00000100:00100000:0.0:1780298646.115920:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.115968:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609759744 00000100:00100000:3.0:1780298646.115973:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609759744 00000100:00100000:3.0:1780298646.116004:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10666 00000100:00100000:3.0:1780298646.116012:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24519ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609759744:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298646.116020:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298646.116021:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:3.0:1780298646.116030:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e63036832 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d561c expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.116049:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609759872 00000100:00100000:0.0:1780298646.116053:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609759872 00000100:00100000:3.0:1780298646.116067:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605137664:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.116078:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10667 00000100:00100000:0.0:1780298646.116084:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609759872:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.116090:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.116098:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e63036839 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5623 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.116157:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24310fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605137792:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298646.116172:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856000 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.116185:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1900 going to sleep for 11 seconds 00000100:00100000:3.0:1780298646.116798:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605137664:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298646.116818:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e63036832 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d561c expref: 15 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.116827:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24310fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605137792:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298646.116831:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e63036832 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d561c expref: 15 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.116839:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92a00, rc 0) 00010000:00010000:1.0:1780298646.116850:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e63036839 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5623 expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.116865:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24519ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609759744:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 853us (944us total) trans 0 rc 301/301 00010000:00010000:1.0:1780298646.116869:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e63036839 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5623 expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.116879:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10666 00010000:00010000:1.0:1780298646.116890:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3800, rc 0) 00000100:00100000:1.0:1780298646.116924:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609759872:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 839us (901us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.116933:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10667 00000100:00100000:2.0:1780298646.117805:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.117828:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.117841:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609760000 00000100:00100000:1.0:1780298646.117846:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609760000 00000100:00100000:3.0:1780298646.117852:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609760128 00000100:00100000:3.0:1780298646.117858:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609760128 00000100:00100000:1.0:1780298646.117876:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10668 00000100:00100000:1.0:1780298646.117883:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2398:x1866777609760000:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298646.117890:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10669 00010000:00010000:1.0:1780298646.117890:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.117900:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e63036840 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d562a expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.117902:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609760128:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298646.117915:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298646.117925:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f8ed852/0xaf35132e63036847 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5631 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.117933:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605137920:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298646.117959:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605138048:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.117971:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298646.117984:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856000 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.118643:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605137920:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298646.118657:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e63036840 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d562a expref: 15 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.118667:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e63036840 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d562a expref: 15 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.118674:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279800, rc 0) 00000100:00100000:1.0:1780298646.118702:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2398:x1866777609760000:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 821us (899us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.118707:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10668 00000100:00100000:3.0:1780298646.118759:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605138048:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298646.118830:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002f8ed852/0xaf35132e63036847 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5631 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.118841:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f8ed852/0xaf35132e63036847 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5631 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.118847:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92c00, rc 0) 00000100:00100000:3.0:1780298646.118870:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609760128:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 972us (1042us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298646.118877:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10669 00000100:00100000:0.0:1780298646.119789:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.119823:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609760256 00000100:00100000:1.0:1780298646.119827:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609760256 00000100:00100000:1.0:1780298646.119851:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10670 00000100:00100000:1.0:1780298646.119858:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24310fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609760256:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.119864:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.119871:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e6303684e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5638 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.119893:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.119903:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605138176:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.119931:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609760384 00000100:00100000:0.0:1780298646.119934:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609760384 00000100:00100000:1.0:1780298646.119954:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.119967:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10671 00000100:00100000:0.0:1780298646.119975:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609760384:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.119981:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.119987:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e2cfab1/0xaf35132e63036855 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d563f expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.120014:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605138304:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.120049:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1900 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.120557:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f2a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605138176:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298646.120572:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e6303684e lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5638 expref: 15 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.120582:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e6303684e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5638 expref: 15 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.120592:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279a00, rc 0) 00000100:00100000:1.0:1780298646.120637:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24310fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2400:x1866777609760256:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 779us (847us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.120645:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10670 00000100:00100000:1.0:1780298646.120742:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605138304:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298646.120754:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003e2cfab1/0xaf35132e63036855 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d563f expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.120763:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e2cfab1/0xaf35132e63036855 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d563f expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.120773:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2e00, rc 0) 00000100:00100000:1.0:1780298646.120802:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609760384:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 828us (910us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.120809:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10671 00000100:00100000:2.0:1780298646.121582:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.121613:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609760512 00000100:00100000:1.0:1780298646.121618:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609760512 00000100:00100000:1.0:1780298646.121639:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10672 00000100:00100000:1.0:1780298646.121644:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:29360:x1866777609760512:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.121649:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.121656:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6416acb/0xaf35132e6303685c lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5646 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.121667:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298646 v=(64 122) 00010000:00010000:1.0:1780298646.121670:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298646 v=(64 122) 00010000:00010000:1.0:1780298646.121677:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6416acb/0xaf35132e6303685c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [372736->2142207] (req 372736->376831) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5646 expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.121685:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a6416acb/0xaf35132e6303685c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [372736->2142207] (req 372736->376831) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5646 expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.121690:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278c00, rc 0) 00000100:00100000:1.0:1780298646.121709:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:29360:x1866777609760512:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 66us (130us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298646.121713:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10672 00000100:00100000:2.0:1780298646.122472:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.122503:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609760640 00000100:00100000:1.0:1780298646.122507:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609760640 00002000:00100000:1.0:1780298646.122528:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x12:0x0] (372736->376831) req@ffff9cf36d4b5500 x1866777609760640/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:352/0 lens 488/0 e 0 to 0 dl 1780298657 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298646.122535:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36d4b5500 with x1866777609760640 ext(372736->376831) 00002000:00010000:1.0:1780298646.122538:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf36d4b5500 00000100:00100000:1.0:1780298646.122554:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6565 00000100:00100000:1.0:1780298646.122564:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609760640:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298646.122681:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609760640:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 121us (210us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298646.122686:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6565 00002000:00100000:1.0:1780298646.122690:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x12:0x0] (372736->376831) req@ffff9cf36d4b5500 x1866777609760640/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:352/0 lens 488/440 e 0 to 0 dl 1780298657 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298646.122696:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36d4b5500 with x1866777609760640 ext(372736->376831) 00002000:00010000:1.0:1780298646.122699:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf36d4b5500 00000100:00100000:0.0:1780298646.123623:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.123699:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609760768 00000100:00100000:3.0:1780298646.123703:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609760768 00000100:00100000:3.0:1780298646.123730:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10673 00000100:00100000:3.0:1780298646.123736:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2398:x1866777609760768:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298646.123746:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298646.123754:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004cdc8249/0xaf35132e63036863 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d564d expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.123769:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.123783:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605138432:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.123802:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609760896 00000100:00100000:0.0:1780298646.123806:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609760896 00000100:00100000:0.0:1780298646.123834:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10674 00000100:00100000:0.0:1780298646.123843:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36d4b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609760896:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.123851:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.123859:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c0d26bbc/0xaf35132e6303686a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5654 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.123869:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856000 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.123888:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605138560:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.123915:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1900 going to sleep for 11 seconds 00000100:00100000:3.0:1780298646.124579:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605138432:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298646.124596:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004cdc8249/0xaf35132e63036863 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d564d expref: 15 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.124612:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004cdc8249/0xaf35132e63036863 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d564d expref: 15 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.124619:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92200, rc 0) 00000100:00100000:3.0:1780298646.124646:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2398:x1866777609760768:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 910us (1025us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298646.124651:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10673 00000100:00100000:0.0:1780298646.124684:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605138560:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.124697:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c0d26bbc/0xaf35132e6303686a lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5654 expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.124707:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c0d26bbc/0xaf35132e6303686a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5654 expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.124714:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3c00, rc 0) 00000100:00100000:0.0:1780298646.124737:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36d4b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609760896:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 896us (969us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.124744:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10674 00000100:00100000:1.0:1780298646.125611:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.125648:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609761024 00000100:00100000:0.0:1780298646.125653:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609761024 00000100:00100000:1.0:1780298646.125659:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.125690:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609761152 00000100:00100000:0.0:1780298646.125694:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609761152 00000100:00100000:3.0:1780298646.125702:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10675 00000100:00100000:2.0:1780298646.125715:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10676 00000100:00100000:2.0:1780298646.125724:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609761152:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.125731:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298646.125738:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e63036871 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5662 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.125761:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609761024:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298646.125763:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24339ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605138688:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298646.125773:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298646.125782:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c1a1ebef/0xaf35132e63036878 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d565b expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.125805:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605138816:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.125805:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:3.0:1780298646.125831:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856000 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.126524:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24339ea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605138688:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298646.126543:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e63036871 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5662 expref: 12 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.126559:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009fd8d343/0xaf35132e63036871 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5662 expref: 12 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.126571:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b000, rc 0) 00000100:00100000:0.0:1780298646.126595:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f9880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605138816:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.126607:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609761152:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 884us (947us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.126614:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10676 00010000:00010000:0.0:1780298646.126621:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c1a1ebef/0xaf35132e63036878 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d565b expref: 15 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.126636:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c1a1ebef/0xaf35132e63036878 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d565b expref: 15 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.126646:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93a00, rc 0) 00000100:00100000:0.0:1780298646.126668:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609761024:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 959us (1059us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.126675:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10675 00000100:00100000:2.0:1780298646.127565:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.127598:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609761280 00000100:00100000:1.0:1780298646.127602:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609761280 00000100:00100000:1.0:1780298646.127628:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10677 00000100:00100000:1.0:1780298646.127635:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:29360:x1866777609761280:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.127645:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.127653:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e6303687f lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5669 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.127666:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(80 169) 00010000:00010000:1.0:1780298646.127670:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(80 169) 00010000:00010000:1.0:1780298646.127676:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e6303687f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [1204224->1261567] (req 1204224->1212415) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5669 expref: 15 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.127686:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e6303687f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [1204224->1261567] (req 1204224->1212415) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5669 expref: 15 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.127693:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279800, rc 0) 00000100:00100000:1.0:1780298646.127713:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:29360:x1866777609761280:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 79us (150us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298646.127718:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10677 00000100:00100000:2.0:1780298646.128295:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.128323:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609761408 00000100:00100000:0.0:1780298646.128327:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609761408 00000100:00100000:0.0:1780298646.128355:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6566 00000100:00100000:0.0:1780298646.128361:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2400:x1866777609761408:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298646.128484:0:8363:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 5755015 -> 2093435 00002000:00010000:0.0:1780298646.128486:0:8363:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298646.128488:0:8363:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298646.128490:0:8363:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298646.128491:0:8363:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 520 -> 520 00000100:00100000:0.0:1780298646.128514:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609761408:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 154us (220us total) trans 4294969814 rc 0/0 00000100:00100000:0.0:1780298646.128519:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6566 00000100:00100000:3.0:1780298646.129327:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.129359:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609761536 00000100:00100000:3.0:1780298646.129362:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.129365:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609761536 00000100:00100000:1.0:1780298646.129399:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609761664 00000100:00100000:1.0:1780298646.129403:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609761664 00000100:00100000:0.0:1780298646.129404:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10678 00000100:00100000:0.0:1780298646.129417:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609761536:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.129426:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298646.129435:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10679 00010000:00010000:0.0:1780298646.129436:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e63036886 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5670 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.129445:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609761664:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.129451:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.129460:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e6303688d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5677 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.129475:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605138944:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.129487:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605139072:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.129532:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.129533:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1900 going to sleep for 11 seconds 00002000:00010000:0.0:1780298646.130189:0:8358:0:(ofd_lvb.c:259:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size: 2093435 -> 5755015 00000100:00100000:0.0:1780298646.130201:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605138944:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.130211:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605139072:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.130228:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e63036886 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5670 expref: 17 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.130229:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e6303688d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5677 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.130255:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e63036886 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5670 expref: 17 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.130261:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e6303688d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5677 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.130285:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2e00, rc 0) 00010000:00010000:1.0:1780298646.130286:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279a00, rc 0) 00000100:00100000:1.0:1780298646.130324:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609761664:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 878us (958us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.130325:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609761536:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 906us (992us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.130334:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10679 00000100:00100000:0.0:1780298646.130340:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10678 00000100:00100000:2.0:1780298646.131215:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.131239:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609761792 00000100:00100000:0.0:1780298646.131244:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609761792 00000100:00100000:2.0:1780298646.131248:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.131290:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609761920 00000100:00100000:0.0:1780298646.131295:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10680 00000100:00100000:3.0:1780298646.131296:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609761920 00000100:00100000:0.0:1780298646.131305:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2398:x1866777609761792:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.131312:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.131321:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a5a492f0/0xaf35132e63036894 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d567e expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.131329:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10681 00000100:00100000:3.0:1780298646.131339:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609761920:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298646.131348:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605139200:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298646.131349:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298646.131361:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f8ed852/0xaf35132e6303689b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5685 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.131386:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1900 going to sleep for 11 seconds 00000100:00100000:3.0:1780298646.131396:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf3481f9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605139328:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298646.131424:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2418563c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.132037:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605139200:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.132055:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a5a492f0/0xaf35132e63036894 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d567e expref: 17 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.132079:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a5a492f0/0xaf35132e63036894 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d567e expref: 17 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.132087:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3600, rc 0) 00000100:00100000:0.0:1780298646.132111:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2398:x1866777609761792:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 810us (899us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.132116:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10680 00000100:00100000:0.0:1780298646.132339:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf3481f9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605139328:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.132351:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002f8ed852/0xaf35132e6303689b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5685 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.132362:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f8ed852/0xaf35132e6303689b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5685 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.132368:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92c00, rc 0) 00000100:00100000:0.0:1780298646.132390:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609761920:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1054us (1145us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.132395:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10681 00000100:00100000:2.0:1780298646.133262:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.133309:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609762048 00000100:00100000:1.0:1780298646.133313:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609762048 00000100:00100000:2.0:1780298646.133329:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.133345:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609762176 00000100:00100000:1.0:1780298646.133348:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609762176 00000100:00100000:0.0:1780298646.133360:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10682 00000100:00100000:0.0:1780298646.133370:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609762048:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298646.133377:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10683 00010000:00010000:0.0:1780298646.133377:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.133388:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e630368a2 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d568c expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.133389:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609762176:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.133402:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.133412:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e630368a9 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5693 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.133426:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605139456:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.133439:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605139584:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.133482:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.133483:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1900 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.134278:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605139584:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298646.134304:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e630368a9 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5693 expref: 12 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.134312:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605139456:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298646.134342:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e630368a9 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5693 expref: 12 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.134351:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279200, rc 0) 00010000:00010000:0.0:1780298646.134355:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e630368a2 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d568c expref: 15 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.134375:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e630368a2 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d568c expref: 15 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.134384:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609762176:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 997us (1055us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.134390:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10683 00010000:00010000:0.0:1780298646.134390:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3200, rc 0) 00000100:00100000:0.0:1780298646.134444:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609762048:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1073us (1180us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.134453:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10682 00000100:00100000:3.0:1780298646.135395:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.135428:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609762304 00000100:00100000:1.0:1780298646.135432:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609762304 00000100:00100000:3.0:1780298646.135449:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.135473:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609762432 00000100:00100000:1.0:1780298646.135476:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609762432 00000100:00100000:0.0:1780298646.135508:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10684 00000100:00100000:1.0:1780298646.135520:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10685 00000100:00100000:0.0:1780298646.135526:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2397:x1866777609762304:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298646.135538:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f734a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609762432:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.135544:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.135550:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.135557:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e630368b0 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d569a expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.135562:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e630368b7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d56a1 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.135606:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605139712:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.135609:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605139840:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.135661:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.135680:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1900 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.136409:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605139840:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298646.136438:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e630368b7 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d56a1 expref: 12 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.136451:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e630368b7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d56a1 expref: 12 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.136460:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278a00, rc 0) 00000100:00100000:1.0:1780298646.136498:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f734a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2397:x1866777609762432:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 966us (1046us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.136507:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10685 00000100:00100000:0.0:1780298646.136515:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605139712:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.136540:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e630368b0 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d569a expref: 15 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.136552:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e630368b0 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d569a expref: 15 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.136560:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3800, rc 0) 00000100:00100000:0.0:1780298646.136582:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2397:x1866777609762304:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1065us (1188us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.136591:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10684 00000100:00100000:2.0:1780298646.137539:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.137577:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609762560 00000100:00100000:0.0:1780298646.137584:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609762560 00000100:00100000:0.0:1780298646.137634:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6567 00000100:00100000:0.0:1780298646.137642:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609762560:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298646.137900:0:16972:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 0 -> 2651671 00002000:00010000:0.0:1780298646.137904:0:16972:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298646.137907:0:16972:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298646.137909:0:16972:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298646.137911:0:16972:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 16 -> 32 00000100:00100000:0.0:1780298646.137941:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609762560:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 300us (404us total) trans 4294969338 rc 0/0 00000100:00100000:0.0:1780298646.137947:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6567 00000100:00100000:3.0:1780298646.138812:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.138845:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609762688 00000100:00100000:0.0:1780298646.138849:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609762688 00000100:00100000:3.0:1780298646.138864:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.138883:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609762816 00000100:00100000:0.0:1780298646.138885:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609762816 00000100:00100000:1.0:1780298646.138918:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10686 00000100:00100000:0.0:1780298646.138927:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10687 00000100:00100000:1.0:1780298646.138932:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f734000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609762688:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298646.138943:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f736d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609762816:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.138946:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.138954:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.138956:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e630368be lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d56a8 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.138964:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c0d26bbc/0xaf35132e630368c5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d56af expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.138987:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605139968:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.138997:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605140096:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.139040:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.139040:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1900 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.139706:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605139968:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.139719:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605140096:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298646.139724:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e630368be lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d56a8 expref: 17 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.139739:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e630368be lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d56a8 expref: 17 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.139744:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c0d26bbc/0xaf35132e630368c5 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d56af expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.139747:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278400, rc 0) 00010000:00010000:0.0:1780298646.139768:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c0d26bbc/0xaf35132e630368c5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d56af expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.139777:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f734000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609762688:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 850us (965us total) trans 0 rc 301/301 00010000:00010000:0.0:1780298646.139778:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3c00, rc 0) 00000100:00100000:1.0:1780298646.139784:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10686 00000100:00100000:0.0:1780298646.139807:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f736d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609762816:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 871us (943us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.139814:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10687 00000100:00100000:2.0:1780298646.140682:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.140725:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.140726:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609762944 00000100:00100000:0.0:1780298646.140732:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609762944 00000100:00100000:1.0:1780298646.140771:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609763072 00000100:00100000:0.0:1780298646.140777:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10688 00000100:00100000:1.0:1780298646.140779:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609763072 00000100:00100000:0.0:1780298646.140786:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2397:x1866777609762944:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.140796:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.140808:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e630368cc lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d56b6 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.140827:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10689 00000100:00100000:1.0:1780298646.140839:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609763072:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298646.140847:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605140224:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298646.140851:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.140864:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002ff2dd4d/0xaf35132e630368d3 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d56bd expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.140892:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605140352:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.140911:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1900 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.140935:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.141636:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605140224:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.141637:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605140352:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298646.141676:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002ff2dd4d/0xaf35132e630368d3 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d56bd expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.141680:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e630368cc lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d56b6 expref: 17 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.141722:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002ff2dd4d/0xaf35132e630368d3 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d56bd expref: 11 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.141727:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e630368cc lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d56b6 expref: 17 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.141747:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278800, rc 0) 00010000:00010000:0.0:1780298646.141761:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2000, rc 0) 00000100:00100000:1.0:1780298646.141783:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609763072:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 948us (1057us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.141793:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2397:x1866777609762944:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1003us (1108us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.141801:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10689 00000100:00100000:0.0:1780298646.141810:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10688 00000100:00100000:3.0:1780298646.142843:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.142893:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609763200 00000100:00100000:3.0:1780298646.142901:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.142902:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609763200 00000100:00100000:1.0:1780298646.142942:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609763328 00000100:00100000:1.0:1780298646.142949:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609763328 00000100:00100000:2.0:1780298646.142960:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10690 00000100:00100000:2.0:1780298646.142981:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f737b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2399:x1866777609763200:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298646.142981:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10691 00010000:00010000:2.0:1780298646.142998:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298646.142999:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609763328:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.143009:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298646.143012:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e630368da lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d56c4 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.143018:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0cae770/0xaf35132e630368e1 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d56cb expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.143053:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605140480:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.143054:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24339ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605140608:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.143110:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6cf00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.143110:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1900 going to sleep for 11 seconds 00000100:00100000:3.0:1780298646.143923:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24339ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605140608:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298646.143951:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e630368da lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d56c4 expref: 17 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.143974:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605140480:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.143994:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e0cae770/0xaf35132e630368e1 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d56cb expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.144014:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e630368da lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d56c4 expref: 17 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.144017:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0cae770/0xaf35132e630368e1 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d56cb expref: 11 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.144058:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2600, rc 0) 00010000:00010000:3.0:1780298646.144074:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9a600, rc 0) 00000100:00100000:0.0:1780298646.144097:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609763328:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1107us (1198us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.144106:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10691 00000100:00100000:3.0:1780298646.144115:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f737b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2399:x1866777609763200:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1138us (1267us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298646.144171:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10690 00000100:00100000:3.0:1780298646.145279:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.145321:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609763456 00000100:00100000:0.0:1780298646.145325:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609763456 00002000:00100000:0.0:1780298646.145349:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (684032->688127) req@ffff9cf244ada300 x1866777609763456/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:352/0 lens 488/0 e 0 to 0 dl 1780298657 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298646.145357:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244ada300 with x1866777609763456 ext(684032->688127) 00002000:00010000:0.0:1780298646.145361:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf244ada300 00000100:00100000:0.0:1780298646.145380:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6568 00000100:00100000:0.0:1780298646.145386:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ada300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609763456:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298646.145515:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ada300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609763456:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 129us (236us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298646.145520:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6568 00002000:00100000:0.0:1780298646.145524:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (684032->688127) req@ffff9cf244ada300 x1866777609763456/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:352/0 lens 488/4536 e 0 to 0 dl 1780298657 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298646.145530:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244ada300 with x1866777609763456 ext(684032->688127) 00002000:00010000:0.0:1780298646.145533:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf244ada300 00000100:00100000:1.0:1780298646.146583:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.146651:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609763584 00000100:00100000:3.0:1780298646.146656:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609763584 00002000:00100000:3.0:1780298646.146680:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (659456->724991) req@ffff9cf2558f1880 x1866777609763584/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:352/0 lens 504/0 e 0 to 0 dl 1780298657 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298646.146690:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2558f1880 with x1866777609763584 ext(659456->724991) 00002000:00010000:3.0:1780298646.146694:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2558f1880 00000100:00100000:3.0:1780298646.146714:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6569 00000100:00100000:3.0:1780298646.146722:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609763584:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298646.147743:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609763584:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 1021us (1158us total) trans 0 rc 61440/61440 00000100:00100000:3.0:1780298646.147749:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6569 00002000:00100000:3.0:1780298646.147754:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (659456->724991) req@ffff9cf2558f1880 x1866777609763584/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:352/0 lens 504/440 e 0 to 0 dl 1780298657 ref 1 fl Complete:/600/0 rc 61440/61440 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298646.147760:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2558f1880 with x1866777609763584 ext(659456->724991) 00002000:00010000:3.0:1780298646.147763:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2558f1880 00000100:00100000:1.0:1780298646.148600:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.148625:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.148641:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609763840 00000100:00100000:2.0:1780298646.148646:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609763840 00000100:00100000:3.0:1780298646.148654:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609763712 00000100:00100000:3.0:1780298646.148658:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609763712 00000100:00100000:2.0:1780298646.148678:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10692 00000100:00100000:3.0:1780298646.148685:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10693 00000100:00100000:2.0:1780298646.148690:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609763840:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298646.148696:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2397:x1866777609763712:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.148702:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298646.148704:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298646.148714:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e90122d2/0xaf35132e630368ef lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d56d2 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.148715:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d71a6aba/0xaf35132e630368e8 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d56d9 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.148757:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244adb800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605140736:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.148759:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24339df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605140864:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298646.148809:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.148811:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6cf00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298646.149440:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244adb800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605140736:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298646.149457:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e90122d2/0xaf35132e630368ef lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d56d2 expref: 17 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.149467:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24339df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605140864:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298646.149471:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e90122d2/0xaf35132e630368ef lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d56d2 expref: 17 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.149481:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93c00, rc 0) 00010000:00010000:0.0:1780298646.149486:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d71a6aba/0xaf35132e630368e8 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d56d9 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.149498:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d71a6aba/0xaf35132e630368e8 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d56d9 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.149506:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9bc00, rc 0) 00000100:00100000:3.0:1780298646.149507:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2397:x1866777609763712:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 814us (880us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298646.149513:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10693 00000100:00100000:0.0:1780298646.149534:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609763840:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 849us (934us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.149540:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10692 00000100:00100000:0.0:1780298646.150443:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.150464:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609763968 00000100:00100000:3.0:1780298646.150468:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609763968 00000100:00100000:0.0:1780298646.150470:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.150494:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609764096 00000100:00100000:3.0:1780298646.150497:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609764096 00000100:00100000:1.0:1780298646.150510:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10694 00000100:00100000:3.0:1780298646.150516:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10695 00000100:00100000:1.0:1780298646.150522:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e427c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2398:x1866777609763968:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298646.150527:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609764096:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.150529:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298646.150535:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.150538:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e630368f6 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d56e0 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.150543:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004cdc8249/0xaf35132e630368fd lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d56e7 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.150565:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605140992:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298646.150572:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244adb800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605141120:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298646.150603:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.150604:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.151218:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605140992:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.151237:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e630368f6 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d56e0 expref: 17 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.151253:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e630368f6 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d56e0 expref: 17 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.151264:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278a00, rc 0) 00000100:00100000:0.0:1780298646.151307:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e427c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2398:x1866777609763968:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 790us (865us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.151315:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244adb800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605141120:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.151319:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10694 00010000:00010000:1.0:1780298646.151341:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004cdc8249/0xaf35132e630368fd lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d56e7 expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.151358:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004cdc8249/0xaf35132e630368fd lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d56e7 expref: 11 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.151368:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92200, rc 0) 00000100:00100000:1.0:1780298646.151393:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2397:x1866777609764096:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 871us (925us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.151399:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10695 00000100:00100000:3.0:1780298646.152262:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.152301:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609764224 00000100:00100000:1.0:1780298646.152306:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609764224 00000100:00100000:1.0:1780298646.152334:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10696 00000100:00100000:1.0:1780298646.152342:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609764224:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.152350:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.152364:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e63036904 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d56ee expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.152394:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605141248:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.152439:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.152450:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.152476:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609764352 00000100:00100000:0.0:1780298646.152481:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609764352 00000100:00100000:0.0:1780298646.152508:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10697 00000100:00100000:0.0:1780298646.152518:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609764352:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.152526:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.152534:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c703984a/0xaf35132e6303690b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d56f5 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.152560:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605141376:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.152593:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1900 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.152960:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605141248:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298646.152971:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e63036904 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d56ee expref: 17 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.152980:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e63036904 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d56ee expref: 17 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.152987:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278800, rc 0) 00000100:00100000:1.0:1780298646.153009:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609764224:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 669us (746us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.153017:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10696 00000100:00100000:0.0:1780298646.153190:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605141376:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.153206:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c703984a/0xaf35132e6303690b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d56f5 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.153219:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c703984a/0xaf35132e6303690b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d56f5 expref: 11 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.153229:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2000, rc 0) 00000100:00100000:0.0:1780298646.153254:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609764352:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 740us (820us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.153261:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10697 00000100:00100000:3.0:1780298646.154026:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.154059:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609764480 00000100:00100000:0.0:1780298646.154068:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609764480 00000100:00100000:0.0:1780298646.154098:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10698 00000100:00100000:0.0:1780298646.154106:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:29360:x1866777609764480:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.154114:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.154146:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c0d26bbc/0xaf35132e63036912 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d56fc expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.154166:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e6303682b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2142208->35696639] (req 2142208->2150399) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5615 expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.154176:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298646 v=(64 122) 00010000:00010000:0.0:1780298646.154181:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298646 v=(65 122) 00010000:00010000:0.0:1780298646.154186:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c0d26bbc/0xaf35132e63036912 lrc: 4/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [4173824->4177919] (req 4173824->4177919) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d56fc expref: 11 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.154213:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e6303682b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2142208->35696639] (req 2142208->2150399) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5615 expref: 11 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.154227:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e6303682b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2142208->35696639] (req 2142208->2150399) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5615 expref: 11 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.154238:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605141504:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298646.154284:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1900 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.154841:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605141504:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298646.154855:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c0d26bbc/0xaf35132e63036912 lrc: 3/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [4173824->4177919] (req 4173824->4177919) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d56fc expref: 11 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.154864:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3c00, rc 0) 00000100:00100000:0.0:1780298646.154889:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:29360:x1866777609764480:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 785us (862us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298646.154896:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10698 00000100:00100000:2.0:1780298646.155086:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.155135:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609764608 00000100:00100000:1.0:1780298646.155140:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609764608 00010000:00010000:1.0:1780298646.155155:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e6303682b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2142208->35696639] (req 2142208->2150399) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5615 expref: 12 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.155177:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15347 00000100:00100000:1.0:1780298646.155183:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2399:x1866777609764608:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298646.155192:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298646.155203:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e6303682b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [2142208->35696639] (req 2142208->2150399) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5615 expref: 12 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.155222:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e6303682b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [2142208->35696639] (req 2142208->2150399) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5615 expref: 12 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.155239:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e6303682b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [2142208->35696639] (req 2142208->2150399) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5615 expref: 12 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.155245:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298646 v=(65 122) 00010000:00010000:1.0:1780298646.155248:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298646 v=(65 122) 00010000:00010000:1.0:1780298646.155253:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c0d26bbc/0xaf35132e63036912 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4173824->4177919) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d56fc expref: 10 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.155258:0:6463:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c0d26bbc/0xaf35132e63036912 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4173824->4177919) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d56fc expref: 10 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.155288:0:6463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000c0d26bbc/0xaf35132e63036912 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4173824->4177919) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d56fc expref: 10 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.155295:0:6463:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f3800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777605141632:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298646.155323:0:6463:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.155959:0:6463:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f3800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777605141632:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298646.155967:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298646.155988:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609764608:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 806us (904us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298646.155993:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15347 00000100:00100000:2.0:1780298646.156160:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.156188:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609764736 00000100:00100000:1.0:1780298646.156192:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609764736 00002000:00100000:1.0:1780298646.156214:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x12:0x0] (4173824->4177919) req@ffff9cf24339c380 x1866777609764736/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:352/0 lens 488/0 e 0 to 0 dl 1780298657 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298646.156223:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24339c380 with x1866777609764736 ext(4173824->4177919) 00002000:00010000:1.0:1780298646.156227:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24339c380 00000100:00100000:1.0:1780298646.156244:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6570 00000100:00100000:1.0:1780298646.156250:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609764736:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298646.156342:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609764736:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 93us (182us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298646.156347:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6570 00002000:00100000:1.0:1780298646.156351:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x12:0x0] (4173824->4177919) req@ffff9cf24339c380 x1866777609764736/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:352/0 lens 488/440 e 0 to 0 dl 1780298657 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298646.156356:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24339c380 with x1866777609764736 ext(4173824->4177919) 00002000:00010000:1.0:1780298646.156358:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24339c380 00000100:00100000:2.0:1780298646.157343:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.157377:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609764864 00000100:00100000:0.0:1780298646.157381:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609764864 00000100:00100000:0.0:1780298646.157409:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10699 00000100:00100000:0.0:1780298646.157416:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609764864:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.157425:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.157433:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e63036919 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5703 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.157464:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605141760:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.157500:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1900 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.158096:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605141760:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.158111:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e63036919 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5703 expref: 17 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.158151:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e63036919 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5703 expref: 17 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.158162:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3200, rc 0) 00000100:00100000:0.0:1780298646.158190:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609764864:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 775us (847us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.158198:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10699 00000100:00100000:2.0:1780298646.158915:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.158938:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609764992 00000100:00100000:3.0:1780298646.158942:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609764992 00000100:00100000:3.0:1780298646.158964:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10700 00000100:00100000:3.0:1780298646.158969:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2399:x1866777609764992:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298646.158976:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298646.158981:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e63036920 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d570a expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.159007:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605141888:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298646.159046:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298646.159674:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605141888:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298646.159690:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e63036920 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d570a expref: 17 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.159705:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e63036920 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d570a expref: 17 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.159715:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92400, rc 0) 00000100:00100000:3.0:1780298646.159744:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2399:x1866777609764992:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 774us (829us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298646.159752:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10700 00000100:00100000:1.0:1780298646.160541:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.160591:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609765120 00000100:00100000:0.0:1780298646.160596:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609765120 00000100:00100000:0.0:1780298646.160622:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10701 00000100:00100000:0.0:1780298646.160631:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609765120:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.160640:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.160647:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a5a492f0/0xaf35132e63036927 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5711 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.160670:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605142016:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.160700:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1900 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.161362:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605142016:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.161374:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a5a492f0/0xaf35132e63036927 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5711 expref: 17 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.161385:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a5a492f0/0xaf35132e63036927 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5711 expref: 17 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.161392:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3600, rc 0) 00000100:00100000:0.0:1780298646.161415:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609765120:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 786us (875us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.161419:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10701 00000100:00100000:2.0:1780298646.162159:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.162190:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609765248 00000100:00100000:0.0:1780298646.162194:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609765248 00000100:00100000:0.0:1780298646.162215:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10702 00000100:00100000:0.0:1780298646.162220:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609765248:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.162226:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.162231:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e6303692e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5718 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.162249:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605142144:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.162288:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1900 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.162866:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605142144:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.162878:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e6303692e lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5718 expref: 17 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.162890:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e6303692e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5718 expref: 17 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.162897:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2600, rc 0) 00000100:00100000:0.0:1780298646.162918:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609765248:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 699us (762us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.162923:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10702 00000100:00100000:2.0:1780298646.163758:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.163792:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609765376 00000100:00100000:0.0:1780298646.163796:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609765376 00000100:00100000:0.0:1780298646.163825:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10703 00000100:00100000:0.0:1780298646.163832:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:29360:x1866777609765376:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.163842:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.163850:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e63036935 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d571f expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.163867:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e6303687f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [1204224->1261567] (req 1204224->1212415) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5669 expref: 15 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.163874:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(80 169) 00010000:00010000:0.0:1780298646.163877:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(81 169) 00010000:00010000:0.0:1780298646.163885:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e63036935 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [1245184->1253375] (req 1245184->1253375) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d571f expref: 15 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.163899:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e6303687f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [1204224->1261567] (req 1204224->1212415) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5669 expref: 15 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.163908:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e6303687f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [1204224->1261567] (req 1204224->1212415) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5669 expref: 15 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.163915:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605142272:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298646.163943:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf342ae1900 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.164598:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605142272:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298646.164609:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e63036935 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [1245184->1253375] (req 1245184->1253375) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d571f expref: 15 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.164618:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2e00, rc 0) 00000100:00100000:0.0:1780298646.164639:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:29360:x1866777609765376:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 809us (884us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298646.164644:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10703 00000100:00100000:1.0:1780298646.164815:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.164846:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609765504 00000100:00100000:2.0:1780298646.164851:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609765504 00010000:00010000:2.0:1780298646.164872:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e6303687f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [1204224->1261567] (req 1204224->1212415) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5669 expref: 16 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.164912:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15348 00000100:00100000:2.0:1780298646.164921:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f3100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2399:x1866777609765504:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298646.164930:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298646.164938:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e6303687f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 8 type: EXT [1204224->1261567] (req 1204224->1212415) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5669 expref: 16 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.164950:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e6303687f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 8 type: EXT [1204224->1261567] (req 1204224->1212415) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5669 expref: 16 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.164961:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e6303687f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 8 type: EXT [1204224->1261567] (req 1204224->1212415) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5669 expref: 16 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.164971:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(81 169) 00010000:00010000:2.0:1780298646.164975:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(81 169) 00010000:00010000:2.0:1780298646.164982:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e63036935 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [1245184->1261567] (req 1245184->1253375) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d571f expref: 14 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.164990:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e63036935 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [1245184->1261567] (req 1245184->1253375) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d571f expref: 14 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.165013:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e63036935 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [1245184->1261567] (req 1245184->1253375) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d571f expref: 14 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.165022:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24339d500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777605142400:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298646.165061:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.165729:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24339d500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777605142400:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298646.165737:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298646.165757:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f3100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2399:x1866777609765504:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 839us (950us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298646.165762:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15348 00000100:00100000:0.0:1780298646.165840:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.165878:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609765632 00000100:00100000:2.0:1780298646.165882:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609765632 00000100:00100000:2.0:1780298646.165915:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6571 00000100:00100000:2.0:1780298646.165919:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2400:x1866777609765632:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298646.166074:0:8363:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 5755015 -> 2093435 00002000:00010000:2.0:1780298646.166077:0:8363:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:2.0:1780298646.166079:0:8363:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298646 -> 1780298646 00002000:00010000:2.0:1780298646.166080:0:8363:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:2.0:1780298646.166082:0:8363:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 520 -> 520 00000100:00100000:2.0:1780298646.166104:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609765632:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 184us (263us total) trans 4294969815 rc 0/0 00000100:00100000:2.0:1780298646.166108:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6571 00000100:00100000:1.0:1780298646.167052:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.167085:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609765760 00000100:00100000:3.0:1780298646.167091:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609765760 00000100:00100000:3.0:1780298646.167145:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10704 00000100:00100000:3.0:1780298646.167156:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2399:x1866777609765760:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298646.167196:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298646.167206:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e90122d2/0xaf35132e6303693c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5726 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.167236:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605142528:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298646.167284:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00002000:00010000:2.0:1780298646.167900:0:19992:0:(ofd_lvb.c:259:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size: 2093435 -> 5755015 00000100:00100000:2.0:1780298646.167909:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad9500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605142528:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298646.167925:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e90122d2/0xaf35132e6303693c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5726 expref: 16 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.167938:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e90122d2/0xaf35132e6303693c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5726 expref: 16 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.167948:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93c00, rc 0) 00000100:00100000:2.0:1780298646.167974:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2399:x1866777609765760:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 822us (922us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298646.167980:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10704 00000100:00100000:0.0:1780298646.168746:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.168780:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609765888 00000100:00100000:2.0:1780298646.168784:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609765888 00000100:00100000:2.0:1780298646.168811:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10705 00000100:00100000:2.0:1780298646.168819:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2399:x1866777609765888:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.168826:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298646.168833:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e63036943 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d572d expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.168862:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24339ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605142656:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.168904:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.169549:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24339ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605142656:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298646.169559:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e63036943 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d572d expref: 16 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.169568:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e63036943 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d572d expref: 16 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.169574:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9a600, rc 0) 00000100:00100000:2.0:1780298646.169595:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2399:x1866777609765888:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 778us (851us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298646.169599:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10705 00000100:00100000:3.0:1780298646.170415:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.170451:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609766144 00000100:00100000:0.0:1780298646.170455:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609766144 00000100:00100000:0.0:1780298646.170483:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62097 00000100:00100000:0.0:1780298646.170488:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+45:29360:x1866777609766144:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:0.0:1780298646.170526:0:6477:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298646.170546:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9be00/0xaf35132e630366d4 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 25 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d54c5 expref: 45 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.170559:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298646.170563:0:6477:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1780298646.170717:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3600/0xaf35132e6303694a lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 24 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.170729:0:6477:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93000/0xaf35132e63036721 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 24 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d550b expref: 52 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.170738:0:6477:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3600/0xaf35132e6303694a lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 24 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.170753:0:6477:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93000/0xaf35132e63036721 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 24 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d550b expref: 52 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.170768:0:6477:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93000/0xaf35132e63036721 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 24 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d550b expref: 52 pid: 17578 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.170776:0:6477:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f3100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6477:1866777605142784:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298646.170805:0:6477:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4240 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.171401:0:6477:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f3100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:6477:1866777605142784:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298646.171423:0:6477:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3600/0xaf35132e6303694a lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 24 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.171626:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.171676:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609766272 00000100:00100000:1.0:1780298646.171681:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609766272 00010000:00010000:1.0:1780298646.171701:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93000/0xaf35132e63036721 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 24 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d550b expref: 53 pid: 17578 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.171730:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15349 00000100:00100000:1.0:1780298646.171738:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b1e050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+53:2399:x1866777609766272:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298646.171748:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298646.171753:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93000/0xaf35132e63036721 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 25 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d550b expref: 53 pid: 17578 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.171763:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93000/0xaf35132e63036721 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 25 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d550b expref: 53 pid: 17578 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.171776:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93000/0xaf35132e63036721 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 25 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d550b expref: 53 pid: 17578 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.171786:0:8374:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298646.171789:0:8374:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3600/0xaf35132e6303694a lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 24 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.171797:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3600/0xaf35132e6303694a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 24 type: IBT flags: 0x40290000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.171803:0:8374:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3600/0xaf35132e6303694a lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 24 type: IBT flags: 0x40290000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.171822:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1780298646.171839:0:6477:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3600/0xaf35132e6303694a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 23 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.171848:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3600/0xaf35132e6303694a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 23 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.171852:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b1e050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+52:2399:x1866777609766272:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 114us (225us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298646.171864:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15349 00000001:00080000:2.0:1780298646.171908:0:6477:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991134 00000004:00010000:2.0:1780298646.171936:0:6477:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 000000003663ad4d reply state 00000000fc86fe06 transno 4294991135 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3600/0xaf35132e6303694a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 23 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.171948:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3600/0xaf35132e6303694a lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 23 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.171955:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298646.171958:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1780298646.171960:0:6477:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1780298646.172018:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+47:29360:x1866777609766144:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1528us (1612us total) trans 4294991135 rc 0/0 00000100:00100000:2.0:1780298646.172025:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62097 00000020:00080000:2.0:1780298646.172029:0:6477:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1780298646.172610:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.172654:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609766400 00000100:00100000:0.0:1780298646.172660:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609766400 00000100:00100000:0.0:1780298646.172704:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10706 00000100:00100000:0.0:1780298646.172717:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:29360:x1866777609766400:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.172728:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.172738:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e63036951 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5734 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.172757:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e63036935 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [1245184->1261567] (req 1245184->1253375) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d571f expref: 17 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.172767:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000056a806f/0xaf35132e63036783 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [2088960->4964351] (req 2088960->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d556d expref: 14 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.172775:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63036759 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [4964352->5750783] (req 4964352->4972543) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5543 expref: 17 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.172783:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63036744 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [5750784->39305215] (req 5750784->5758975) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d552e expref: 17 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.172791:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000038a88423/0xaf35132e630367b4 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [1265664->2088959] (req 1265664->1269759) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d559e expref: 17 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.172799:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 5 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(81 169) 00010000:00010000:0.0:1780298646.172803:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(86 169) 00010000:00010000:0.0:1780298646.172808:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e63036951 lrc: 8/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5734 expref: 17 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.172826:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000038a88423/0xaf35132e630367b4 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [1265664->2088959] (req 1265664->1269759) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d559e expref: 17 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.172844:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000038a88423/0xaf35132e630367b4 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [1265664->2088959] (req 1265664->1269759) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d559e expref: 17 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.172855:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f1180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605142912:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298646.172899:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63036744 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [5750784->39305215] (req 5750784->5758975) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d552e expref: 17 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.172910:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63036744 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [5750784->39305215] (req 5750784->5758975) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d552e expref: 17 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.172920:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605143040:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298646.172948:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63036759 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [4964352->5750783] (req 4964352->4972543) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5543 expref: 17 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.172959:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63036759 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [4964352->5750783] (req 4964352->4972543) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5543 expref: 17 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.172969:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605143168:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298646.173007:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000056a806f/0xaf35132e63036783 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [2088960->4964351] (req 2088960->2097151) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d556d expref: 14 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.173017:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000056a806f/0xaf35132e63036783 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [2088960->4964351] (req 2088960->2097151) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d556d expref: 14 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.173043:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605143296:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298646.173076:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e63036935 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [1245184->1261567] (req 1245184->1253375) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d571f expref: 17 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.173085:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e63036935 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [1245184->1261567] (req 1245184->1253375) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d571f expref: 17 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.173095:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605143424:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298646.173120:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4300 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.173481:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f1180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605142912:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298646.173672:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605143040:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298646.173765:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605143168:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298646.173862:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605143296:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298646.173969:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f1f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605143424:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298646.173986:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e63036951 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5734 expref: 17 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.173996:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2000, rc 0) 00000100:00100000:0.0:1780298646.174023:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:29360:x1866777609766400:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1311us (1412us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298646.174031:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10706 00000100:00100000:2.0:1780298646.174290:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.174320:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609766784 00000100:00100000:0.0:1780298646.174324:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609766784 00002000:00100000:0.0:1780298646.174345:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (2228224->2240511) req@ffff9cf24339ca80 x1866777609766784/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:352/0 lens 12776/0 e 0 to 0 dl 1780298657 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00000100:00100000:3.0:1780298646.174351:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00002000:00100000:0.0:1780298646.174357:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24339ca80 with x1866777609766784 ext(2228224->2240511) 00010000:00010000:0.0:1780298646.174363:0:16972:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000056a806f/0xaf35132e63036783 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [2088960->4964351] (req 2088960->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d556d expref: 15 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.174373:0:16972:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000056a806f/0xaf35132e63036783 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [2088960->4964351] (req 2088960->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d556d expref: 15 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.174377:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00002000:00010000:0.0:1780298646.174384:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf24339ca80 00000100:00100000:1.0:1780298646.174394:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609766912 00000100:00100000:1.0:1780298646.174399:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609766912 00000100:00100000:3.0:1780298646.174404:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.174408:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 225 00000100:00100000:0.0:1780298646.174417:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+15:2399:x1866777609766784:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:2.0:1780298646.174420:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609766528 00010000:00010000:1.0:1780298646.174422:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e63036935 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [1245184->1261567] (req 1245184->1253375) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d571f expref: 17 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.174425:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609766528 00010000:00010000:2.0:1780298646.174442:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000038a88423/0xaf35132e630367b4 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [1265664->2088959] (req 1265664->1269759) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d559e expref: 18 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.174463:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15350 00000100:00100000:1.0:1780298646.174472:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adb100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2397:x1866777609766912:12345-192.168.203.25@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1780298646.174478:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15351 00010000:00010000:1.0:1780298646.174483:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1780298646.174491:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609766528:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298646.174498:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e63036935 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 8 type: EXT [1245184->1261567] (req 1245184->1253375) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d571f expref: 18 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.174504:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298646.174508:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e63036935 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 8 type: EXT [1245184->1261567] (req 1245184->1253375) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d571f expref: 18 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.174520:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e63036935 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 9 type: EXT [1245184->1261567] (req 1245184->1253375) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d571f expref: 18 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.174527:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000038a88423/0xaf35132e630367b4 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 9 type: EXT [1265664->2088959] (req 1265664->1269759) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d559e expref: 18 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.174528:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1780298646.174543:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000038a88423/0xaf35132e630367b4 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [1265664->2088959] (req 1265664->1269759) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d559e expref: 17 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.174555:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adb100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2397:x1866777609766912:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 85us (206us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298646.174567:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15350 00010000:00010000:3.0:1780298646.174578:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000038a88423/0xaf35132e630367b4 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [1265664->2088959] (req 1265664->1269759) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d559e expref: 16 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.174592:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298646.174602:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609766656 00000100:00100000:1.0:1780298646.174605:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609766656 00002000:00100000:1.0:1780298646.174625:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (4964352->5755014) req@ffff9cf244ad9180 x1866777609766656/t0(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:352/0 lens 504/0 e 0 to 0 dl 1780298657 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00000100:00100000:3.0:1780298646.174629:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609766528:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 139us (251us total) trans 0 rc 0/0 00002000:00100000:1.0:1780298646.174645:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244ad9180 with x1866777609766656 ext(4964352->5755014) 00010000:00010000:1.0:1780298646.174653:0:19950:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63036759 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [4964352->5750783] (req 4964352->4972543) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5543 expref: 16 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.174664:0:19950:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63036759 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [4964352->5750783] (req 4964352->4972543) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5543 expref: 16 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.174670:0:19950:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63036744 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [5750784->39305215] (req 5750784->5758975) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d552e expref: 16 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.174676:0:19950:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63036744 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [5750784->39305215] (req 5750784->5758975) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d552e expref: 16 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1780298646.174680:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 2 locks timeout for req ffff9cf244ad9180 00000100:00100000:1.0:1780298646.174696:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 226 00000100:00100000:1.0:1780298646.174704:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609766656:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:3.0:1780298646.174710:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15351 00000100:00100000:0.0:1780298646.174896:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2399:x1866777609766784:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 481us (605us total) trans 4294969816 rc 0/0 00000100:00100000:0.0:1780298646.174902:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 225 00002000:00100000:0.0:1780298646.174908:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (2228224->2240511) req@ffff9cf24339ca80 x1866777609766784/t4294969816(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:352/0 lens 12776/448 e 0 to 0 dl 1780298657 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298646.174914:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24339ca80 with x1866777609766784 ext(2228224->2240511) 00010000:00010000:0.0:1780298646.174919:0:16972:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000056a806f/0xaf35132e63036783 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2088960->4964351] (req 2088960->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d556d expref: 17 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.174926:0:16972:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000056a806f/0xaf35132e63036783 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2088960->4964351] (req 2088960->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d556d expref: 17 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1780298646.174935:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf24339ca80 00000100:00100000:3.0:1780298646.175602:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.175629:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609766656:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 924us (1223us total) trans 4294969817 rc 0/0 00000100:00100000:1.0:1780298646.175638:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 226 00000100:00100000:2.0:1780298646.175645:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609767040 00002000:00100000:1.0:1780298646.175646:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x2c0000401:0x11:0x0] (4964352->5755014) req@ffff9cf244ad9180 x1866777609766656/t4294969817(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:352/0 lens 504/448 e 0 to 0 dl 1780298657 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00000100:00100000:2.0:1780298646.175651:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609767040 00002000:00100000:1.0:1780298646.175662:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf244ad9180 with x1866777609766656 ext(4964352->5755014) 00010000:00010000:2.0:1780298646.175673:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000056a806f/0xaf35132e63036783 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [2088960->4964351] (req 2088960->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d556d expref: 17 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.175673:0:19950:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63036759 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [4964352->5750783] (req 4964352->4972543) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5543 expref: 17 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.175700:0:19950:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63036759 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [4964352->5750783] (req 4964352->4972543) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5543 expref: 17 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.175711:0:19950:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63036744 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [5750784->39305215] (req 5750784->5758975) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d552e expref: 17 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.175714:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15352 00010000:00010000:1.0:1780298646.175724:0:19950:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63036744 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [5750784->39305215] (req 5750784->5758975) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d552e expref: 17 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.175728:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adb800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2398:x1866777609767040:12345-192.168.203.25@tcp:103:ldlm_bl.0 00002000:00010000:1.0:1780298646.175734:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 2 locks timeout for req ffff9cf244ad9180 00010000:00010000:2.0:1780298646.175740:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:2.0:1780298646.175748:0:6462:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 520 -> 576 00010000:00010000:2.0:1780298646.175754:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000056a806f/0xaf35132e63036783 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [2088960->4964351] (req 2088960->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d556d expref: 17 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.175771:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000056a806f/0xaf35132e63036783 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [2088960->4964351] (req 2088960->2097151) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d556d expref: 17 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.175784:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000056a806f/0xaf35132e63036783 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [2088960->4964351] (req 2088960->2097151) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d556d expref: 17 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.175794:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298646.175817:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adb800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2398:x1866777609767040:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 94us (216us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298646.175823:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15352 00000100:00100000:0.0:1780298646.176222:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.176262:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609767168 00000100:00100000:1.0:1780298646.176277:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609767168 00010000:00010000:1.0:1780298646.176295:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63036759 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [4964352->5750783] (req 4964352->4972543) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5543 expref: 17 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.176320:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15353 00000100:00100000:1.0:1780298646.176328:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2397:x1866777609767168:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298646.176336:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298646.176347:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63036759 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [4964352->5750783] (req 4964352->4972543) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5543 expref: 17 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.176358:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63036759 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [4964352->5750783] (req 4964352->4972543) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5543 expref: 17 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.176371:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63036759 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [4964352->5750783] (req 4964352->4972543) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5543 expref: 17 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.176381:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298646.176386:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.176409:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609767296 00000100:00100000:0.0:1780298646.176414:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609767296 00000100:00100000:1.0:1780298646.176420:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609767168:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 91us (196us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298646.176427:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15353 00010000:00010000:0.0:1780298646.176436:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63036744 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [5750784->39305215] (req 5750784->5758975) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d552e expref: 17 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.176468:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15354 00000100:00100000:0.0:1780298646.176478:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25488aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2399:x1866777609767296:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298646.176488:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298646.176497:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63036744 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [5750784->39305215] (req 5750784->5758975) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d552e expref: 16 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.176508:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63036744 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [5750784->39305215] (req 5750784->5758975) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d552e expref: 16 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.176520:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63036744 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [5750784->39305215] (req 5750784->5758975) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d552e expref: 16 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.176530:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(86 169) 00010000:00010000:0.0:1780298646.176534:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(86 169) 00010000:00010000:0.0:1780298646.176539:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e63036951 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5734 expref: 15 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.176554:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e63036951 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5734 expref: 15 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.176578:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e63036951 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5734 expref: 15 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.176591:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f2d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777605143552:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298646.176632:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4300 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.177197:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f2d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777605143552:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298646.177209:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298646.177235:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25488aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2399:x1866777609767296:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 760us (852us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298646.177246:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15354 00000100:00100000:3.0:1780298646.177318:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.177341:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609767424 00000100:00100000:0.0:1780298646.177345:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609767424 00000100:00100000:0.0:1780298646.177374:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10707 00000100:00100000:0.0:1780298646.177382:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25488b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:29360:x1866777609767424:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.177392:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.177404:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008361787d/0xaf35132e63036958 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d573b expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.177422:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000c0d26bbc/0xaf35132e63036912 lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4173824->4177919) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d56fc expref: 10 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.177435:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000a6416acb/0xaf35132e6303685c lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [372736->2142207] (req 372736->376831) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5646 expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.177442:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298646 v=(65 122) 00010000:00010000:0.0:1780298646.177445:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298646 v=(66 122) 00010000:00010000:0.0:1780298646.177450:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008361787d/0xaf35132e63036958 lrc: 5/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d573b expref: 12 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.177469:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000a6416acb/0xaf35132e6303685c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [372736->2142207] (req 372736->376831) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5646 expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.177480:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000a6416acb/0xaf35132e6303685c lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [372736->2142207] (req 372736->376831) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5646 expref: 10 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.177490:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605143680:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298646.177536:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000c0d26bbc/0xaf35132e63036912 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4173824->4177919) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d56fc expref: 10 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.177546:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000c0d26bbc/0xaf35132e63036912 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4173824->4177919) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d56fc expref: 10 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.177556:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605143808:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298646.177586:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4300 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.177997:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605143680:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298646.178155:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605143808:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298646.178172:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008361787d/0xaf35132e63036958 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d573b expref: 12 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.178184:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2800, rc 0) 00000100:00100000:0.0:1780298646.178216:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25488b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:29360:x1866777609767424:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 835us (897us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298646.178224:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10707 00000100:00100000:3.0:1780298646.178352:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.178385:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609767552 00000100:00100000:0.0:1780298646.178390:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609767552 00010000:00010000:0.0:1780298646.178406:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000a6416acb/0xaf35132e6303685c lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [372736->2142207] (req 372736->376831) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5646 expref: 11 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.178432:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15355 00000100:00100000:0.0:1780298646.178440:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254888a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609767552:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298646.178449:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298646.178463:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000a6416acb/0xaf35132e6303685c lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [372736->2142207] (req 372736->376831) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5646 expref: 11 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.178474:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000a6416acb/0xaf35132e6303685c lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [372736->2142207] (req 372736->376831) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5646 expref: 11 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.178489:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6416acb/0xaf35132e6303685c lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [372736->2142207] (req 372736->376831) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5646 expref: 11 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.178504:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:0.0:1780298646.178510:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298646.178540:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254888a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609767552:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 101us (188us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298646.178548:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15355 00000100:00100000:0.0:1780298646.178561:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609767680 00000100:00100000:0.0:1780298646.178564:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609767680 00010000:00010000:0.0:1780298646.178576:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000c0d26bbc/0xaf35132e63036912 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 4173824->4177919) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d56fc expref: 10 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.178599:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15356 00000100:00100000:0.0:1780298646.178605:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b1e450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609767680:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298646.178613:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298646.178621:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000c0d26bbc/0xaf35132e63036912 lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4173824->4177919) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d56fc expref: 10 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.178631:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000c0d26bbc/0xaf35132e63036912 lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4173824->4177919) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d56fc expref: 10 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.178643:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c0d26bbc/0xaf35132e63036912 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 4173824->4177919) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d56fc expref: 10 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.178653:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298646 v=(66 122) 00010000:00010000:0.0:1780298646.178657:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298646 v=(66 122) 00010000:00010000:0.0:1780298646.178662:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000008361787d/0xaf35132e63036958 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d573b expref: 11 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.178671:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008361787d/0xaf35132e63036958 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d573b expref: 11 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.178693:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000008361787d/0xaf35132e63036958 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d573b expref: 11 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.178702:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f1880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:8374:1866777605143936:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298646.178739:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4300 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.179291:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f1880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:8374:1866777605143936:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298646.179298:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298646.179316:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b1e450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609767680:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 713us (818us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298646.179321:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15356 00000100:00100000:2.0:1780298646.179442:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.179475:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609767808 00000100:00100000:0.0:1780298646.179479:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609767808 00000100:00100000:2.0:1780298646.179495:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00002000:00010000:0.0:1780298646.179499:0:8362:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (1737119->18446744073709551615) 00002000:00100000:0.0:1780298646.179503:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24339ed80 with x1866777609767808 ext(1737119->18446744073709551615) 00002000:00010000:0.0:1780298646.179509:0:8362:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24339ed80. 00000100:00100000:0.0:1780298646.179533:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609767936 00000100:00100000:0.0:1780298646.179537:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609767936 00000100:00100000:1.0:1780298646.179541:0:19928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6572 00002000:00010000:0.0:1780298646.179550:0:8362:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00000100:00100000:1.0:1780298646.179551:0:19928:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+15:2400:x1866777609767808:12345-192.168.203.25@tcp:10:fsx.0 00002000:00100000:0.0:1780298646.179557:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24339df80 with x1866777609767936 ext(0->18446744073709551615) 00002000:00010000:0.0:1780298646.179562:0:8362:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24339df80. 00000100:00100000:0.0:1780298646.179585:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6573 00000100:00100000:0.0:1780298646.179596:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+12:2400:x1866777609767936:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:0.0:1780298646.179894:0:8362:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 2651671 -> 0 00002000:00010000:0.0:1780298646.179901:0:8362:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298646.179905:0:8362:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298646 -> 1780298646 00002000:00010000:1.0:1780298646.179907:0:19928:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 5755015 -> 1737119 00002000:00010000:0.0:1780298646.179907:0:8362:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298646.179911:0:8362:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 32 -> 0 00002000:00010000:1.0:1780298646.179916:0:19928:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:1.0:1780298646.179918:0:19928:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298646 -> 1780298646 00002000:00010000:1.0:1780298646.179920:0:19928:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:1.0:1780298646.179922:0:19928:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 576 -> 504 00000100:00100000:0.0:1780298646.179944:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2400:x1866777609767936:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 351us (448us total) trans 4294969339 rc 0/0 00000100:00100000:1.0:1780298646.179953:0:19928:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609767808:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 402us (511us total) trans 4294969818 rc 0/0 00000100:00100000:0.0:1780298646.179957:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6573 00002000:00010000:0.0:1780298646.179963:0:8362:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00000100:00100000:1.0:1780298646.179968:0:19928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6572 00002000:00100000:0.0:1780298646.179968:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24339df80 with x1866777609767936 ext(0->18446744073709551615) 00002000:00010000:1.0:1780298646.179974:0:19928:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (1737119->18446744073709551615) 00002000:00010000:0.0:1780298646.179976:0:8362:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf24339df80. 00002000:00100000:1.0:1780298646.179980:0:19928:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24339ed80 with x1866777609767808 ext(1737119->18446744073709551615) 00002000:00010000:1.0:1780298646.179987:0:19928:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24339ed80. 00000100:00100000:2.0:1780298646.180855:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.180896:0:17567:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609768064 00000100:00100000:0.0:1780298646.180901:0:17567:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609768064 00000100:00100000:0.0:1780298646.180935:0:17567:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62098 00000100:00100000:0.0:1780298646.180942:0:17567:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+47:29360:x1866777609768064:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.180958:0:17567:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.180971:0:17567:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3200/0xaf35132e6303695f lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 24 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5742 expref: -99 pid: 17567 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.181050:0:17567:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2600/0xaf35132e63036966 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 25 type: IBT flags: 0x40000000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.181063:0:17567:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2600/0xaf35132e63036966 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 25 type: IBT flags: 0x50210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.181072:0:17567:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2600/0xaf35132e63036966 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 25 type: IBT flags: 0x40210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1780298646.181099:0:17567:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 1737119 00000004:00200000:0.0:1780298646.181103:0:17567:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1780298646.181121:0:17567:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2600/0xaf35132e63036966 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 25 type: IBT flags: 0x40210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.181162:0:17567:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3200/0xaf35132e6303695f lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 25 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5742 expref: 49 pid: 17567 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.181175:0:17567:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2600/0xaf35132e63036966 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 24 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5742 expref: 48 pid: 17567 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.181183:0:17567:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298646.181186:0:17567:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1780298646.181189:0:17567:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2600, rc 0) 00000100:00100000:0.0:1780298646.181217:0:17567:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+48:29360:x1866777609768064:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 276us (363us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298646.181223:0:17567:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62098 00000100:00100000:2.0:1780298646.182878:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.182918:0:9976:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609768320 00000100:00100000:0.0:1780298646.182922:0:9976:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609768320 00000100:00100000:0.0:1780298646.182948:0:9976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62099 00000100:00100000:0.0:1780298646.182954:0:9976:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+48:29360:x1866777609768320:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:0.0:1780298646.182989:0:9976:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298646.182999:0:9976:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2600/0xaf35132e63036966 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 25 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5742 expref: 48 pid: 17567 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.183009:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298646.183011:0:9976:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1780298646.183103:0:9976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2200/0xaf35132e6303696d lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 24 type: IBT flags: 0x40000000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.183113:0:9976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2200/0xaf35132e6303696d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 24 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.183118:0:9976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2200/0xaf35132e6303696d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 24 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1780298646.183175:0:9976:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991136 00000004:00010000:0.0:1780298646.183189:0:9976:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 00000000ec21e1d6 reply state 00000000921fbafa transno 4294991137 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2200/0xaf35132e6303696d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 24 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.183197:0:9976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff2200/0xaf35132e6303696d lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 24 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.183202:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298646.183203:0:9976:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1780298646.183205:0:9976:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1780298646.183244:0:9976:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+50:29360:x1866777609768320:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 290us (368us total) trans 4294991137 rc 0/0 00000100:00100000:0.0:1780298646.183248:0:9976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62099 00000020:00080000:0.0:1780298646.183251:0:9976:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1780298646.183943:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.183989:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609768448 00000100:00100000:2.0:1780298646.183994:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609768448 00002000:00010000:2.0:1780298646.184012:0:8363:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4156114->18446744073709551615) 00000100:00100000:3.0:1780298646.184014:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00002000:00100000:2.0:1780298646.184019:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf254889c00 with x1866777609768448 ext(4156114->18446744073709551615) 00002000:00010000:2.0:1780298646.184025:0:8363:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf254889c00. 00000100:00100000:1.0:1780298646.184053:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609768576 00000100:00100000:1.0:1780298646.184057:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609768576 00000100:00100000:2.0:1780298646.184059:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6574 00000100:00100000:2.0:1780298646.184070:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254889c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609768448:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:1.0:1780298646.184073:0:19949:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00002000:00100000:1.0:1780298646.184077:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf254888000 with x1866777609768576 ext(0->18446744073709551615) 00002000:00010000:1.0:1780298646.184081:0:19949:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf254888000. 00000100:00100000:1.0:1780298646.184098:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6575 00000100:00100000:1.0:1780298646.184105:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254888000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2397:x1866777609768576:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:1.0:1780298646.184203:0:19949:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:1.0:1780298646.184206:0:19949:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:1.0:1780298646.184208:0:19949:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298646 -> 1780298646 00002000:00010000:1.0:1780298646.184210:0:19949:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:1.0:1780298646.184211:0:19949:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 0 -> 0 00002000:00010000:2.0:1780298646.184214:0:8363:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 1737119 -> 4156114 00002000:00010000:2.0:1780298646.184218:0:8363:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:2.0:1780298646.184221:0:8363:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298646 -> 1780298646 00002000:00010000:2.0:1780298646.184225:0:8363:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:2.0:1780298646.184227:0:8363:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 504 -> 504 00000100:00100000:1.0:1780298646.184236:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254888000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2397:x1866777609768576:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 133us (261us total) trans 4294969340 rc 0/0 00000100:00100000:1.0:1780298646.184241:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6575 00002000:00010000:1.0:1780298646.184244:0:19949:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (0->18446744073709551615) 00002000:00100000:1.0:1780298646.184246:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf254888000 with x1866777609768576 ext(0->18446744073709551615) 00002000:00010000:1.0:1780298646.184250:0:19949:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf254888000. 00000100:00100000:2.0:1780298646.184259:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254889c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2397:x1866777609768448:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 189us (377us total) trans 4294969819 rc 0/0 00000100:00100000:2.0:1780298646.184278:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6574 00002000:00010000:2.0:1780298646.184282:0:8363:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4156114->18446744073709551615) 00002000:00100000:2.0:1780298646.184286:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf254889c00 with x1866777609768448 ext(4156114->18446744073709551615) 00002000:00010000:2.0:1780298646.184291:0:8363:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf254889c00. 00000100:00100000:2.0:1780298646.185002:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.185066:0:17578:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609768704 00000100:00100000:3.0:1780298646.185076:0:17578:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609768704 00000100:00100000:3.0:1780298646.185118:0:17578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62100 00000100:00100000:3.0:1780298646.185158:0:17578:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+50:29360:x1866777609768704:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298646.185176:0:17578:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298646.185191:0:17578:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93600/0xaf35132e63036974 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 25 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5749 expref: -99 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.185256:0:17578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93e00/0xaf35132e6303697b lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 26 type: IBT flags: 0x40000000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.185279:0:17578:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93e00/0xaf35132e6303697b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 26 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.185293:0:17578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93e00/0xaf35132e6303697b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 26 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1780298646.185312:0:17578:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 4156114 00000004:00200000:3.0:1780298646.185315:0:17578:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1780298646.185331:0:17578:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93e00/0xaf35132e6303697b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 26 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.185353:0:17578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93600/0xaf35132e63036974 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 26 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5749 expref: 52 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.185370:0:17578:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93e00/0xaf35132e6303697b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 25 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5749 expref: 51 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.185376:0:17578:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:3.0:1780298646.185377:0:17578:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1780298646.185380:0:17578:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93e00, rc 0) 00000100:00100000:3.0:1780298646.185432:0:17578:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+51:29360:x1866777609768704:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 308us (432us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298646.185438:0:17578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62100 00000100:00100000:0.0:1780298646.187286:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.187323:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609768832 00000100:00100000:2.0:1780298646.187330:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609768832 00000100:00100000:2.0:1780298646.187367:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62101 00000100:00100000:2.0:1780298646.187376:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+52:29360:x1866777609768832:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.187389:0:6477:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298646.187400:0:6477:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a200/0xaf35132e63036982 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 26 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5750 expref: -99 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.187456:0:6477:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9bc00/0xaf35132e63036989 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 27 type: IBT flags: 0x40000000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.187465:0:6477:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9bc00/0xaf35132e63036989 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 27 type: IBT flags: 0x50210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.187471:0:6477:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9bc00/0xaf35132e63036989 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 27 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298646.187492:0:6477:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 4156114 00000004:00200000:2.0:1780298646.187496:0:6477:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298646.187511:0:6477:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9bc00/0xaf35132e63036989 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 27 type: IBT flags: 0x40210000000000 pid: 6477 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.187534:0:6477:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9a200/0xaf35132e63036982 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 27 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5750 expref: 54 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.187550:0:6477:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9bc00/0xaf35132e63036989 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 26 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5750 expref: 53 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.187557:0:6477:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298646.187559:0:6477:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298646.187562:0:6477:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9bc00, rc 0) 00000100:00100000:2.0:1780298646.187607:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+53:29360:x1866777609768832:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 232us (321us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298646.187616:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62101 00000100:00100000:1.0:1780298646.188455:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.188488:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.188510:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609768960 00000100:00100000:2.0:1780298646.188515:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609768960 00000100:00100000:3.0:1780298646.188539:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609769088 00000100:00100000:3.0:1780298646.188546:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609769088 00000100:00100000:2.0:1780298646.188559:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10708 00000100:00100000:2.0:1780298646.188570:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609768960:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.188579:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298646.188591:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e63036990 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5757 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.188596:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10709 00000100:00100000:3.0:1780298646.188615:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609769088:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298646.188628:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298646.188642:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e63036997 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d575e expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.188649:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24339d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605144064:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298646.188714:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf254888380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605144192:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.188739:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6cf00 going to sleep for 11 seconds 00000100:00100000:3.0:1780298646.188771:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2418563c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.189472:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24339d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605144064:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298646.189501:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e63036990 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5757 expref: 17 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.189517:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e63036990 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5757 expref: 17 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.189529:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9a600, rc 0) 00000100:00100000:2.0:1780298646.189562:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:2400:x1866777609768960:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 997us (1109us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298646.189567:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10708 00000100:00100000:1.0:1780298646.189587:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf254888380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605144192:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298646.189609:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e63036997 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d575e expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.189624:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c361b03c/0xaf35132e63036997 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d575e expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.189635:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92a00, rc 0) 00000100:00100000:1.0:1780298646.189664:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609769088:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1055us (1176us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.189669:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10709 00000100:00100000:3.0:1780298646.190507:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.190546:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609769216 00000100:00100000:0.0:1780298646.190552:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609769216 00000100:00100000:0.0:1780298646.190588:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10710 00000100:00100000:0.0:1780298646.190599:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254888380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+16:29360:x1866777609769216:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.190609:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.190619:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e6303699e lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5765 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.190634:0:8356:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e63036951 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5734 expref: 16 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.190640:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(86 169) 00010000:00010000:0.0:1780298646.190643:0:8356:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(87 169) 00010000:00010000:0.0:1780298646.190646:0:8356:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e6303699e lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [4939776->4947967] (req 4939776->4947967) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5765 expref: 17 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.190663:0:8356:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e63036951 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5734 expref: 16 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.190673:0:8356:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e63036951 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5734 expref: 16 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.190681:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605144320:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298646.190719:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4f00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.191299:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605144320:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298646.191312:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e6303699e lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [4939776->4947967] (req 4939776->4947967) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5765 expref: 17 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.191320:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2600, rc 0) 00000100:00100000:0.0:1780298646.191345:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254888380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:29360:x1866777609769216:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 751us (840us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298646.191352:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10710 00000100:00100000:2.0:1780298646.191499:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.191534:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609769344 00000100:00100000:1.0:1780298646.191543:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609769344 00010000:00010000:1.0:1780298646.191559:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e63036951 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5734 expref: 17 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.191583:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15357 00000100:00100000:1.0:1780298646.191588:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609769344:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298646.191595:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298646.191604:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e63036951 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5734 expref: 17 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.191618:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e63036951 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5734 expref: 17 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.191629:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e63036951 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5734 expref: 17 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.191636:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(87 169) 00010000:00010000:1.0:1780298646.191638:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(87 169) 00010000:00010000:1.0:1780298646.191643:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e6303699e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [4939776->38494207] (req 4939776->4947967) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5765 expref: 16 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.191649:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e6303699e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [4939776->38494207] (req 4939776->4947967) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5765 expref: 16 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.191664:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e6303699e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [4939776->38494207] (req 4939776->4947967) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5765 expref: 16 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.191674:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f3480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777605144448:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298646.191702:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e900 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.192285:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f3480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:6461:1866777605144448:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298646.192298:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298646.192326:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2400:x1866777609769344:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 738us (828us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298646.192332:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15357 00000100:00100000:2.0:1780298646.192473:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.192512:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609769472 00000100:00100000:1.0:1780298646.192517:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609769472 00000100:00100000:1.0:1780298646.192556:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6576 00000100:00100000:1.0:1780298646.192564:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243062300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+17:2400:x1866777609769472:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298646.192723:0:19950:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4156114 -> 4944749 00002000:00010000:1.0:1780298646.192727:0:19950:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:1.0:1780298646.192729:0:19950:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298646 -> 1780298646 00002000:00010000:1.0:1780298646.192731:0:19950:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:1.0:1780298646.192734:0:19950:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 504 -> 520 00000100:00100000:1.0:1780298646.192764:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243062300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2400:x1866777609769472:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 201us (293us total) trans 4294969820 rc 0/0 00000100:00100000:1.0:1780298646.192772:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6576 00000100:00100000:3.0:1780298646.193596:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.193635:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.193646:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609769600 00000100:00100000:0.0:1780298646.193651:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609769600 00000100:00100000:0.0:1780298646.193675:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609769728 00000100:00100000:0.0:1780298646.193677:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609769728 00000100:00100000:2.0:1780298646.193687:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10711 00000100:00100000:2.0:1780298646.193698:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25488aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2397:x1866777609769600:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.193707:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298646.193708:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10712 00000100:00100000:0.0:1780298646.193718:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254888a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609769728:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.193721:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009fd8d343/0xaf35132e630369a5 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d576c expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.193727:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.193738:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000101214ae/0xaf35132e630369ac lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5773 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.193758:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243060000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605144576:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.193766:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605144704:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.193811:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6cf00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.193812:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4f00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.194491:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243060000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605144576:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298646.194513:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009fd8d343/0xaf35132e630369a5 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d576c expref: 19 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.194525:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009fd8d343/0xaf35132e630369a5 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d576c expref: 19 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.194537:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b000, rc 0) 00000100:00100000:0.0:1780298646.194555:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605144704:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.194569:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25488aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2397:x1866777609769600:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 873us (1011us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298646.194576:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10711 00010000:00010000:0.0:1780298646.194607:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000101214ae/0xaf35132e630369ac lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5773 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.194629:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000101214ae/0xaf35132e630369ac lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5773 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.194643:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3200, rc 0) 00000100:00100000:0.0:1780298646.194681:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254888a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609769728:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 965us (1047us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.194688:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10712 00000100:00100000:3.0:1780298646.195450:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.195546:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609769856 00000100:00100000:1.0:1780298646.195552:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609769856 00000100:00100000:3.0:1780298646.195565:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.195620:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609769984 00000100:00100000:2.0:1780298646.195626:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609769984 00000100:00100000:1.0:1780298646.195626:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10713 00000100:00100000:1.0:1780298646.195638:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2399:x1866777609769856:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.195651:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.195660:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e630369b3 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d577a expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.195671:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10714 00000100:00100000:2.0:1780298646.195683:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609769984:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.195690:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298646.195698:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000019780e8d/0xaf35132e630369ba lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5781 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.195728:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243060000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605144832:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.195769:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605144960:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.195772:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6cf00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.195933:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e900 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.196357:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243060000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605144832:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298646.196373:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000019780e8d/0xaf35132e630369ba lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5781 expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.196388:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000019780e8d/0xaf35132e630369ba lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5781 expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.196398:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9a200, rc 0) 00000100:00100000:2.0:1780298646.196431:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609769984:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 753us (866us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298646.196438:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10714 00000100:00100000:1.0:1780298646.196702:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605144960:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298646.196718:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e630369b3 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d577a expref: 19 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.196732:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e630369b3 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d577a expref: 19 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.196742:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278800, rc 0) 00000100:00100000:1.0:1780298646.196816:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2399:x1866777609769856:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1165us (1349us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.196823:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10713 00000100:00100000:2.0:1780298646.197670:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.197709:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609770112 00000100:00100000:3.0:1780298646.197717:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609770112 00000100:00100000:3.0:1780298646.197757:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10715 00000100:00100000:3.0:1780298646.197769:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243060000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2398:x1866777609770112:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298646.197782:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298646.197794:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e630369c1 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5788 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.197838:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bf800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605145088:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298646.197899:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856000 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.198504:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bf800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605145088:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298646.198524:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e630369c1 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5788 expref: 17 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.198538:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e630369c1 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5788 expref: 17 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.198548:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93600, rc 0) 00000100:00100000:2.0:1780298646.198580:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243060000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2398:x1866777609770112:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 815us (910us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298646.198587:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10715 00000100:00100000:2.0:1780298646.199339:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.199381:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609770240 00000100:00100000:0.0:1780298646.199386:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609770240 00002000:00100000:0.0:1780298646.199413:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (1028096->1032191) req@ffff9cf243063480 x1866777609770240/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:352/0 lens 488/0 e 0 to 0 dl 1780298657 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298646.199423:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243063480 with x1866777609770240 ext(1028096->1032191) 00002000:00010000:0.0:1780298646.199429:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf243063480 00000100:00100000:0.0:1780298646.199452:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6577 00000100:00100000:0.0:1780298646.199458:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243063480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609770240:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298646.199598:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243063480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2400:x1866777609770240:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 139us (258us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298646.199606:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6577 00002000:00100000:0.0:1780298646.199614:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (1028096->1032191) req@ffff9cf243063480 x1866777609770240/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:352/0 lens 488/440 e 0 to 0 dl 1780298657 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298646.199622:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243063480 with x1866777609770240 ext(1028096->1032191) 00002000:00010000:0.0:1780298646.199625:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf243063480 00000100:00100000:3.0:1780298646.200384:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.200412:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609770368 00000100:00100000:3.0:1780298646.200417:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.200417:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609770368 00000100:00100000:0.0:1780298646.200450:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609770496 00000100:00100000:0.0:1780298646.200455:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609770496 00000100:00100000:2.0:1780298646.200482:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10716 00000100:00100000:0.0:1780298646.200498:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10717 00000100:00100000:2.0:1780298646.200509:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7be680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2398:x1866777609770368:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298646.200518:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7be300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609770496:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.200528:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.200530:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298646.200541:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e630369c8 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d578f expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.200543:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c0d26bbc/0xaf35132e630369cf lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5796 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.200609:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605145216:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.200617:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605145344:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.200682:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6cf00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.200683:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4f00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.201312:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605145216:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.201315:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605145344:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298646.201352:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e630369c8 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d578f expref: 19 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.201356:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c0d26bbc/0xaf35132e630369cf lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5796 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.201394:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e630369c8 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d578f expref: 19 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.201402:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c0d26bbc/0xaf35132e630369cf lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5796 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.201406:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9a600, rc 0) 00010000:00010000:0.0:1780298646.201423:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3c00, rc 0) 00000100:00100000:2.0:1780298646.201448:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7be680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2398:x1866777609770368:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 954us (1061us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.201457:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7be300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609770496:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 945us (1034us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298646.201469:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10716 00000100:00100000:0.0:1780298646.201480:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10717 00000100:00100000:3.0:1780298646.202227:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.202285:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609770624 00000100:00100000:1.0:1780298646.202293:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609770624 00000100:00100000:1.0:1780298646.202349:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10718 00000100:00100000:1.0:1780298646.202359:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2397:x1866777609770624:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298646.202371:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:1.0:1780298646.202392:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298646.202396:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609770752 00000100:00100000:2.0:1780298646.202401:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609770752 00010000:00010000:1.0:1780298646.202407:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e630369d6 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d579d expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.202433:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10719 00000100:00100000:2.0:1780298646.202442:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609770752:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.202451:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298646.202463:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad103424/0xaf35132e630369dd lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d57a4 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.202467:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605145472:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.202500:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605145600:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.202547:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e900 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.202550:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6cf00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.203260:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605145600:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298646.203291:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ad103424/0xaf35132e630369dd lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d57a4 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.203307:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad103424/0xaf35132e630369dd lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d57a4 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.203323:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9aa00, rc 0) 00000100:00100000:1.0:1780298646.203342:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605145472:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.203354:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609770752:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 913us (989us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298646.203364:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10719 00010000:00010000:1.0:1780298646.203364:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e630369d6 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d579d expref: 19 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.203384:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e630369d6 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d579d expref: 19 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.203399:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279800, rc 0) 00000100:00100000:1.0:1780298646.203614:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2397:x1866777609770624:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1255us (1384us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.203633:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10718 00000100:00100000:3.0:1780298646.204406:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.204438:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609770880 00000100:00100000:1.0:1780298646.204444:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609770880 00000100:00100000:1.0:1780298646.204480:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10720 00000100:00100000:1.0:1780298646.204488:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609770880:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.204498:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.204506:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e630369e4 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d57ab expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.204537:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605145728:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.204577:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e900 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.205301:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605145728:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298646.205316:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e630369e4 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d57ab expref: 17 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.205326:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e630369e4 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d57ab expref: 17 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.205332:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279200, rc 0) 00000100:00100000:1.0:1780298646.205358:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609770880:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 873us (954us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.205363:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10720 00000100:00100000:3.0:1780298646.206290:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.206332:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609771008 00000100:00100000:0.0:1780298646.206337:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609771008 00000100:00100000:0.0:1780298646.206367:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10721 00000100:00100000:0.0:1780298646.206373:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609771008:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.206382:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.206392:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001d4afe0b/0xaf35132e630369eb lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d57b2 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.206418:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605145856:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.206458:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4f00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.207058:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605145856:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.207073:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001d4afe0b/0xaf35132e630369eb lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d57b2 expref: 17 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.207087:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001d4afe0b/0xaf35132e630369eb lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d57b2 expref: 17 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.207095:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3000, rc 0) 00000100:00100000:0.0:1780298646.207117:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609771008:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 745us (829us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.207141:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10721 00000100:00100000:3.0:1780298646.207816:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.207857:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609771136 00000100:00100000:1.0:1780298646.207866:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609771136 00002000:00100000:1.0:1780298646.207898:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x12:0x0] (884736->888831) req@ffff9cf37f7bf800 x1866777609771136/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:352/0 lens 488/0 e 0 to 0 dl 1780298657 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298646.207909:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f7bf800 with x1866777609771136 ext(884736->888831) 00002000:00010000:1.0:1780298646.207914:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f7bf800 00000100:00100000:1.0:1780298646.207940:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6578 00000100:00100000:1.0:1780298646.207949:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2398:x1866777609771136:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298646.208102:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2398:x1866777609771136:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 154us (285us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298646.208114:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6578 00002000:00100000:1.0:1780298646.208122:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x12:0x0] (884736->888831) req@ffff9cf37f7bf800 x1866777609771136/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:352/0 lens 488/440 e 0 to 0 dl 1780298657 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298646.208164:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f7bf800 with x1866777609771136 ext(884736->888831) 00002000:00010000:1.0:1780298646.208168:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f7bf800 00000100:00100000:3.0:1780298646.208880:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.208920:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609771264 00000100:00100000:2.0:1780298646.208926:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609771264 00000100:00100000:2.0:1780298646.208959:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10722 00000100:00100000:2.0:1780298646.208970:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609771264:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.208980:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298646.208991:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e630369f2 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d57b9 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.209020:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605145984:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.209066:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6cf00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.209679:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605145984:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298646.209696:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e630369f2 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d57b9 expref: 17 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.209712:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e630369f2 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d57b9 expref: 17 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.209722:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9a200, rc 0) 00000100:00100000:2.0:1780298646.209755:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609771264:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 790us (875us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298646.209761:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10722 00000100:00100000:3.0:1780298646.210391:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.210426:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609771392 00000100:00100000:0.0:1780298646.210431:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609771392 00002000:00100000:0.0:1780298646.210458:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x12:0x0] (888832->892927) req@ffff9cf3481fea00 x1866777609771392/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:352/0 lens 488/0 e 0 to 0 dl 1780298657 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298646.210465:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fea00 with x1866777609771392 ext(888832->892927) 00002000:00010000:0.0:1780298646.210468:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf3481fea00 00000100:00100000:0.0:1780298646.210486:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6579 00000100:00100000:0.0:1780298646.210491:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2398:x1866777609771392:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298646.210602:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2398:x1866777609771392:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 109us (211us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298646.210609:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6579 00002000:00100000:0.0:1780298646.210615:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x12:0x0] (888832->892927) req@ffff9cf3481fea00 x1866777609771392/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:352/0 lens 488/440 e 0 to 0 dl 1780298657 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298646.210623:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fea00 with x1866777609771392 ext(888832->892927) 00002000:00010000:0.0:1780298646.210627:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf3481fea00 00000100:00100000:3.0:1780298646.211386:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.211421:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609771520 00000100:00100000:1.0:1780298646.211428:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609771520 00000100:00100000:1.0:1780298646.211462:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10723 00000100:00100000:1.0:1780298646.211470:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2398:x1866777609771520:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.211479:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.211488:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e630369f9 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d57c0 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.211518:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605146112:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.211565:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e900 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.212256:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605146112:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298646.212285:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e630369f9 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d57c0 expref: 17 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.212299:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002ff2dd4d/0xaf35132e630369f9 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d57c0 expref: 17 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.212308:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278800, rc 0) 00000100:00100000:1.0:1780298646.212339:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2398:x1866777609771520:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 870us (952us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.212376:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10723 00000100:00100000:3.0:1780298646.213074:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.213120:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609771648 00000100:00100000:2.0:1780298646.213151:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609771648 00002000:00100000:2.0:1780298646.213181:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x12:0x0] (892928->897023) req@ffff9cf3481fca80 x1866777609771648/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:352/0 lens 488/0 e 0 to 0 dl 1780298657 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298646.213191:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fca80 with x1866777609771648 ext(892928->897023) 00002000:00010000:2.0:1780298646.213196:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf3481fca80 00000100:00100000:2.0:1780298646.213216:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6580 00000100:00100000:2.0:1780298646.213223:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2397:x1866777609771648:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298646.213334:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2397:x1866777609771648:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 110us (262us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298646.213343:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6580 00002000:00100000:2.0:1780298646.213350:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x12:0x0] (892928->897023) req@ffff9cf3481fca80 x1866777609771648/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:352/0 lens 488/440 e 0 to 0 dl 1780298657 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298646.213359:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fca80 with x1866777609771648 ext(892928->897023) 00002000:00010000:2.0:1780298646.213363:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf3481fca80 00000100:00100000:3.0:1780298646.214338:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.214375:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609771776 00000100:00100000:1.0:1780298646.214381:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609771776 00000100:00100000:1.0:1780298646.214411:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10724 00000100:00100000:1.0:1780298646.214417:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609771776:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.214425:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.214433:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63036a00 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d57c7 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.214463:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605146240:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.214516:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e900 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.215189:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605146240:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298646.215208:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63036a00 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d57c7 expref: 17 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.215223:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63036a00 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d57c7 expref: 17 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.215234:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278a00, rc 0) 00000100:00100000:1.0:1780298646.215275:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609771776:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 848us (929us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.215283:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10724 00000100:00100000:3.0:1780298646.216048:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.216090:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609771904 00000100:00100000:0.0:1780298646.216096:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609771904 00000100:00100000:0.0:1780298646.216159:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10725 00000100:00100000:0.0:1780298646.216170:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2398:x1866777609771904:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.216182:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.216195:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001d4afe0b/0xaf35132e63036a07 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d57ce expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.216237:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605146368:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.216301:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4f00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.216910:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605146368:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.216928:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001d4afe0b/0xaf35132e63036a07 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d57ce expref: 17 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.216940:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001d4afe0b/0xaf35132e63036a07 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d57ce expref: 17 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.216948:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3000, rc 0) 00000100:00100000:0.0:1780298646.216971:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2398:x1866777609771904:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 805us (924us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.216976:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10725 00000100:00100000:3.0:1780298646.217811:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.217847:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609772032 00000100:00100000:2.0:1780298646.217851:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609772032 00000100:00100000:2.0:1780298646.217876:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10726 00000100:00100000:2.0:1780298646.217881:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609772032:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.217887:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298646.217892:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63036a0e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d57d5 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.217918:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605146496:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.217947:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6cf00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.218615:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605146496:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298646.218626:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63036a0e lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d57d5 expref: 17 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.218635:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63036a0e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d57d5 expref: 17 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.218641:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9aa00, rc 0) 00000100:00100000:2.0:1780298646.218663:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609772032:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 782us (854us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298646.218668:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10726 00000100:00100000:3.0:1780298646.219484:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.219532:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609772160 00000100:00100000:1.0:1780298646.219540:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609772160 00000100:00100000:1.0:1780298646.219583:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10727 00000100:00100000:1.0:1780298646.219594:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481ff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2398:x1866777609772160:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.219606:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.219619:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63036a15 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d57dc expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.219669:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605146624:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.219730:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e900 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.220344:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605146624:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298646.220365:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63036a15 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d57dc expref: 17 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.220382:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000078761089/0xaf35132e63036a15 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d57dc expref: 17 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.220393:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279200, rc 0) 00000100:00100000:1.0:1780298646.220427:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481ff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2398:x1866777609772160:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 836us (942us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.220436:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10727 00000100:00100000:3.0:1780298646.221190:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.221223:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609772288 00000100:00100000:1.0:1780298646.221227:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609772288 00002000:00100000:1.0:1780298646.221250:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (4124672->4128767) req@ffff9cf3481ff480 x1866777609772288/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:352/0 lens 488/0 e 0 to 0 dl 1780298657 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298646.221258:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481ff480 with x1866777609772288 ext(4124672->4128767) 00002000:00010000:1.0:1780298646.221262:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf3481ff480 00000100:00100000:1.0:1780298646.221292:0:19950:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6581 00000100:00100000:1.0:1780298646.221299:0:19950:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481ff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2398:x1866777609772288:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298646.221420:0:19950:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481ff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2398:x1866777609772288:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 123us (233us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298646.221425:0:19950:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6581 00002000:00100000:1.0:1780298646.221430:0:19950:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x280000401:0x12:0x0] (4124672->4128767) req@ffff9cf3481ff480 x1866777609772288/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:352/0 lens 488/440 e 0 to 0 dl 1780298657 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298646.221436:0:19950:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481ff480 with x1866777609772288 ext(4124672->4128767) 00002000:00010000:1.0:1780298646.221439:0:19950:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf3481ff480 00000100:00100000:3.0:1780298646.222204:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.222236:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609772416 00000100:00100000:1.0:1780298646.222239:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609772416 00000100:00100000:1.0:1780298646.222262:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10728 00000100:00100000:1.0:1780298646.222278:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f755180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609772416:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.222285:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.222291:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e63036a1c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d57e3 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.222311:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605146752:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.222345:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e900 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.222955:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605146752:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298646.222968:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e63036a1c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d57e3 expref: 17 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.222978:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e63036a1c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d57e3 expref: 17 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.222985:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279800, rc 0) 00000100:00100000:1.0:1780298646.223011:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f755180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:2397:x1866777609772416:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 745us (809us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.223017:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10728 00000100:00100000:3.0:1780298646.223774:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.223805:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609772544 00000100:00100000:0.0:1780298646.223811:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609772544 00002000:00100000:0.0:1780298646.223841:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (4128768->4132863) req@ffff9cf37f755f80 x1866777609772544/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:352/0 lens 488/0 e 0 to 0 dl 1780298657 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298646.223853:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f755f80 with x1866777609772544 ext(4128768->4132863) 00002000:00010000:0.0:1780298646.223856:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f755f80 00000100:00100000:0.0:1780298646.223874:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6582 00000100:00100000:0.0:1780298646.223880:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f755f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2397:x1866777609772544:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298646.223993:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f755f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2397:x1866777609772544:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 113us (220us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298646.223998:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6582 00002000:00100000:0.0:1780298646.224004:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (4128768->4132863) req@ffff9cf37f755f80 x1866777609772544/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:352/0 lens 488/440 e 0 to 0 dl 1780298657 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298646.224010:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f755f80 with x1866777609772544 ext(4128768->4132863) 00002000:00010000:0.0:1780298646.224013:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f755f80 00000100:00100000:3.0:1780298646.224835:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.224866:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609772672 00000100:00100000:0.0:1780298646.224870:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609772672 00000100:00100000:0.0:1780298646.224897:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10729 00000100:00100000:0.0:1780298646.224904:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f756680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2397:x1866777609772672:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.224912:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.224922:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e63036a23 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d57ea expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.224932:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.224955:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609772800 00000100:00100000:0.0:1780298646.224959:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605146880:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.224960:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609772800 00000100:00100000:2.0:1780298646.224992:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10730 00000100:00100000:2.0:1780298646.225005:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f756300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609772800:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298646.225012:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4f00 going to sleep for 11 seconds 00010000:00010000:2.0:1780298646.225013:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298646.225025:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000019780e8d/0xaf35132e63036a2a lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d57f1 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.225059:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605147008:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.225086:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6cf00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.225679:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605146880:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.225701:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e63036a23 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d57ea expref: 19 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.225719:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e63036a23 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d57ea expref: 19 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.225731:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3c00, rc 0) 00000100:00100000:0.0:1780298646.225767:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f756680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2397:x1866777609772672:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 862us (930us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298646.225778:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605147008:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.225779:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10729 00010000:00010000:2.0:1780298646.225799:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000019780e8d/0xaf35132e63036a2a lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d57f1 expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.225810:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000019780e8d/0xaf35132e63036a2a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d57f1 expref: 10 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.225817:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9a200, rc 0) 00000100:00100000:2.0:1780298646.225835:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f756300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609772800:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 836us (906us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298646.225850:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10730 00000100:00100000:3.0:1780298646.226621:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.226640:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.226659:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609772928 00000100:00100000:1.0:1780298646.226665:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609772928 00000100:00100000:2.0:1780298646.226691:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609773056 00000100:00100000:2.0:1780298646.226699:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609773056 00000100:00100000:1.0:1780298646.226704:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10731 00000100:00100000:1.0:1780298646.226713:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f755c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2398:x1866777609772928:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.226722:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.226730:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63036a31 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d57f8 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.226744:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10732 00000100:00100000:2.0:1780298646.226756:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f757b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609773056:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298646.226762:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605147136:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298646.226772:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298646.226784:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c1d9addf/0xaf35132e63036a38 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d57ff expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.226809:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e900 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.226826:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605147264:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.226865:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6cf00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.227584:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605147264:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.227602:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605147136:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298646.227609:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c1d9addf/0xaf35132e63036a38 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d57ff expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.227624:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63036a31 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d57f8 expref: 19 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.227629:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c1d9addf/0xaf35132e63036a38 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d57ff expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.227663:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9a600, rc 0) 00010000:00010000:1.0:1780298646.227663:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63036a31 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d57f8 expref: 19 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.227677:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279a00, rc 0) 00000100:00100000:2.0:1780298646.227697:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f757b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609773056:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 945us (1055us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298646.227710:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10732 00000100:00100000:1.0:1780298646.227711:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f755c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2398:x1866777609772928:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 992us (1082us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.227724:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10731 00000100:00100000:3.0:1780298646.228503:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.228531:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609773184 00000100:00100000:2.0:1780298646.228535:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609773184 00000100:00100000:2.0:1780298646.228558:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10733 00000100:00100000:2.0:1780298646.228563:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f757100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+16:29360:x1866777609773184:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.228571:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298646.228579:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009fd8d343/0xaf35132e63036a3f lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5806 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.228591:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(87 169) 00010000:00010000:2.0:1780298646.228595:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(87 169) 00010000:00010000:2.0:1780298646.228602:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009fd8d343/0xaf35132e63036a3f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2568192->4939775] (req 2568192->2580479) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5806 expref: 17 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.228612:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009fd8d343/0xaf35132e63036a3f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [2568192->4939775] (req 2568192->2580479) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5806 expref: 17 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.228619:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b000, rc 0) 00000100:00100000:2.0:1780298646.228642:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f757100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:29360:x1866777609773184:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 79us (140us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298646.228647:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10733 00000100:00100000:3.0:1780298646.229308:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.229340:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609773312 00000100:00100000:1.0:1780298646.229345:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609773312 00000100:00100000:1.0:1780298646.229378:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6583 00000100:00100000:1.0:1780298646.229383:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f754000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+17:2400:x1866777609773312:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298646.229563:0:19943:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4944749 -> 4944749 00002000:00010000:1.0:1780298646.229567:0:19943:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:1.0:1780298646.229569:0:19943:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298646 -> 1780298646 00002000:00010000:1.0:1780298646.229572:0:19943:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:1.0:1780298646.229574:0:19943:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 520 -> 520 00000100:00100000:1.0:1780298646.229610:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f754000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609773312:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 226us (302us total) trans 4294969821 rc 0/0 00000100:00100000:1.0:1780298646.229621:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6583 00000100:00100000:3.0:1780298646.230348:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.230377:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609773440 00000100:00100000:2.0:1780298646.230382:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609773440 00000100:00100000:2.0:1780298646.230404:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10734 00000100:00100000:2.0:1780298646.230409:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f754a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2399:x1866777609773440:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.230416:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298646.230423:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63036a46 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d580d expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.230445:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605147392:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.230479:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6cf00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.231060:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605147392:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298646.231072:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63036a46 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d580d expref: 19 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.231082:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63036a46 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d580d expref: 19 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.231089:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9ac00, rc 0) 00000100:00100000:2.0:1780298646.231111:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f754a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2399:x1866777609773440:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 703us (766us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298646.231116:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10734 00000100:00100000:3.0:1780298646.231863:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.231891:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609773568 00000100:00100000:0.0:1780298646.231895:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609773568 00000100:00100000:0.0:1780298646.231917:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10735 00000100:00100000:0.0:1780298646.231923:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a80a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2399:x1866777609773568:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.231928:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.231935:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001d4afe0b/0xaf35132e63036a4d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5814 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.231955:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605147520:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.231982:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4f00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.232533:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605147520:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.232548:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001d4afe0b/0xaf35132e63036a4d lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5814 expref: 19 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.232557:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001d4afe0b/0xaf35132e63036a4d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5814 expref: 19 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.232568:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3000, rc 0) 00000100:00100000:0.0:1780298646.232589:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a80a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2399:x1866777609773568:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 667us (728us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.232594:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10735 00000100:00100000:3.0:1780298646.233346:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.233363:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.233379:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609773696 00000100:00100000:1.0:1780298646.233385:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609773696 00000100:00100000:0.0:1780298646.233404:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609773824 00000100:00100000:0.0:1780298646.233409:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609773824 00000100:00100000:1.0:1780298646.233422:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10736 00000100:00100000:1.0:1780298646.233431:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a81f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2398:x1866777609773696:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.233439:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.233450:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63036a54 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d581b expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.233462:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10737 00000100:00100000:2.0:1780298646.233473:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a81880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609773824:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.233481:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298646.233488:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605147648:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298646.233493:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000019780e8d/0xaf35132e63036a5b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5822 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.233521:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605147776:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.233542:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e900 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.233556:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6cf00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.234215:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605147776:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298646.234235:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000019780e8d/0xaf35132e63036a5b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5822 expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.234247:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000019780e8d/0xaf35132e63036a5b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5822 expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.234253:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605147648:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298646.234258:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9a200, rc 0) 00010000:00010000:1.0:1780298646.234288:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63036a54 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d581b expref: 19 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.234300:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a81880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609773824:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 826us (930us total) trans 0 rc 301/301 00010000:00010000:1.0:1780298646.234309:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63036a54 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d581b expref: 19 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.234310:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10737 00010000:00010000:1.0:1780298646.234328:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278a00, rc 0) 00000100:00100000:1.0:1780298646.234354:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a81f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:2398:x1866777609773696:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 923us (1006us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.234361:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10736 00000100:00100000:3.0:1780298646.235092:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.235118:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609773952 00000100:00100000:2.0:1780298646.235152:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609773952 00000100:00100000:2.0:1780298646.235197:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10738 00000100:00100000:2.0:1780298646.235204:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a82680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+18:29360:x1866777609773952:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.235210:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298646.235217:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63036a62 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5829 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.235231:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(87 169) 00010000:00010000:2.0:1780298646.235234:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(87 169) 00010000:00010000:2.0:1780298646.235240:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63036a62 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [487424->2568191] (req 487424->491519) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5829 expref: 19 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.235249:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63036a62 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [487424->2568191] (req 487424->491519) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5829 expref: 19 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.235254:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9aa00, rc 0) 00000100:00100000:2.0:1780298646.235288:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a82680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+19:29360:x1866777609773952:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 85us (197us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298646.235293:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10738 00000100:00100000:3.0:1780298646.235842:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.235873:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609774080 00000100:00100000:0.0:1780298646.235878:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609774080 00000100:00100000:0.0:1780298646.235908:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6584 00000100:00100000:0.0:1780298646.235913:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a83b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+19:2399:x1866777609774080:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298646.236055:0:16972:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4944749 -> 4944749 00002000:00010000:0.0:1780298646.236059:0:16972:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298646.236062:0:16972:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298646.236064:0:16972:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298646.236066:0:16972:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 520 -> 528 00000100:00100000:0.0:1780298646.236097:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a83b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2399:x1866777609774080:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 183us (253us total) trans 4294969822 rc 0/0 00000100:00100000:0.0:1780298646.236105:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6584 00000100:00100000:3.0:1780298646.236870:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.236893:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609774208 00000100:00100000:0.0:1780298646.236897:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609774208 00000100:00100000:0.0:1780298646.236919:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10739 00000100:00100000:0.0:1780298646.236925:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a82a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2400:x1866777609774208:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.236931:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.236940:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e63036a69 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5830 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.236963:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.236976:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605147904:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.236994:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609774336 00000100:00100000:1.0:1780298646.236999:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609774336 00000100:00100000:1.0:1780298646.237039:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10740 00000100:00100000:1.0:1780298646.237050:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a82d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609774336:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298646.237050:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4f00 going to sleep for 11 seconds 00010000:00010000:1.0:1780298646.237065:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.237078:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002ff2dd4d/0xaf35132e63036a70 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5837 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.237106:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605148032:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.237196:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e900 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.237604:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605147904:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.237623:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e63036a69 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5830 expref: 21 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.237639:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000101214ae/0xaf35132e63036a69 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5830 expref: 21 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.237650:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3200, rc 0) 00000100:00100000:0.0:1780298646.237677:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a82a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2400:x1866777609774208:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 752us (807us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.237685:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10739 00000100:00100000:0.0:1780298646.237795:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605148032:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.237840:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002ff2dd4d/0xaf35132e63036a70 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5837 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.237855:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002ff2dd4d/0xaf35132e63036a70 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5837 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.237865:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278800, rc 0) 00000100:00100000:0.0:1780298646.237894:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a82d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609774336:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 848us (929us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.237902:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10740 00000100:00100000:3.0:1780298646.238666:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.238710:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609774464 00000100:00100000:1.0:1780298646.238717:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609774464 00000100:00100000:3.0:1780298646.238746:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.238759:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609774592 00000100:00100000:1.0:1780298646.238762:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609774592 00000100:00100000:1.0:1780298646.238789:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10741 00000100:00100000:1.0:1780298646.238799:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e424c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2397:x1866777609774464:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298646.238805:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10742 00010000:00010000:1.0:1780298646.238809:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.238820:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e63036a77 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d583e expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.238821:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b41880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609774592:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.238833:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298646.238843:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c1d9addf/0xaf35132e63036a7e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5845 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.238853:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605148160:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.238872:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605148288:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.238914:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6cf00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.238915:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.239550:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f3480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605148160:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.239565:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605148288:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298646.239568:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e63036a77 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d583e expref: 21 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.239580:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000017dab243/0xaf35132e63036a77 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d583e expref: 21 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.239583:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c1d9addf/0xaf35132e63036a7e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5845 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.239598:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279800, rc 0) 00010000:00010000:2.0:1780298646.239609:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c1d9addf/0xaf35132e63036a7e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5845 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.239617:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9a600, rc 0) 00000100:00100000:1.0:1780298646.239629:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e424c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2397:x1866777609774464:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 831us (962us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.239634:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10741 00000100:00100000:2.0:1780298646.239640:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b41880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2397:x1866777609774592:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 825us (891us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298646.239649:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10742 00000100:00100000:3.0:1780298646.240661:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.240691:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609774720 00000100:00100000:2.0:1780298646.240695:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609774720 00000100:00100000:2.0:1780298646.240720:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10743 00000100:00100000:2.0:1780298646.240725:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2399:x1866777609774720:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.240730:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298646.240737:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63036a85 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d584c expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.240757:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605148416:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.240788:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6cf00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.241372:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605148416:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298646.241382:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63036a85 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d584c expref: 19 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.241391:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63036a85 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d584c expref: 19 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.241400:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9be00, rc 0) 00000100:00100000:2.0:1780298646.241423:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2399:x1866777609774720:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 699us (764us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298646.241427:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10743 00000100:00100000:3.0:1780298646.242084:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.242122:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609774848 00000100:00100000:0.0:1780298646.242169:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609774848 00000100:00100000:0.0:1780298646.242211:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10744 00000100:00100000:0.0:1780298646.242225:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b41c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609774848:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.242236:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.242248:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e63036a8c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5853 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.242302:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605148544:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.242359:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4f00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.242964:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605148544:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.242983:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e63036a8c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5853 expref: 19 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.242999:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e63036a8c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5853 expref: 19 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.243010:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3c00, rc 0) 00000100:00100000:0.0:1780298646.243041:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b41c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609774848:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 822us (958us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.243049:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10744 00000100:00100000:3.0:1780298646.243812:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.243847:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609774976 00000100:00100000:1.0:1780298646.243852:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609774976 00000100:00100000:1.0:1780298646.243877:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10745 00000100:00100000:1.0:1780298646.243882:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b43800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:29360:x1866777609774976:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.243892:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.243900:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63036a93 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d585a expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.243919:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000008361787d/0xaf35132e63036958 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d573b expref: 13 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.243927:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298646 v=(66 122) 00010000:00010000:1.0:1780298646.243931:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298646 v=(67 122) 00010000:00010000:1.0:1780298646.243935:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63036a93 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3616768->3620863] (req 3616768->3620863) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d585a expref: 10 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.243952:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000008361787d/0xaf35132e63036958 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d573b expref: 13 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.243962:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000008361787d/0xaf35132e63036958 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d573b expref: 13 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.243971:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605148672:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298646.244004:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.244628:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605148672:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298646.244645:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63036a93 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3616768->3620863] (req 3616768->3620863) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d585a expref: 10 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.244657:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278a00, rc 0) 00000100:00100000:1.0:1780298646.244689:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:29360:x1866777609774976:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 807us (878us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298646.244694:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10745 00000100:00100000:1.0:1780298646.244849:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.244881:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609775104 00000100:00100000:2.0:1780298646.244885:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609775104 00010000:00010000:2.0:1780298646.244899:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000008361787d/0xaf35132e63036958 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d573b expref: 14 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.244920:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15358 00000100:00100000:2.0:1780298646.244926:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f3800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609775104:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298646.244933:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298646.244940:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000008361787d/0xaf35132e63036958 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d573b expref: 14 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.244947:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000008361787d/0xaf35132e63036958 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d573b expref: 14 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.244959:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008361787d/0xaf35132e63036958 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d573b expref: 14 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.244967:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298646 v=(67 122) 00010000:00010000:2.0:1780298646.244969:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298646 v=(67 122) 00010000:00010000:2.0:1780298646.244974:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63036a93 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3616768->37171199] (req 3616768->3620863) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d585a expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.244980:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63036a93 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3616768->37171199] (req 3616768->3620863) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d585a expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.244995:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63036a93 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3616768->37171199] (req 3616768->3620863) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d585a expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.245001:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777605148800:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298646.245028:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6cf00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.245773:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777605148800:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298646.245786:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1780298646.245796:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.245810:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f3800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+13:2399:x1866777609775104:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 884us (962us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298646.245815:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15358 00000100:00100000:0.0:1780298646.245854:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609775232 00000100:00100000:0.0:1780298646.245861:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609775232 00000100:00100000:0.0:1780298646.245917:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6585 00000100:00100000:0.0:1780298646.245925:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b41180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2400:x1866777609775232:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298646.246101:0:8363:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 0 -> 3619518 00002000:00010000:0.0:1780298646.246105:0:8363:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298646.246108:0:8363:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298646.246110:0:8363:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298646.246111:0:8363:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 0 -> 8 00000100:00100000:0.0:1780298646.246169:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b41180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609775232:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 246us (377us total) trans 4294969341 rc 0/0 00000100:00100000:0.0:1780298646.246175:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6585 00000100:00100000:1.0:1780298646.247019:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.247045:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.247073:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609775360 00000100:00100000:3.0:1780298646.247079:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609775360 00000100:00100000:0.0:1780298646.247085:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609775488 00000100:00100000:0.0:1780298646.247090:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609775488 00000100:00100000:3.0:1780298646.247119:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10746 00000100:00100000:3.0:1780298646.247157:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2397:x1866777609775360:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298646.247166:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10747 00010000:00010000:3.0:1780298646.247169:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298646.247184:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609775488:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298646.247185:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e63036a9a lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5861 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.247194:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.247205:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000101214ae/0xaf35132e63036aa1 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5868 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.247242:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605148928:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.247251:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605149056:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298646.247318:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856180 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.247318:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4f00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.247996:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b43b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605148928:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298646.248021:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e63036a9a lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5861 expref: 21 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.248034:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605149056:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298646.248041:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000eb0d13ab/0xaf35132e63036a9a lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5861 expref: 21 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.248050:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92400, rc 0) 00010000:00010000:0.0:1780298646.248065:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000101214ae/0xaf35132e63036aa1 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5868 expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.248082:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2397:x1866777609775360:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 955us (1063us total) trans 0 rc 301/301 00010000:00010000:0.0:1780298646.248089:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000101214ae/0xaf35132e63036aa1 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5868 expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.248099:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10746 00010000:00010000:0.0:1780298646.248118:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3200, rc 0) 00000100:00100000:0.0:1780298646.248175:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609775488:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1000us (1128us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.248186:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10747 00000100:00100000:2.0:1780298646.249030:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.249071:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609775616 00000100:00100000:2.0:1780298646.249078:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.249078:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609775616 00000100:00100000:0.0:1780298646.249117:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609775744 00000100:00100000:0.0:1780298646.249120:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609775744 00000100:00100000:3.0:1780298646.249192:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10748 00000100:00100000:1.0:1780298646.249203:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10749 00000100:00100000:3.0:1780298646.249211:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243061500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2400:x1866777609775616:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298646.249223:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243060000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609775744:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298646.249224:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298646.249235:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e63036aa8 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d586f expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.249236:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.249248:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63036aaf lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5876 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.249291:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf242b42300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605149184:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.249303:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605149312:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298646.249356:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856180 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.249356:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.249992:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf242b42300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605149184:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298646.250013:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e63036aa8 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d586f expref: 21 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.250026:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b87f9a7e/0xaf35132e63036aa8 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d586f expref: 21 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.250033:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93600, rc 0) 00000100:00100000:1.0:1780298646.250061:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605149312:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.250066:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243061500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2400:x1866777609775616:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 866us (1037us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298646.250073:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10748 00010000:00010000:1.0:1780298646.250077:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63036aaf lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5876 expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.250095:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63036aaf lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5876 expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.250106:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279200, rc 0) 00000100:00100000:1.0:1780298646.250163:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243060000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609775744:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 948us (1082us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.250173:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10749 00000100:00100000:0.0:1780298646.251065:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.251099:0:19950:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609775872 00000100:00100000:1.0:1780298646.251103:0:19950:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609775872 00000100:00100000:0.0:1780298646.251164:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6586 00000100:00100000:0.0:1780298646.251181:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609775872:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298646.251375:0:8364:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4944749 -> 4944749 00002000:00010000:0.0:1780298646.251377:0:8364:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298646.251379:0:8364:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298646.251381:0:8364:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298646.251382:0:8364:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 528 -> 536 00000100:00100000:0.0:1780298646.251407:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2400:x1866777609775872:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 237us (342us total) trans 4294969823 rc 0/0 00000100:00100000:0.0:1780298646.251411:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6586 00000100:00100000:3.0:1780298646.252195:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.252224:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609776000 00000100:00100000:1.0:1780298646.252228:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609776000 00000100:00100000:3.0:1780298646.252233:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.252256:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609776128 00000100:00100000:1.0:1780298646.252259:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609776128 00000100:00100000:2.0:1780298646.252282:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10750 00000100:00100000:0.0:1780298646.252294:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10751 00000100:00100000:2.0:1780298646.252295:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2400:x1866777609776000:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.252304:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298646.252308:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609776128:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.252313:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63036ab6 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d587d expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.252317:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.252328:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e2cfab1/0xaf35132e63036abd lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5884 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.252342:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605149440:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.252360:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605149568:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.252392:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6cf00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.252392:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4f00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.252996:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605149440:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.253007:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605149568:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298646.253013:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63036ab6 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d587d expref: 21 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.253025:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e7fd0135/0xaf35132e63036ab6 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d587d expref: 21 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.253031:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003e2cfab1/0xaf35132e63036abd lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5884 expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.253034:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9be00, rc 0) 00010000:00010000:0.0:1780298646.253050:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e2cfab1/0xaf35132e63036abd lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5884 expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.253058:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2e00, rc 0) 00000100:00100000:2.0:1780298646.253065:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2400:x1866777609776000:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 775us (872us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298646.253071:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10750 00000100:00100000:0.0:1780298646.253082:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609776128:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 778us (848us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.253091:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10751 00000100:00100000:3.0:1780298646.253952:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.253981:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609776256 00000100:00100000:2.0:1780298646.253985:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609776256 00000100:00100000:3.0:1780298646.253992:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.254019:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609776384 00000100:00100000:2.0:1780298646.254023:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609776384 00000100:00100000:0.0:1780298646.254041:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10752 00000100:00100000:2.0:1780298646.254054:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10753 00000100:00100000:0.0:1780298646.254055:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2397:x1866777609776256:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:2.0:1780298646.254064:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609776384:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.254072:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298646.254078:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c1d9addf/0xaf35132e63036ac4 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5892 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.254085:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.254094:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e63036acb lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d588b expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.254100:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605149696:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.254145:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605149824:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.254160:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6cf00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.254182:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.254903:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605149824:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.254916:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605149696:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298646.254920:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e63036acb lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d588b expref: 21 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.254933:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e63036acb lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d588b expref: 21 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.254940:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278400, rc 0) 00010000:00010000:2.0:1780298646.254941:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c1d9addf/0xaf35132e63036ac4 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5892 expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.254956:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c1d9addf/0xaf35132e63036ac4 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5892 expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.254967:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9a600, rc 0) 00000100:00100000:1.0:1780298646.254968:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2397:x1866777609776256:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 918us (1017us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.254976:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10752 00000100:00100000:2.0:1780298646.254995:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609776384:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 933us (1008us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298646.255000:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10753 00000100:00100000:3.0:1780298646.255974:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.255994:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.256015:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609776512 00000100:00100000:0.0:1780298646.256019:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609776512 00000100:00100000:0.0:1780298646.256047:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609776640 00000100:00100000:0.0:1780298646.256049:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609776640 00000100:00100000:2.0:1780298646.256055:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10754 00000100:00100000:2.0:1780298646.256066:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2400:x1866777609776512:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298646.256072:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10755 00010000:00010000:2.0:1780298646.256074:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298646.256080:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609776640:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.256085:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63036ad2 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5899 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.256087:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.256097:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c0d26bbc/0xaf35132e63036ad9 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d58a0 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.256153:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605149952:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.256171:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605150080:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.256203:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4f00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.256215:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6cf00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.256841:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605149952:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.256855:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c0d26bbc/0xaf35132e63036ad9 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58a0 expref: 12 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.256867:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605150080:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.256867:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c0d26bbc/0xaf35132e63036ad9 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58a0 expref: 12 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.256882:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3c00, rc 0) 00010000:00010000:2.0:1780298646.256899:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63036ad2 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5899 expref: 21 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.256911:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2399:x1866777609776640:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 833us (916us total) trans 0 rc 301/301 00010000:00010000:2.0:1780298646.256917:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63036ad2 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5899 expref: 21 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.256919:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10755 00010000:00010000:2.0:1780298646.256936:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9ac00, rc 0) 00000100:00100000:2.0:1780298646.256965:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2400:x1866777609776512:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 900us (991us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298646.256974:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10754 00000100:00100000:3.0:1780298646.257830:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.257866:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609776768 00000100:00100000:1.0:1780298646.257870:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609776768 00000100:00100000:1.0:1780298646.257898:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10756 00000100:00100000:1.0:1780298646.257906:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:29360:x1866777609776768:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.257914:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.257923:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63036ae0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d58a7 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.257939:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298646 v=(67 122) 00010000:00010000:1.0:1780298646.257943:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298646 v=(67 122) 00010000:00010000:1.0:1780298646.257951:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63036ae0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [491520->3616767] (req 491520->495615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58a7 expref: 12 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.257964:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63036ae0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [491520->3616767] (req 491520->495615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58a7 expref: 12 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.257970:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279200, rc 0) 00000100:00100000:1.0:1780298646.257995:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:29360:x1866777609776768:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 92us (169us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298646.258000:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10756 00000100:00100000:3.0:1780298646.258772:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.258806:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609776896 00000100:00100000:0.0:1780298646.258811:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609776896 00002000:00100000:0.0:1780298646.258831:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (491520->495615) req@ffff9cf2548bbb80 x1866777609776896/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:352/0 lens 488/0 e 0 to 0 dl 1780298657 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298646.258838:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2548bbb80 with x1866777609776896 ext(491520->495615) 00002000:00010000:0.0:1780298646.258841:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2548bbb80 00000100:00100000:0.0:1780298646.258859:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6587 00000100:00100000:0.0:1780298646.258864:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609776896:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298646.258991:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609776896:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 127us (221us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298646.258996:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6587 00002000:00100000:0.0:1780298646.259000:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (491520->495615) req@ffff9cf2548bbb80 x1866777609776896/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:352/0 lens 488/4536 e 0 to 0 dl 1780298657 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298646.259007:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2548bbb80 with x1866777609776896 ext(491520->495615) 00002000:00010000:0.0:1780298646.259010:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2548bbb80 00000100:00100000:3.0:1780298646.260065:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.260096:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609777024 00000100:00100000:1.0:1780298646.260100:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609777024 00002000:00100000:1.0:1780298646.260120:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x12:0x0] (495616->499711) req@ffff9cf2548b9f80 x1866777609777024/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:352/0 lens 488/0 e 0 to 0 dl 1780298657 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298646.260151:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2548b9f80 with x1866777609777024 ext(495616->499711) 00002000:00010000:1.0:1780298646.260155:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2548b9f80 00000100:00100000:1.0:1780298646.260181:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6588 00000100:00100000:1.0:1780298646.260190:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609777024:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:1.0:1780298646.260288:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2400:x1866777609777024:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 98us (223us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1780298646.260295:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6588 00002000:00100000:1.0:1780298646.260303:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x280000401:0x12:0x0] (495616->499711) req@ffff9cf2548b9f80 x1866777609777024/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:352/0 lens 488/4536 e 0 to 0 dl 1780298657 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1780298646.260312:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2548b9f80 with x1866777609777024 ext(495616->499711) 00002000:00010000:1.0:1780298646.260315:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2548b9f80 00000100:00100000:3.0:1780298646.261465:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.261500:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609777152 00000100:00100000:0.0:1780298646.261505:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609777152 00002000:00100000:0.0:1780298646.261522:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x12:0x0] (499712->503807) req@ffff9cf2548b9180 x1866777609777152/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:352/0 lens 488/0 e 0 to 0 dl 1780298657 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298646.261529:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2548b9180 with x1866777609777152 ext(499712->503807) 00002000:00010000:0.0:1780298646.261532:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2548b9180 00000100:00100000:0.0:1780298646.261553:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6589 00000100:00100000:0.0:1780298646.261558:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2548b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609777152:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298646.261651:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2548b9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2399:x1866777609777152:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 92us (188us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298646.261659:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6589 00002000:00100000:0.0:1780298646.261666:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x12:0x0] (499712->503807) req@ffff9cf2548b9180 x1866777609777152/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:352/0 lens 488/4536 e 0 to 0 dl 1780298657 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298646.261674:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2548b9180 with x1866777609777152 ext(499712->503807) 00002000:00010000:0.0:1780298646.261677:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2548b9180 00000100:00100000:3.0:1780298646.262795:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.262829:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609777280 00000100:00100000:0.0:1780298646.262833:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609777280 00002000:00100000:0.0:1780298646.262849:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x12:0x0] (491520->503807) req@ffff9cf37f8b0380 x1866777609777280/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:352/0 lens 12776/0 e 0 to 0 dl 1780298657 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298646.262859:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f8b0380 with x1866777609777280 ext(491520->503807) 00002000:00010000:0.0:1780298646.262861:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f8b0380 00000100:00100000:0.0:1780298646.262876:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6590 00000100:00100000:0.0:1780298646.262880:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+12:2397:x1866777609777280:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:0.0:1780298646.263303:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2397:x1866777609777280:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 422us (511us total) trans 4294969342 rc 0/0 00000100:00100000:0.0:1780298646.263308:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6590 00002000:00100000:0.0:1780298646.263312:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x12:0x0] (491520->503807) req@ffff9cf37f8b0380 x1866777609777280/t4294969342(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:352/0 lens 12776/448 e 0 to 0 dl 1780298657 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298646.263318:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f8b0380 with x1866777609777280 ext(491520->503807) 00002000:00010000:0.0:1780298646.263321:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f8b0380 00000100:00100000:3.0:1780298646.263979:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.264022:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609777408 00000100:00100000:2.0:1780298646.264029:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609777408 00000100:00100000:2.0:1780298646.264065:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10757 00000100:00100000:2.0:1780298646.264075:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+14:2398:x1866777609777408:12345-192.168.203.25@tcp:16:fsx.0 00000001:00080000:1.0:1780298646.264900:0:8351:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0000: transno 4294969342 is committed 00000001:00080000:1.0:1780298646.264909:0:8351:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0000: transno 4294969340 is committed 00000100:00100000:2.0:1780298646.264990:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609777408:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 916us (1009us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298646.264999:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10757 00000100:00100000:3.0:1780298646.265896:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.265929:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.265932:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609777536 00000100:00100000:1.0:1780298646.265937:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609777536 00000100:00100000:0.0:1780298646.265968:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609777664 00000100:00100000:1.0:1780298646.265973:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10758 00000100:00100000:0.0:1780298646.265973:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609777664 00000100:00100000:1.0:1780298646.265985:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2400:x1866777609777536:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.265994:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298646.266002:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10759 00010000:00010000:1.0:1780298646.266006:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e63036ae7 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d58ae expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.266011:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609777664:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.266023:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.266031:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e2cfab1/0xaf35132e63036aee lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d58b5 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.266046:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605150208:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.266057:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605150336:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.266103:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4f00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.266104:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e540 going to sleep for 11 seconds 00002000:00010000:0.0:1780298646.266940:0:20252:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks: 8 -> 32 00000100:00100000:0.0:1780298646.266948:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605150336:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.266951:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605150208:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.266970:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003e2cfab1/0xaf35132e63036aee lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58b5 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.266976:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e63036ae7 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58ae expref: 20 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.266985:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e2cfab1/0xaf35132e63036aee lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58b5 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.267006:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2e00, rc 0) 00010000:00010000:1.0:1780298646.267015:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003e5b3111/0xaf35132e63036ae7 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58ae expref: 20 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.267027:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278400, rc 0) 00000100:00100000:0.0:1780298646.267033:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609777664:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1025us (1104us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.267038:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10759 00000100:00100000:1.0:1780298646.267051:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2400:x1866777609777536:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1069us (1162us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.267057:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10758 00000100:00100000:3.0:1780298646.268038:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.268081:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609777792 00000100:00100000:1.0:1780298646.268087:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609777792 00000100:00100000:3.0:1780298646.268093:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.268122:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609777920 00000100:00100000:1.0:1780298646.268147:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609777920 00000100:00100000:0.0:1780298646.268164:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10760 00000100:00100000:0.0:1780298646.268174:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2397:x1866777609777792:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.268182:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298646.268184:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10761 00010000:00010000:0.0:1780298646.268190:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e63036af5 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d58bc expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.268198:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609777920:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.268212:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298646.268217:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605150464:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298646.268223:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e63036afc lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d58c3 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.268254:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243ba4f00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.268262:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605150592:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.268308:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6cf00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.268917:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605150464:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.268938:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e63036af5 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58bc expref: 20 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.268961:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e63036af5 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58bc expref: 20 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.268978:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3c00, rc 0) 00000100:00100000:0.0:1780298646.269014:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2397:x1866777609777792:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 843us (979us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.269020:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10760 00000100:00100000:2.0:1780298646.269062:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605150592:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298646.269080:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e63036afc lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58c3 expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.269097:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e63036afc lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58c3 expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.269116:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9ac00, rc 0) 00000100:00100000:2.0:1780298646.269183:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609777920:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 989us (1093us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298646.269190:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10761 00000100:00100000:3.0:1780298646.269919:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.269969:0:17567:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609778176 00000100:00100000:0.0:1780298646.269976:0:17567:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609778176 00000100:00100000:0.0:1780298646.270005:0:17567:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62102 00000100:00100000:0.0:1780298646.270010:0:17567:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+53:29360:x1866777609778176:12345-192.168.203.25@tcp:36:fsx.0 00010000:00010000:0.0:1780298646.270047:0:17567:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298646.270061:0:17567:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9bc00/0xaf35132e63036989 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 27 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5750 expref: 53 pid: 6477 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.270075:0:17567:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298646.270078:0:17567:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1780298646.270245:0:17567:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3200/0xaf35132e63036b03 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 26 type: IBT flags: 0x40000000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.270257:0:17567:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93e00/0xaf35132e6303697b lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 26 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5749 expref: 50 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.270278:0:17567:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3200/0xaf35132e63036b03 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 26 type: IBT flags: 0x50210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.270302:0:17567:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93e00/0xaf35132e6303697b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 26 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5749 expref: 50 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.270321:0:17567:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93e00/0xaf35132e6303697b lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 26 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5749 expref: 50 pid: 17578 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.270333:0:17567:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f2d80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:17567:1866777605150720:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298646.270366:0:17567:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.271035:0:17567:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f2d80 pname:cluuid:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000_UUID:17567:1866777605150720:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298646.271049:0:17567:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3200/0xaf35132e63036b03 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 26 type: IBT flags: 0x40210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.271217:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.271258:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609778304 00000100:00100000:1.0:1780298646.271275:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609778304 00010000:00010000:1.0:1780298646.271302:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93e00/0xaf35132e6303697b lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 26 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5749 expref: 51 pid: 17578 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.271332:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15359 00000100:00100000:1.0:1780298646.271342:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+51:2400:x1866777609778304:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298646.271353:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298646.271358:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93e00/0xaf35132e6303697b lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 27 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5749 expref: 51 pid: 17578 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.271370:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93e00/0xaf35132e6303697b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 27 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5749 expref: 51 pid: 17578 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.271385:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf375e93e00/0xaf35132e6303697b lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 27 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5749 expref: 51 pid: 17578 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.271397:0:6463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298646.271401:0:6463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3200/0xaf35132e63036b03 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 26 type: IBT flags: 0x40210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.271412:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3200/0xaf35132e63036b03 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 26 type: IBT flags: 0x40290000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.271420:0:6463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3200/0xaf35132e63036b03 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 26 type: IBT flags: 0x40290000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.271444:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1780298646.271475:0:17567:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3200/0xaf35132e63036b03 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 25 type: IBT flags: 0x40210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.271480:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b2300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+50:2400:x1866777609778304:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 139us (263us total) trans 0 rc 0/0 00010000:00010000:0.0:1780298646.271489:0:17567:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3200/0xaf35132e63036b03 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 25 type: IBT flags: 0x40210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.271494:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15359 00000001:00080000:0.0:1780298646.271543:0:17567:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991138 00000004:00010000:0.0:1780298646.271568:0:17567:0:(mdt_handler.c:4479:mdt_save_lock()) ### save lock request 000000004284837a reply state 00000000619fc4c5 transno 4294991139 ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3200/0xaf35132e63036b03 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 25 type: IBT flags: 0x40210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.271577:0:17567:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf344ff3200/0xaf35132e63036b03 lrc: 3/0,1 mode: TXN/TXN res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 25 type: IBT flags: 0x50210000000000 pid: 17567 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.271582:0:17567:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:0.0:1780298646.271584:0:17567:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1780298646.271585:0:17567:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1780298646.271653:0:17567:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+55:29360:x1866777609778176:12345-192.168.203.25@tcp:36:fsx.0 Request processed in 1641us (1733us total) trans 4294991139 rc 0/0 00000100:00100000:0.0:1780298646.271660:0:17567:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62102 00000020:00080000:0.0:1780298646.271664:0:17567:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1780298646.272307:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.272347:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609778432 00000100:00100000:2.0:1780298646.272353:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609778432 00000100:00100000:2.0:1780298646.272385:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10762 00000100:00100000:2.0:1780298646.272393:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:29360:x1866777609778432:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.272401:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298646.272410:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e63036b0a lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d58ca expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.272436:0:19992:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63036a62 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [487424->2568191] (req 487424->491519) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5829 expref: 21 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.272446:0:19992:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000009fd8d343/0xaf35132e63036a3f lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2568192->4939775] (req 2568192->2580479) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5806 expref: 18 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.272454:0:19992:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e6303699e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [4939776->38494207] (req 4939776->4947967) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5765 expref: 21 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.272458:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 3 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(87 169) 00010000:00010000:2.0:1780298646.272462:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(90 169) 00010000:00010000:2.0:1780298646.272464:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e63036b0a lrc: 6/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58ca expref: 21 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.272479:0:19992:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e6303699e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [4939776->38494207] (req 4939776->4947967) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5765 expref: 21 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.272487:0:19992:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e6303699e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [4939776->38494207] (req 4939776->4947967) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5765 expref: 21 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.272494:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605150848:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298646.272536:0:19992:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000009fd8d343/0xaf35132e63036a3f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2568192->4939775] (req 2568192->2580479) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5806 expref: 18 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.272546:0:19992:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000009fd8d343/0xaf35132e63036a3f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2568192->4939775] (req 2568192->2580479) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5806 expref: 18 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.272554:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605150976:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298646.272578:0:19992:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63036a62 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [487424->2568191] (req 487424->491519) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5829 expref: 21 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.272584:0:19992:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63036a62 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [487424->2568191] (req 487424->491519) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5829 expref: 21 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.272589:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605151104:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298646.272600:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6cf00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.273289:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605150848:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298646.273301:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605150976:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298646.273309:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605151104:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298646.273349:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e63036b0a lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58ca expref: 21 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.273360:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9a600, rc 0) 00000100:00100000:2.0:1780298646.273393:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:29360:x1866777609778432:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1002us (1088us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298646.273401:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10762 00000100:00100000:3.0:1780298646.273602:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.273631:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.273656:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609778688 00000100:00100000:0.0:1780298646.273661:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609778688 00000100:00100000:3.0:1780298646.273670:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:0.0:1780298646.273686:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e6303699e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [4939776->38494207] (req 4939776->4947967) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5765 expref: 21 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.273687:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609778816 00000100:00100000:1.0:1780298646.273694:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609778816 00010000:00010000:1.0:1780298646.273717:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63036a62 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [487424->2568191] (req 487424->491519) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5829 expref: 22 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.273729:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609778560 00000100:00100000:0.0:1780298646.273734:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609778560 00000100:00100000:2.0:1780298646.273735:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15360 00000100:00100000:2.0:1780298646.273749:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b3100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2400:x1866777609778688:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298646.273758:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000009fd8d343/0xaf35132e63036a3f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2568192->4939775] (req 2568192->2580479) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5806 expref: 19 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.273760:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15361 00010000:00010000:2.0:1780298646.273760:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298646.273783:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e6303699e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [4939776->38494207] (req 4939776->4947967) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5765 expref: 22 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.273787:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348007100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2398:x1866777609778816:12345-192.168.203.25@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1780298646.273804:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15362 00010000:00010000:2.0:1780298646.273811:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e6303699e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [4939776->38494207] (req 4939776->4947967) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5765 expref: 22 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.273814:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:0.0:1780298646.273816:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348004380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2400:x1866777609778560:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298646.273834:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298646.273844:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e0cae770/0xaf35132e6303699e lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 9 type: EXT [4939776->38494207] (req 4939776->4947967) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5765 expref: 22 pid: 8356 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.273846:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63036a62 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 9 type: EXT [487424->2568191] (req 487424->491519) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5829 expref: 22 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.273850:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000009fd8d343/0xaf35132e63036a3f lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 10 type: EXT [2568192->4939775] (req 2568192->2580479) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5806 expref: 19 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.273904:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63036a62 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [487424->2568191] (req 487424->491519) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5829 expref: 21 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.273920:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000009fd8d343/0xaf35132e63036a3f lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [2568192->4939775] (req 2568192->2580479) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5806 expref: 19 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.273931:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1780298646.273945:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63036a62 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [487424->2568191] (req 487424->491519) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5829 expref: 21 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.273956:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009fd8d343/0xaf35132e63036a3f lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [2568192->4939775] (req 2568192->2580479) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5806 expref: 19 pid: 8359 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.273971:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(90 169) 00000100:00100000:2.0:1780298646.273979:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b3100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2400:x1866777609778688:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 227us (374us total) trans 0 rc 0/0 00010000:00010000:3.0:1780298646.273980:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(90 169) 00010000:00010000:3.0:1780298646.273996:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e63036b0a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58ca expref: 20 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.274003:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15360 00010000:00010000:3.0:1780298646.274018:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e63036b0a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58ca expref: 20 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.274028:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1780298646.274056:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e63036b0a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58ca expref: 19 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.274058:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348004380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609778560:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 242us (391us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298646.274086:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15362 00000100:00100000:3.0:1780298646.274179:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf348006680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777605151232:192.168.203.25@tcp:105: 00000100:00100000:3.0:1780298646.274274:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856000 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.274894:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348006680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:6462:1866777605151232:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298646.274907:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298646.274940:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348007100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+19:2398:x1866777609778816:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1166us (1308us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298646.274947:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15361 00000100:00100000:1.0:1780298646.275064:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.275164:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609778944 00000100:00100000:3.0:1780298646.275172:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609778944 00000100:00100000:3.0:1780298646.275217:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10763 00000100:00100000:3.0:1780298646.275228:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:29360:x1866777609778944:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298646.275241:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298646.275254:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd0af3fe/0xaf35132e63036b11 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d58d1 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.275287:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63036ae0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [491520->3616767] (req 491520->495615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58a7 expref: 9 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.275303:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63036a93 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3616768->37171199] (req 3616768->3620863) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d585a expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.275310:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 2 to [0x280000401:0x12:0x0].0x0 time=1780298646 v=(67 122) 00010000:00010000:3.0:1780298646.275313:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298646 v=(69 122) 00010000:00010000:3.0:1780298646.275317:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd0af3fe/0xaf35132e63036b11 lrc: 5/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58d1 expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.275340:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63036a93 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3616768->37171199] (req 3616768->3620863) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d585a expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.275354:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63036a93 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3616768->37171199] (req 3616768->3620863) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d585a expref: 9 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.275366:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf348004e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605151360:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298646.275416:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63036ae0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [491520->3616767] (req 491520->495615) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58a7 expref: 9 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.275427:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63036ae0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [491520->3616767] (req 491520->495615) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58a7 expref: 9 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.275437:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf348007480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605151488:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298646.275463:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2418563c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.276054:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348004e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605151360:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298646.276279:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf348007480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605151488:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298646.276310:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:1.0:1780298646.276316:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000cd0af3fe/0xaf35132e63036b11 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58d1 expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.276333:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93400, rc 0) 00000100:00100000:2.0:1780298646.276343:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609779072 00000100:00100000:2.0:1780298646.276348:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609779072 00010000:00010000:2.0:1780298646.276365:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63036a93 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [3616768->37171199] (req 3616768->3620863) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d585a expref: 10 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.276375:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:29360:x1866777609778944:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1147us (1309us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298646.276386:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10763 00000100:00100000:2.0:1780298646.276402:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15363 00000100:00100000:2.0:1780298646.276410:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b1c850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609779072:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298646.276418:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298646.276427:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63036a93 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [3616768->37171199] (req 3616768->3620863) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d585a expref: 9 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.276438:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63036a93 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [3616768->37171199] (req 3616768->3620863) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d585a expref: 9 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.276449:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005fc17034/0xaf35132e63036a93 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 5 type: EXT [3616768->37171199] (req 3616768->3620863) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d585a expref: 9 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.276457:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298646.276475:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b1c850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609779072:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 66us (169us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298646.276480:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15363 00000100:00100000:0.0:1780298646.276539:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.276576:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609779200 00000100:00100000:1.0:1780298646.276582:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609779200 00010000:00010000:1.0:1780298646.276602:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63036ae0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [491520->3616767] (req 491520->495615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58a7 expref: 8 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.276635:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15364 00000100:00100000:1.0:1780298646.276647:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609779200:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298646.276656:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298646.276665:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63036ae0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [491520->3616767] (req 491520->495615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58a7 expref: 8 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.276675:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63036ae0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [491520->3616767] (req 491520->495615) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58a7 expref: 8 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.276689:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63036ae0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [491520->3616767] (req 491520->495615) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58a7 expref: 8 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.276704:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298646 v=(69 122) 00010000:00010000:1.0:1780298646.276708:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298646 v=(69 122) 00010000:00010000:1.0:1780298646.276714:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000cd0af3fe/0xaf35132e63036b11 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58d1 expref: 7 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.276724:0:6463:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd0af3fe/0xaf35132e63036b11 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58d1 expref: 7 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.276748:0:6463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000cd0af3fe/0xaf35132e63036b11 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58d1 expref: 7 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.276760:0:6463:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f3480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777605151616:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298646.276801:0:6463:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e540 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.277675:0:6463:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f3480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:6463:1866777605151616:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298646.277690:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298646.277693:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.277729:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2399:x1866777609779200:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1085us (1189us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298646.277735:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.277736:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609779328 00000100:00100000:2.0:1780298646.277742:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609779328 00000100:00100000:1.0:1780298646.277746:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15364 00002000:00010000:2.0:1780298646.277782:0:8364:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:2.0:1780298646.277788:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2425f3800 with x1866777609779328 ext(4194304->18446744073709551615) 00002000:00010000:2.0:1780298646.277799:0:8364:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2425f3800. 00000100:00100000:2.0:1780298646.277819:0:8364:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609779456 00000100:00100000:2.0:1780298646.277823:0:8364:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609779456 00000100:00100000:0.0:1780298646.277826:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6591 00000100:00100000:0.0:1780298646.277835:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+19:2400:x1866777609779328:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:2.0:1780298646.277840:0:8364:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (2765177->18446744073709551615) 00002000:00100000:2.0:1780298646.277844:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2425f1180 with x1866777609779456 ext(2765177->18446744073709551615) 00002000:00010000:2.0:1780298646.277849:0:8364:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2425f1180. 00000100:00100000:2.0:1780298646.277875:0:8364:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6592 00000100:00100000:2.0:1780298646.277884:0:8364:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+7:2400:x1866777609779456:12345-192.168.203.25@tcp:10:fsx.0 00002000:00010000:0.0:1780298646.278028:0:8362:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4944749 -> 4194304 00002000:00010000:0.0:1780298646.278032:0:8362:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298646.278035:0:8362:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298646.278037:0:8362:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298646.278039:0:8362:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 536 -> 520 00000100:00100000:0.0:1780298646.278070:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2400:x1866777609779328:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 237us (379us total) trans 4294969824 rc 0/0 00000100:00100000:0.0:1780298646.278078:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6591 00002000:00010000:0.0:1780298646.278082:0:8362:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 11811161089/17 (4194304->18446744073709551615) 00002000:00100000:0.0:1780298646.278085:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2425f3800 with x1866777609779328 ext(4194304->18446744073709551615) 00002000:00010000:0.0:1780298646.278091:0:8362:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2425f3800. 00002000:00010000:2.0:1780298646.278167:0:8364:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 3619518 -> 2765177 00002000:00010000:2.0:1780298646.278172:0:8364:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:2.0:1780298646.278174:0:8364:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298646 -> 1780298646 00002000:00010000:2.0:1780298646.278177:0:8364:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:2.0:1780298646.278179:0:8364:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 32 -> 24 00000100:00100000:2.0:1780298646.278207:0:8364:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2400:x1866777609779456:12345-192.168.203.25@tcp:10:fsx.0 Request processed in 323us (473us total) trans 4294969343 rc 0/0 00000100:00100000:2.0:1780298646.278214:0:8364:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6592 00002000:00010000:2.0:1780298646.278218:0:8364:0:(ofd_dev.c:2821:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 10737419265/18 (2765177->18446744073709551615) 00002000:00100000:2.0:1780298646.278221:0:8364:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2425f1180 with x1866777609779456 ext(2765177->18446744073709551615) 00002000:00010000:2.0:1780298646.278226:0:8364:0:(ofd_dev.c:2826:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf2425f1180. 00000100:00100000:0.0:1780298646.278888:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.278930:0:9976:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609779584 00000100:00100000:2.0:1780298646.278935:0:9976:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609779584 00000100:00100000:2.0:1780298646.278971:0:9976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62103 00000100:00100000:2.0:1780298646.278979:0:9976:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+55:29360:x1866777609779584:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.279001:0:9976:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298646.279018:0:9976:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ac00/0xaf35132e63036b18 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 26 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d58d8 expref: -99 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.279094:0:9976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9be00/0xaf35132e63036b1f lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 27 type: IBT flags: 0x40000000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.279107:0:9976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9be00/0xaf35132e63036b1f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 27 type: IBT flags: 0x50210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.279117:0:9976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9be00/0xaf35132e63036b1f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 27 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1780298646.279169:0:9976:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 6959481 00000004:00200000:2.0:1780298646.279174:0:9976:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1780298646.279194:0:9976:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9be00/0xaf35132e63036b1f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 27 type: IBT flags: 0x40210000000000 pid: 9976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.279215:0:9976:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9ac00/0xaf35132e63036b18 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 27 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58d8 expref: 57 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.279230:0:9976:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243d9be00/0xaf35132e63036b1f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 26 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58d8 expref: 56 pid: 9976 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.279238:0:9976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:2.0:1780298646.279241:0:9976:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1780298646.279244:0:9976:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9be00, rc 0) 00000100:00100000:2.0:1780298646.279288:0:9976:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+56:29360:x1866777609779584:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 310us (399us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298646.279295:0:9976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62103 00000100:00100000:3.0:1780298646.280505:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.280540:0:17578:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609779712 00000100:00100000:1.0:1780298646.280545:0:17578:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609779712 00000100:00100000:1.0:1780298646.280588:0:17578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 62104 00000100:00100000:1.0:1780298646.280597:0:17578:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348004380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+50:29360:x1866777609779712:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.280609:0:17578:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.280620:0:17578:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279e00/0xaf35132e63036b26 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 27 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d58df expref: -99 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.280672:0:17578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278400/0xaf35132e63036b2d lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 28 type: IBT flags: 0x40000000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.280683:0:17578:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278400/0xaf35132e63036b2d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 28 type: IBT flags: 0x50210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.280692:0:17578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278400/0xaf35132e63036b2d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 28 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1780298646.280707:0:17578:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 6959481 00000004:00200000:1.0:1780298646.280709:0:17578:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1780298646.280719:0:17578:0:(mdt_handler.c:2510:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278400/0xaf35132e63036b2d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 28 type: IBT flags: 0x40210000000000 pid: 17578 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.280732:0:17578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243279e00/0xaf35132e63036b26 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x33:0x0].0x0 bits 0x2/0x0 rrc: 28 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58df expref: 52 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.280742:0:17578:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff9cf243278400/0xaf35132e63036b2d lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 27 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58df expref: 51 pid: 17578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.280747:0:17578:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff9cf2447a4300) 00010000:00010000:1.0:1780298646.280749:0:17578:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1780298646.280751:0:17578:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278400, rc 0) 00000100:00100000:1.0:1780298646.280777:0:17578:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348004380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+51:29360:x1866777609779712:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 183us (274us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298646.280784:0:17578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 62104 00000100:00100000:3.0:1780298646.281651:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.281688:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609779840 00000100:00100000:2.0:1780298646.281694:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609779840 00000100:00100000:2.0:1780298646.281735:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10764 00000100:00100000:2.0:1780298646.281745:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348007100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609779840:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.281754:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298646.281761:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:2.0:1780298646.281765:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b4561703/0xaf35132e63036b34 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d58e6 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.281790:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609779968 00000100:00100000:0.0:1780298646.281795:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609779968 00000100:00100000:2.0:1780298646.281821:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605151744:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.281827:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10765 00000100:00100000:0.0:1780298646.281836:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348007480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609779968:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.281845:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.281855:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c703984a/0xaf35132e63036b3b lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d58ed expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.281886:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605151872:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.281891:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.281924:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12300 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.282663:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605151744:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.282666:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605151872:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.282689:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c703984a/0xaf35132e63036b3b lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58ed expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.282691:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b4561703/0xaf35132e63036b34 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58e6 expref: 19 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.282729:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c703984a/0xaf35132e63036b3b lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58ed expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.282730:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b4561703/0xaf35132e63036b34 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58e6 expref: 19 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.282755:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2000, rc 0) 00010000:00010000:2.0:1780298646.282756:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b600, rc 0) 00000100:00100000:0.0:1780298646.282790:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348007480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609779968:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 956us (1029us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298646.282792:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348007100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:2400:x1866777609779840:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1049us (1139us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298646.282805:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10764 00000100:00100000:0.0:1780298646.282813:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10765 00000100:00100000:3.0:1780298646.283725:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.283761:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609780096 00000100:00100000:2.0:1780298646.283766:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609780096 00000100:00100000:2.0:1780298646.283797:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10766 00000100:00100000:2.0:1780298646.283804:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348004e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+18:29360:x1866777609780096:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.283813:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298646.283821:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009fd8d343/0xaf35132e63036b42 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d58f4 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.283836:0:19992:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e63036b0a lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58ca expref: 19 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.283845:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(90 169) 00010000:00010000:2.0:1780298646.283848:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(91 169) 00010000:00010000:2.0:1780298646.283851:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009fd8d343/0xaf35132e63036b42 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [5492736->5505023] (req 5492736->5505023) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58f4 expref: 19 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.283866:0:19992:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e63036b0a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58ca expref: 19 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.283875:0:19992:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e63036b0a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58ca expref: 19 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.283882:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605152000:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298646.283931:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.284539:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605152000:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298646.284550:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009fd8d343/0xaf35132e63036b42 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [5492736->5505023] (req 5492736->5505023) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58f4 expref: 19 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.284557:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b000, rc 0) 00000100:00100000:2.0:1780298646.284579:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348004e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:29360:x1866777609780096:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 776us (856us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298646.284583:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10766 00000100:00100000:3.0:1780298646.284777:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.284806:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609780224 00000100:00100000:0.0:1780298646.284810:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609780224 00010000:00010000:0.0:1780298646.284824:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e63036b0a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58ca expref: 20 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.284845:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15365 00000100:00100000:0.0:1780298646.284850:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348006680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2400:x1866777609780224:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298646.284857:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298646.284865:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e63036b0a lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58ca expref: 20 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.284873:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e63036b0a lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58ca expref: 20 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.284888:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e63036b0a lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58ca expref: 20 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.284896:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(91 169) 00010000:00010000:0.0:1780298646.284898:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(91 169) 00010000:00010000:0.0:1780298646.284903:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000009fd8d343/0xaf35132e63036b42 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [5492736->39047167] (req 5492736->5505023) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58f4 expref: 18 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.284909:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009fd8d343/0xaf35132e63036b42 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [5492736->39047167] (req 5492736->5505023) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58f4 expref: 18 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.284923:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000009fd8d343/0xaf35132e63036b42 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [5492736->39047167] (req 5492736->5505023) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58f4 expref: 18 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.284930:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f0700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777605152128:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298646.284959:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12300 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.285656:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f0700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:7400:1866777605152128:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298646.285667:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298646.285694:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348006680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+19:2400:x1866777609780224:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 843us (917us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298646.285702:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15365 00000100:00100000:0.0:1780298646.285861:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.285898:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609780352 00000100:00100000:1.0:1780298646.285904:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609780352 00000100:00100000:1.0:1780298646.285956:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6593 00000100:00100000:1.0:1780298646.285965:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+19:2400:x1866777609780352:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:1.0:1780298646.286169:0:19943:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:1.0:1780298646.286179:0:19943:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:1.0:1780298646.286181:0:19943:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298646 -> 1780298646 00002000:00010000:1.0:1780298646.286183:0:19943:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:1.0:1780298646.286185:0:19943:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 520 -> 544 00000100:00100000:1.0:1780298646.286208:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2400:x1866777609780352:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 245us (347us total) trans 4294969825 rc 0/0 00000100:00100000:1.0:1780298646.286214:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6593 00000100:00100000:3.0:1780298646.287081:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.287114:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.287148:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609780608 00000100:00100000:1.0:1780298646.287153:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609780608 00000100:00100000:1.0:1780298646.287175:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609780480 00000100:00100000:1.0:1780298646.287178:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609780480 00000100:00100000:0.0:1780298646.287187:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10767 00000100:00100000:0.0:1780298646.287199:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243845f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609780608:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298646.287205:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10768 00010000:00010000:0.0:1780298646.287210:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298646.287219:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243847480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2400:x1866777609780480:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.287222:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e63036b49 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5902 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.287230:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298646.287241:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004cdc8249/0xaf35132e63036b50 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d58fb expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.287257:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f3100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605152256:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298646.287288:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243846d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605152384:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298646.287335:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.287391:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12300 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.288014:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243846d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605152384:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.288041:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004cdc8249/0xaf35132e63036b50 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58fb expref: 21 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.288060:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f3100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605152256:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.288061:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004cdc8249/0xaf35132e63036b50 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58fb expref: 21 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.288077:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92200, rc 0) 00010000:00010000:1.0:1780298646.288086:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e63036b49 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5902 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.288102:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e63036b49 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5902 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.288114:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3800, rc 0) 00000100:00100000:0.0:1780298646.288120:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243847480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2400:x1866777609780480:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 901us (1001us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.288160:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243845f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609780608:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 963us (1078us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.288172:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10768 00000100:00100000:1.0:1780298646.288187:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10767 00000100:00100000:3.0:1780298646.289064:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.289101:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609780736 00000100:00100000:0.0:1780298646.289107:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609780736 00000100:00100000:0.0:1780298646.289161:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10769 00000100:00100000:0.0:1780298646.289172:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243845c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2399:x1866777609780736:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.289182:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.289191:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e63036b57 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5909 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.289206:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.289235:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605152512:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.289236:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609780864 00000100:00100000:2.0:1780298646.289241:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609780864 00000100:00100000:2.0:1780298646.289274:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10770 00000100:00100000:2.0:1780298646.289281:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243844a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609780864:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.289288:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298646.289295:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ad103424/0xaf35132e63036b5e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5910 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.289299:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12d80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.289321:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605152640:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.289343:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.289920:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605152512:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.289938:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e63036b57 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5909 expref: 21 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.289950:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e63036b57 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5909 expref: 21 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.289957:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3c00, rc 0) 00000100:00100000:0.0:1780298646.289992:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243845c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2399:x1866777609780736:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 824us (929us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.289996:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10769 00000100:00100000:2.0:1780298646.290029:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605152640:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298646.290045:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ad103424/0xaf35132e63036b5e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5910 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.290059:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ad103424/0xaf35132e63036b5e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5910 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.290070:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9aa00, rc 0) 00000100:00100000:2.0:1780298646.290089:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243844a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609780864:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 809us (887us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298646.290094:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10770 00000100:00100000:2.0:1780298646.291024:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.291057:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609780992 00000100:00100000:3.0:1780298646.291063:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609780992 00000100:00100000:3.0:1780298646.291098:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10771 00000100:00100000:3.0:1780298646.291107:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+19:29360:x1866777609780992:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298646.291117:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298646.291143:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e90122d2/0xaf35132e63036b65 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5917 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.291163:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(91 169) 00010000:00010000:3.0:1780298646.291168:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(91 169) 00010000:00010000:3.0:1780298646.291177:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e90122d2/0xaf35132e63036b65 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1409024->5492735] (req 1409024->1413119) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5917 expref: 20 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.291191:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e90122d2/0xaf35132e63036b65 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 3 type: EXT [1409024->5492735] (req 1409024->1413119) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5917 expref: 20 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.291200:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93c00, rc 0) 00000100:00100000:3.0:1780298646.291231:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:29360:x1866777609780992:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 126us (207us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298646.291237:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10771 00000100:00100000:3.0:1780298646.291986:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.292017:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609781120 00000100:00100000:0.0:1780298646.292022:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609781120 00002000:00100000:0.0:1780298646.292046:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (1409024->1413119) req@ffff9cf243845180 x1866777609781120/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:352/0 lens 488/0 e 0 to 0 dl 1780298657 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298646.292053:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243845180 with x1866777609781120 ext(1409024->1413119) 00002000:00010000:0.0:1780298646.292056:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf243845180 00000100:00100000:0.0:1780298646.292072:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6594 00000100:00100000:0.0:1780298646.292077:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243845180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2399:x1866777609781120:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298646.292246:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243845180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2399:x1866777609781120:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 169us (261us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298646.292254:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6594 00002000:00100000:0.0:1780298646.292259:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x2c0000401:0x11:0x0] (1409024->1413119) req@ffff9cf243845180 x1866777609781120/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:352/0 lens 488/4536 e 0 to 0 dl 1780298657 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298646.292277:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf243845180 with x1866777609781120 ext(1409024->1413119) 00002000:00010000:0.0:1780298646.292280:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf243845180 00000100:00100000:1.0:1780298646.293357:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.293387:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609781248 00000100:00100000:0.0:1780298646.293391:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609781248 00000100:00100000:0.0:1780298646.293420:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10772 00000100:00100000:0.0:1780298646.293427:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2400:x1866777609781248:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.293434:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.293442:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e63036b6c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d591e expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.293470:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605152768:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.293509:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12000 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.294167:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f1c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605152768:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.294182:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e63036b6c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d591e expref: 21 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.294194:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e63036b6c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d591e expref: 21 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.294201:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2000, rc 0) 00000100:00100000:0.0:1780298646.294223:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2400:x1866777609781248:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 798us (869us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.294229:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10772 00000100:00100000:3.0:1780298646.294923:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.294953:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609781376 00000100:00100000:1.0:1780298646.294957:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609781376 00000100:00100000:1.0:1780298646.294994:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10773 00000100:00100000:1.0:1780298646.295002:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243846d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2399:x1866777609781376:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.295013:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.295020:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63036b73 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5925 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.295065:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2558f0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605152896:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.295105:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e540 going to sleep for 11 seconds 00000100:00100000:3.0:1780298646.295759:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2558f0e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605152896:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298646.295771:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63036b73 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5925 expref: 21 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.295781:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005b92bd2b/0xaf35132e63036b73 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5925 expref: 21 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.295787:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279c00, rc 0) 00000100:00100000:3.0:1780298646.295809:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243846d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2399:x1866777609781376:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 810us (889us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298646.295816:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10773 00000100:00100000:2.0:1780298646.296820:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.296858:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609781504 00000100:00100000:0.0:1780298646.296862:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609781504 00002000:00100000:0.0:1780298646.296883:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (1888256->1892351) req@ffff9cf24304d880 x1866777609781504/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:352/0 lens 488/0 e 0 to 0 dl 1780298657 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298646.296891:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24304d880 with x1866777609781504 ext(1888256->1892351) 00002000:00010000:0.0:1780298646.296894:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24304d880 00000100:00100000:0.0:1780298646.296914:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6595 00000100:00100000:0.0:1780298646.296918:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2400:x1866777609781504:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298646.297053:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2400:x1866777609781504:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 133us (233us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298646.297061:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6595 00002000:00100000:0.0:1780298646.297068:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x2c0000401:0x11:0x0] (1888256->1892351) req@ffff9cf24304d880 x1866777609781504/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:352/0 lens 488/4536 e 0 to 0 dl 1780298657 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298646.297077:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf24304d880 with x1866777609781504 ext(1888256->1892351) 00002000:00010000:0.0:1780298646.297080:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf24304d880 00000100:00100000:0.0:1780298646.298019:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.298053:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609781632 00000100:00100000:2.0:1780298646.298057:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609781632 00000100:00100000:2.0:1780298646.298081:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10774 00000100:00100000:2.0:1780298646.298087:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2400:x1866777609781632:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.298095:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298646.298103:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e63036b7a lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d592c expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.298159:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605153024:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.298209:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.298936:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304c000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605153024:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298646.298949:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e63036b7a lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d592c expref: 21 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.298960:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e63036b7a lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d592c expref: 21 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.298969:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9a200, rc 0) 00000100:00100000:2.0:1780298646.298997:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2400:x1866777609781632:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 910us (979us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298646.299004:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10774 00000100:00100000:1.0:1780298646.299832:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.299877:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609781760 00000100:00100000:3.0:1780298646.299888:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609781760 00000100:00100000:3.0:1780298646.299928:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10775 00000100:00100000:3.0:1780298646.299935:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2400:x1866777609781760:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298646.299948:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298646.299962:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e63036b81 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5933 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.300006:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433de300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605153152:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298646.300070:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:3.0:1780298646.300765:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433de300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605153152:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298646.300784:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e63036b81 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5933 expref: 21 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.300799:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e63036b81 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5933 expref: 21 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.300806:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92a00, rc 0) 00000100:00100000:3.0:1780298646.300833:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2400:x1866777609781760:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 899us (1003us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298646.300839:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10775 00000100:00100000:1.0:1780298646.301721:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.301746:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.301763:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609781888 00000100:00100000:0.0:1780298646.301768:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609781888 00000100:00100000:2.0:1780298646.301783:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609782016 00000100:00100000:2.0:1780298646.301789:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609782016 00000100:00100000:0.0:1780298646.301807:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10776 00000100:00100000:0.0:1780298646.301818:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2399:x1866777609781888:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298646.301821:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10777 00010000:00010000:0.0:1780298646.301828:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298646.301834:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609782016:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.301837:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e63036b88 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d593a expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.301843:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298646.301854:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63036b8f lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5941 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.301870:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2425f3100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605153280:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298646.301887:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433de300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605153408:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.301915:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12000 going to sleep for 11 seconds 00000100:00100000:3.0:1780298646.301916:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.302633:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2425f3100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605153280:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.302653:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e63036b88 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d593a expref: 21 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.302667:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e63036b88 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d593a expref: 21 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.302678:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2e00, rc 0) 00000100:00100000:0.0:1780298646.302707:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:2399:x1866777609781888:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 891us (986us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298646.302711:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433de300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605153408:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.302715:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10776 00010000:00010000:3.0:1780298646.302760:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63036b8f lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5941 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.302770:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63036b8f lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5941 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.302800:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93000, rc 0) 00000100:00100000:3.0:1780298646.302821:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609782016:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 993us (1074us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298646.302825:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10777 00000100:00100000:1.0:1780298646.303723:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.303759:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609782144 00000100:00100000:2.0:1780298646.303764:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609782144 00000100:00100000:2.0:1780298646.303792:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10778 00000100:00100000:2.0:1780298646.303798:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+20:29360:x1866777609782144:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.303808:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298646.303820:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63036b96 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5948 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.303837:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(91 169) 00010000:00010000:2.0:1780298646.303841:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(91 169) 00010000:00010000:2.0:1780298646.303847:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63036b96 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3792896->5492735] (req 3792896->3796991) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5948 expref: 21 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.303873:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63036b96 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 4 type: EXT [3792896->5492735] (req 3792896->3796991) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5948 expref: 21 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.303882:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9aa00, rc 0) 00000100:00100000:2.0:1780298646.303928:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:29360:x1866777609782144:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 130us (205us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298646.303935:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10778 00000100:00100000:1.0:1780298646.304775:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.304808:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609782272 00000100:00100000:0.0:1780298646.304813:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609782272 00002000:00100000:0.0:1780298646.304837:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (3792896->3796991) req@ffff9cf2558f2680 x1866777609782272/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:352/0 lens 488/0 e 0 to 0 dl 1780298657 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298646.304844:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2558f2680 with x1866777609782272 ext(3792896->3796991) 00002000:00010000:0.0:1780298646.304848:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2558f2680 00000100:00100000:0.0:1780298646.304865:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6596 00000100:00100000:0.0:1780298646.304870:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2398:x1866777609782272:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298646.304997:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2398:x1866777609782272:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 127us (225us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1780298646.305005:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6596 00002000:00100000:0.0:1780298646.305010:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x2c0000401:0x11:0x0] (3792896->3796991) req@ffff9cf2558f2680 x1866777609782272/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:352/0 lens 488/4536 e 0 to 0 dl 1780298657 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298646.305017:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2558f2680 with x1866777609782272 ext(3792896->3796991) 00002000:00010000:0.0:1780298646.305020:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2558f2680 00000100:00100000:0.0:1780298646.306056:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.306110:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609782400 00000100:00100000:2.0:1780298646.306116:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609782400 00002000:00100000:2.0:1780298646.306182:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (3796992->3801087) req@ffff9cf2425f3100 x1866777609782400/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:352/0 lens 488/0 e 0 to 0 dl 1780298657 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298646.306193:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2425f3100 with x1866777609782400 ext(3796992->3801087) 00002000:00010000:2.0:1780298646.306198:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2425f3100 00000100:00100000:2.0:1780298646.306219:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6597 00000100:00100000:2.0:1780298646.306229:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2425f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2397:x1866777609782400:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298646.306395:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2425f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2397:x1866777609782400:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 168us (339us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1780298646.306402:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6597 00002000:00100000:2.0:1780298646.306410:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (3796992->3801087) req@ffff9cf2425f3100 x1866777609782400/t0(0) o3->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:352/0 lens 488/4536 e 0 to 0 dl 1780298657 ref 1 fl Complete:/600/0 rc 4096/4096 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298646.306419:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf2425f3100 with x1866777609782400 ext(3796992->3801087) 00002000:00010000:2.0:1780298646.306422:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf2425f3100 00010000:00010000:0.0:1780298646.307292:0:6464:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1780298646.307365:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.307401:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.307415:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609782656 00000100:00100000:3.0:1780298646.307422:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609782656 00000100:00100000:0.0:1780298646.307439:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609782528 00000100:00100000:0.0:1780298646.307443:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609782528 00000100:00100000:3.0:1780298646.307457:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10779 00000100:00100000:3.0:1780298646.307470:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609782656:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298646.307481:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298646.307487:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10780 00010000:00010000:3.0:1780298646.307493:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000056a806f/0xaf35132e63036b9d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5956 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.307499:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2398:x1866777609782528:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.307510:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.307518:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e63036ba4 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d594f expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.307534:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433de300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605153536:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.307549:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605153664:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.307601:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12d80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298646.307602:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856b40 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.308244:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433de300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605153536:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298646.308286:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000056a806f/0xaf35132e63036b9d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5956 expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.308306:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000056a806f/0xaf35132e63036b9d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5956 expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.308326:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92800, rc 0) 00000100:00100000:0.0:1780298646.308356:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605153664:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.308365:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609782656:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 898us (998us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298646.308376:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10779 00010000:00010000:0.0:1780298646.308385:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e63036ba4 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d594f expref: 22 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.308403:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e63036ba4 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d594f expref: 22 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.308418:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2000, rc 0) 00000100:00100000:0.0:1780298646.308447:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2398:x1866777609782528:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 953us (1049us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.308453:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10780 00000100:00100000:1.0:1780298646.309243:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.309281:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609782784 00000100:00100000:3.0:1780298646.309286:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609782784 00000100:00100000:1.0:1780298646.309297:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.309339:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609782912 00000100:00100000:3.0:1780298646.309344:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609782912 00000100:00100000:2.0:1780298646.309350:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10781 00000100:00100000:2.0:1780298646.309364:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2398:x1866777609782784:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.309379:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298646.309390:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e63036bab lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d595d expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.309410:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10782 00000100:00100000:3.0:1780298646.309418:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609782912:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298646.309426:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298646.309430:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf24304e680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605153792:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298646.309434:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63036bb2 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5964 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.309465:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433dce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605153920:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.309491:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:3.0:1780298646.309496:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2418563c0 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.310229:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf24304e680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605153792:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298646.310252:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e63036bab lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d595d expref: 22 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.310260:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433dce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605153920:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298646.310283:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e63036bab lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d595d expref: 22 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.310295:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9a200, rc 0) 00010000:00010000:3.0:1780298646.310304:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63036bb2 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5964 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.310326:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63036bb2 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5964 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.310328:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2398:x1866777609782784:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 967us (1084us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298646.310356:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10781 00010000:00010000:3.0:1780298646.310361:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93000, rc 0) 00000100:00100000:3.0:1780298646.310386:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609782912:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 969us (1091us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298646.310392:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10782 00000100:00100000:1.0:1780298646.311235:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.311278:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609783040 00000100:00100000:0.0:1780298646.311284:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609783040 00000100:00100000:0.0:1780298646.311316:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10783 00000100:00100000:0.0:1780298646.311323:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2558f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2398:x1866777609783040:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.311332:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.311339:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008361787d/0xaf35132e63036bb9 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d596b expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.311362:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605154048:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.311407:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12d80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.311964:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605154048:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.311978:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008361787d/0xaf35132e63036bb9 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d596b expref: 21 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.311987:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008361787d/0xaf35132e63036bb9 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d596b expref: 21 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.311994:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2800, rc 0) 00000100:00100000:0.0:1780298646.312021:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2558f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2398:x1866777609783040:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 700us (788us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.312027:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10783 00000100:00100000:1.0:1780298646.312795:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.312832:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609783168 00000100:00100000:0.0:1780298646.312837:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609783168 00000100:00100000:0.0:1780298646.312866:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10784 00000100:00100000:0.0:1780298646.312871:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2398:x1866777609783168:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.312877:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.312885:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001d4afe0b/0xaf35132e63036bc0 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5972 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.312904:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605154176:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.312935:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12d80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.313570:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605154176:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.313595:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001d4afe0b/0xaf35132e63036bc0 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5972 expref: 21 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.313609:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001d4afe0b/0xaf35132e63036bc0 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5972 expref: 21 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.313619:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3000, rc 0) 00000100:00100000:0.0:1780298646.313647:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2398:x1866777609783168:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 775us (851us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.313651:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10784 00000100:00100000:1.0:1780298646.314355:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.314384:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609783296 00000100:00100000:0.0:1780298646.314387:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609783296 00000100:00100000:0.0:1780298646.314414:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10785 00000100:00100000:0.0:1780298646.314421:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2398:x1866777609783296:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.314430:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.314438:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e63036bc7 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5979 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.314463:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605154304:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.314494:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12d80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.315065:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605154304:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.315076:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e63036bc7 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5979 expref: 21 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.315085:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e63036bc7 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5979 expref: 21 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.315092:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2e00, rc 0) 00000100:00100000:0.0:1780298646.315113:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2398:x1866777609783296:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 694us (760us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.315118:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10785 00000100:00100000:1.0:1780298646.315914:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.315951:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609783424 00000100:00100000:3.0:1780298646.315958:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609783424 00000100:00100000:3.0:1780298646.315983:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10786 00000100:00100000:3.0:1780298646.315988:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2398:x1866777609783424:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298646.315996:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298646.316003:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e63036bce lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5980 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.316025:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433dce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605154432:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298646.316057:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2418563c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298646.316664:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433dce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605154432:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298646.316676:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e63036bce lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5980 expref: 21 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.316686:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e63036bce lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5980 expref: 21 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.316693:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92a00, rc 0) 00000100:00100000:3.0:1780298646.316714:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2398:x1866777609783424:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 727us (803us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298646.316719:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10786 00000100:00100000:1.0:1780298646.317551:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.317580:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609783552 00000100:00100000:0.0:1780298646.317584:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609783552 00000100:00100000:0.0:1780298646.317608:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10787 00000100:00100000:0.0:1780298646.317617:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2398:x1866777609783552:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.317623:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.317630:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e63036bd5 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5987 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.317651:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605154560:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.317680:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12d80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.318252:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605154560:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.318281:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e63036bd5 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5987 expref: 21 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.318293:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e63036bd5 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5987 expref: 21 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.318300:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3c00, rc 0) 00000100:00100000:0.0:1780298646.318322:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2398:x1866777609783552:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 710us (772us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.318327:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10787 00000100:00100000:1.0:1780298646.319111:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.319185:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609783680 00000100:00100000:2.0:1780298646.319194:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609783680 00000100:00100000:2.0:1780298646.319244:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6598 00000100:00100000:2.0:1780298646.319252:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609783680:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298646.319450:0:19943:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 2765177 -> 2765177 00002000:00010000:2.0:1780298646.319454:0:19943:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:2.0:1780298646.319457:0:19943:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298646 -> 1780298646 00002000:00010000:2.0:1780298646.319459:0:19943:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:2.0:1780298646.319462:0:19943:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 24 -> 40 00000100:00100000:2.0:1780298646.319509:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609783680:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 257us (398us total) trans 4294969344 rc 0/0 00000100:00100000:2.0:1780298646.319517:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6598 00000100:00100000:1.0:1780298646.320219:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.320245:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609783808 00000100:00100000:0.0:1780298646.320250:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609783808 00000100:00100000:0.0:1780298646.320287:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10788 00000100:00100000:0.0:1780298646.320297:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e426c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2397:x1866777609783808:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.320305:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.320319:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e63036bdc lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d598e expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.320342:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605154688:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.320373:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12d80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.320971:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605154688:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.320984:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e63036bdc lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d598e expref: 21 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.320994:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e63036bdc lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d598e expref: 21 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.321001:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2000, rc 0) 00000100:00100000:0.0:1780298646.321024:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e426c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2397:x1866777609783808:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 730us (807us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.321029:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10788 00000100:00100000:1.0:1780298646.321740:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.321767:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609783936 00000100:00100000:0.0:1780298646.321771:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609783936 00000100:00100000:0.0:1780298646.321794:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10789 00000100:00100000:0.0:1780298646.321798:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2398:x1866777609783936:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.321804:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.321810:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e63036be3 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5995 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.321829:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605154816:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.321857:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12d80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.322469:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605154816:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.322481:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e63036be3 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5995 expref: 21 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.322495:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e63036be3 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5995 expref: 21 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.322502:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3800, rc 0) 00000100:00100000:0.0:1780298646.322527:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2398:x1866777609783936:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 728us (787us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.322531:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10789 00000100:00100000:1.0:1780298646.323178:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.323211:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609784064 00000100:00100000:3.0:1780298646.323215:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609784064 00000100:00100000:3.0:1780298646.323238:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10790 00000100:00100000:3.0:1780298646.323243:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:29360:x1866777609784064:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298646.323250:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298646.323257:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e63036bea lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d599c expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.323290:0:8358:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e90122d2/0xaf35132e63036b65 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1409024->5492735] (req 1409024->1413119) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5917 expref: 21 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.323298:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(91 169) 00010000:00010000:3.0:1780298646.323301:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(92 169) 00010000:00010000:3.0:1780298646.323305:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e63036bea lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2355200->2363391] (req 2355200->2363391) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d599c expref: 21 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.323321:0:8358:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e90122d2/0xaf35132e63036b65 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1409024->5492735] (req 1409024->1413119) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5917 expref: 21 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.323332:0:8358:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e90122d2/0xaf35132e63036b65 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1409024->5492735] (req 1409024->1413119) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5917 expref: 21 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.323340:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433dce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605154944:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298646.323370:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2418563c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298646.323842:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433dce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605154944:192.168.203.25@tcp:104: 00010000:00010000:3.0:1780298646.323854:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e63036bea lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2355200->2363391] (req 2355200->2363391) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d599c expref: 21 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.323861:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92a00, rc 0) 00000100:00100000:3.0:1780298646.323882:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:29360:x1866777609784064:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 640us (706us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298646.323888:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10790 00000100:00100000:1.0:1780298646.324172:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.324200:0:8374:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609784192 00000100:00100000:0.0:1780298646.324204:0:8374:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609784192 00010000:00010000:0.0:1780298646.324219:0:8374:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e90122d2/0xaf35132e63036b65 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [1409024->5492735] (req 1409024->1413119) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5917 expref: 21 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.324239:0:8374:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15366 00000100:00100000:0.0:1780298646.324250:0:8374:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2399:x1866777609784192:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298646.324261:0:8374:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298646.324284:0:8374:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000e90122d2/0xaf35132e63036b65 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [1409024->5492735] (req 1409024->1413119) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5917 expref: 21 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.324296:0:8374:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e90122d2/0xaf35132e63036b65 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [1409024->5492735] (req 1409024->1413119) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5917 expref: 21 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.324314:0:8374:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e90122d2/0xaf35132e63036b65 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [1409024->5492735] (req 1409024->1413119) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5917 expref: 21 pid: 12101 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.324329:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(92 169) 00010000:00010000:0.0:1780298646.324332:0:8374:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(92 169) 00010000:00010000:0.0:1780298646.324338:0:8374:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e63036bea lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2355200->3792895] (req 2355200->2363391) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d599c expref: 20 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.324344:0:8374:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e63036bea lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2355200->3792895] (req 2355200->2363391) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d599c expref: 20 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.324360:0:8374:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000c361b03c/0xaf35132e63036bea lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [2355200->3792895] (req 2355200->2363391) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d599c expref: 20 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.324366:0:8374:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad8380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777605155072:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298646.324393:0:8374:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12d80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.325033:0:8374:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad8380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:8374:1866777605155072:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298646.325045:0:8374:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298646.325080:0:8374:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2399:x1866777609784192:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 833us (907us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298646.325090:0:8374:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15366 00000100:00100000:1.0:1780298646.325096:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.325173:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609784320 00000100:00100000:3.0:1780298646.325178:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609784320 00000100:00100000:3.0:1780298646.325216:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6599 00000100:00100000:3.0:1780298646.325222:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+20:2399:x1866777609784320:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:3.0:1780298646.325397:0:16972:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:3.0:1780298646.325401:0:16972:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:3.0:1780298646.325402:0:16972:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298646 -> 1780298646 00002000:00010000:3.0:1780298646.325405:0:16972:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:3.0:1780298646.325407:0:16972:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 544 -> 544 00000100:00100000:3.0:1780298646.325439:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2399:x1866777609784320:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 216us (343us total) trans 4294969826 rc 0/0 00000100:00100000:3.0:1780298646.325446:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6599 00000100:00100000:1.0:1780298646.326165:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.326191:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609784448 00000100:00100000:0.0:1780298646.326194:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609784448 00000100:00100000:0.0:1780298646.326216:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10791 00000100:00100000:0.0:1780298646.326221:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24339e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2400:x1866777609784448:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.326229:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.326236:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e63036bf1 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d59a3 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.326257:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605155200:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.326299:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12d80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.326869:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605155200:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.326884:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e63036bf1 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d59a3 expref: 22 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.326896:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e63036bf1 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d59a3 expref: 22 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.326906:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2e00, rc 0) 00000100:00100000:0.0:1780298646.326934:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24339e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2400:x1866777609784448:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 712us (769us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.326941:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10791 00000100:00100000:2.0:1780298646.327674:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.327709:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609784576 00000100:00100000:0.0:1780298646.327714:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609784576 00000100:00100000:0.0:1780298646.327741:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10792 00000100:00100000:0.0:1780298646.327747:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2400:x1866777609784576:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.327754:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.327760:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001d4afe0b/0xaf35132e63036bf8 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d59aa expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.327778:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605155328:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.327805:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12d80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.328369:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605155328:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.328384:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001d4afe0b/0xaf35132e63036bf8 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d59aa expref: 22 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.328398:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001d4afe0b/0xaf35132e63036bf8 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d59aa expref: 22 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.328408:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3000, rc 0) 00000100:00100000:0.0:1780298646.328438:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2400:x1866777609784576:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 691us (765us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.328445:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10792 00000100:00100000:1.0:1780298646.329160:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.329197:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609784704 00000100:00100000:2.0:1780298646.329201:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609784704 00000100:00100000:2.0:1780298646.329229:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6600 00000100:00100000:2.0:1780298646.329234:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25488b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609784704:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:2.0:1780298646.329332:0:8363:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 2765177 -> 2765177 00002000:00010000:2.0:1780298646.329334:0:8363:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:2.0:1780298646.329336:0:8363:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298646 -> 1780298646 00002000:00010000:2.0:1780298646.329337:0:8363:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:2.0:1780298646.329338:0:8363:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 40 -> 40 00000100:00100000:2.0:1780298646.329362:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25488b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609784704:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 129us (204us total) trans 4294969345 rc 0/0 00000100:00100000:2.0:1780298646.329366:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6600 00000100:00100000:2.0:1780298646.330022:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.330051:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.330069:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609784832 00000100:00100000:0.0:1780298646.330075:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609784832 00000100:00100000:3.0:1780298646.330089:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609784960 00000100:00100000:3.0:1780298646.330094:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609784960 00000100:00100000:0.0:1780298646.330112:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10793 00000100:00100000:3.0:1780298646.330120:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10794 00000100:00100000:3.0:1780298646.330146:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609784960:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298646.330151:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2399:x1866777609784832:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298646.330152:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298646.330161:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63036bff lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d59b8 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.330162:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.330172:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e63036c06 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d59b1 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.330186:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433dce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605155456:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.330208:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605155584:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298646.330224:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2418563c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.330241:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12d80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298646.330883:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433dce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605155456:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298646.330898:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63036bff lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d59b8 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.330909:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605155584:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298646.330917:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63036bff lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d59b8 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.330926:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93000, rc 0) 00010000:00010000:0.0:1780298646.330930:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e63036c06 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d59b1 expref: 22 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.330946:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e63036c06 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d59b1 expref: 22 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.330951:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609784960:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 806us (899us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298646.330963:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10794 00010000:00010000:0.0:1780298646.330963:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3c00, rc 0) 00000100:00100000:0.0:1780298646.330989:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2399:x1866777609784832:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 869us (967us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.330996:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10793 00000100:00100000:1.0:1780298646.331795:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.331822:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609785088 00000100:00100000:0.0:1780298646.331826:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609785088 00000100:00100000:1.0:1780298646.331831:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.331855:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609785216 00000100:00100000:0.0:1780298646.331859:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609785216 00000100:00100000:3.0:1780298646.331865:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10795 00000100:00100000:0.0:1780298646.331887:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10796 00000100:00100000:0.0:1780298646.331897:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf25488aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609785216:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298646.331902:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254888380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2399:x1866777609785088:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.331910:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298646.331911:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.331921:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008361787d/0xaf35132e63036c0d lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d59c6 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.331922:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002f8ed852/0xaf35132e63036c14 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d59bf expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.331956:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433dce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605155712:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.331967:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605155840:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298646.331991:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2418563c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.331993:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12d80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298646.332643:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433dce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605155712:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298646.332656:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002f8ed852/0xaf35132e63036c14 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d59bf expref: 22 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.332666:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002f8ed852/0xaf35132e63036c14 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d59bf expref: 22 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.332674:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92c00, rc 0) 00000100:00100000:3.0:1780298646.332694:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254888380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:2399:x1866777609785088:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 823us (900us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.332719:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605155840:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298646.332736:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10795 00010000:00010000:0.0:1780298646.332740:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008361787d/0xaf35132e63036c0d lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d59c6 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.332752:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008361787d/0xaf35132e63036c0d lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d59c6 expref: 10 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.332759:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2800, rc 0) 00000100:00100000:0.0:1780298646.332781:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf25488aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609785216:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 889us (951us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.332786:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10796 00000100:00100000:1.0:1780298646.333488:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.333508:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609785344 00000100:00100000:3.0:1780298646.333512:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609785344 00000100:00100000:3.0:1780298646.333533:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10797 00000100:00100000:3.0:1780298646.333538:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf254888a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+21:29360:x1866777609785344:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298646.333544:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298646.333549:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c1a1ebef/0xaf35132e63036c1b lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d59cd expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.333560:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(92 169) 00010000:00010000:3.0:1780298646.333563:0:8358:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(92 169) 00010000:00010000:3.0:1780298646.333568:0:8358:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c1a1ebef/0xaf35132e63036c1b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [733184->2355199] (req 733184->741375) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d59cd expref: 22 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.333576:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c1a1ebef/0xaf35132e63036c1b lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 5 type: EXT [733184->2355199] (req 733184->741375) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d59cd expref: 22 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.333581:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93a00, rc 0) 00000100:00100000:3.0:1780298646.333600:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf254888a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:29360:x1866777609785344:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 63us (113us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298646.333604:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10797 00000100:00100000:2.0:1780298646.334195:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.334223:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609785472 00000100:00100000:0.0:1780298646.334228:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609785472 00000100:00100000:0.0:1780298646.334259:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6601 00000100:00100000:0.0:1780298646.334276:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf24304f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+22:2399:x1866777609785472:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:0.0:1780298646.334386:0:8362:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size from disk: 4194304 -> 4194304 00002000:00010000:0.0:1780298646.334388:0:8362:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298646.334390:0:8362:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb atime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298646.334391:0:8362:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:0.0:1780298646.334393:0:8362:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 544 -> 560 00000100:00100000:0.0:1780298646.334415:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf24304f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2399:x1866777609785472:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 150us (220us total) trans 4294969827 rc 0/0 00000100:00100000:0.0:1780298646.334419:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6601 00000100:00100000:1.0:1780298646.335056:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.335086:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609785600 00000100:00100000:0.0:1780298646.335090:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609785600 00000100:00100000:0.0:1780298646.335110:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10798 00000100:00100000:0.0:1780298646.335115:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2397:x1866777609785600:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.335120:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.335142:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001d4afe0b/0xaf35132e63036c22 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d59d4 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.335165:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244ad8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605155968:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.335193:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12d80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.335715:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244ad8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605155968:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.335726:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001d4afe0b/0xaf35132e63036c22 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d59d4 expref: 22 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.335735:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001d4afe0b/0xaf35132e63036c22 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d59d4 expref: 22 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.335741:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3000, rc 0) 00000100:00100000:0.0:1780298646.335761:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2397:x1866777609785600:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 646us (706us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.335766:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10798 00000100:00100000:2.0:1780298646.336433:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.336468:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609785728 00000100:00100000:1.0:1780298646.336472:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609785728 00000100:00100000:1.0:1780298646.336494:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10799 00000100:00100000:1.0:1780298646.336501:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2398:x1866777609785728:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.336509:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.336517:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63036c29 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d59db expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.336543:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7bc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605156096:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.336575:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e540 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.337094:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7bc700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605156096:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298646.337105:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63036c29 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d59db expref: 22 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.337113:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63036c29 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d59db expref: 22 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.337119:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278a00, rc 0) 00000100:00100000:2.0:1780298646.337159:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2398:x1866777609785728:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 658us (724us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298646.337168:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10799 00000100:00100000:0.0:1780298646.337831:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.337864:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609785856 00000100:00100000:1.0:1780298646.337867:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609785856 00000100:00100000:1.0:1780298646.337890:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10800 00000100:00100000:1.0:1780298646.337897:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2397:x1866777609785856:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.337904:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.337911:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e63036c30 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d59e2 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.337933:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f7be680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605156224:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.337963:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.338588:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f7be680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605156224:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.338603:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e63036c30 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d59e2 expref: 22 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.338616:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009aa43742/0xaf35132e63036c30 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d59e2 expref: 22 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.338625:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279e00, rc 0) 00000100:00100000:0.0:1780298646.338653:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2397:x1866777609785856:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 757us (821us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.338660:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10800 00000100:00100000:2.0:1780298646.339426:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.339453:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609785984 00000100:00100000:3.0:1780298646.339457:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609785984 00000100:00100000:3.0:1780298646.339477:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10801 00000100:00100000:3.0:1780298646.339482:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2398:x1866777609785984:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298646.339488:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298646.339494:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000186136f8/0xaf35132e63036c37 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d59e9 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.339512:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433dce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605156352:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298646.339540:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2418563c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298646.340155:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433dce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605156352:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298646.340167:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000186136f8/0xaf35132e63036c37 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d59e9 expref: 22 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.340175:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000186136f8/0xaf35132e63036c37 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d59e9 expref: 22 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.340181:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93000, rc 0) 00000100:00100000:3.0:1780298646.340201:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2398:x1866777609785984:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 720us (777us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298646.340207:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10801 00000100:00100000:1.0:1780298646.340927:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.340955:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609786112 00000100:00100000:0.0:1780298646.340959:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609786112 00000100:00100000:0.0:1780298646.340983:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10802 00000100:00100000:0.0:1780298646.340990:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7be300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2397:x1866777609786112:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.340996:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.341001:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008361787d/0xaf35132e63036c3e lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d59f0 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.341021:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244adad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605156480:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.341050:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12d80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.341590:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244adad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605156480:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.341603:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008361787d/0xaf35132e63036c3e lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d59f0 expref: 22 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.341613:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008361787d/0xaf35132e63036c3e lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d59f0 expref: 22 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.341619:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2800, rc 0) 00000100:00100000:0.0:1780298646.341641:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7be300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2397:x1866777609786112:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 652us (714us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.341645:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10802 00000100:00100000:1.0:1780298646.342295:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.342340:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609786240 00000100:00100000:2.0:1780298646.342345:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609786240 00000100:00100000:2.0:1780298646.342370:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10803 00000100:00100000:2.0:1780298646.342375:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2398:x1866777609786240:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.342382:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298646.342388:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63036c45 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d59f7 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.342408:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605156608:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.342444:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.343029:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8b0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605156608:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298646.343047:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63036c45 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d59f7 expref: 22 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.343056:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63036c45 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d59f7 expref: 22 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.343062:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9ac00, rc 0) 00000100:00100000:2.0:1780298646.343084:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2398:x1866777609786240:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 710us (790us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298646.343089:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10803 00000100:00100000:1.0:1780298646.344175:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.344201:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609786368 00000100:00100000:0.0:1780298646.344205:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609786368 00000100:00100000:0.0:1780298646.344236:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10804 00000100:00100000:0.0:1780298646.344246:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:29360:x1866777609786368:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.344255:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.344276:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001d4afe0b/0xaf35132e63036c4c lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d59fe expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.344296:0:20252:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63036b96 lrc: 2/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [3792896->5492735] (req 3792896->3796991) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5948 expref: 24 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.344302:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(92 169) 00010000:00010000:0.0:1780298646.344305:0:20252:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(93 169) 00010000:00010000:0.0:1780298646.344308:0:20252:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001d4afe0b/0xaf35132e63036c4c lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [4222976->4227071] (req 4222976->4227071) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d59fe expref: 24 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.344323:0:20252:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63036b96 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [3792896->5492735] (req 3792896->3796991) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5948 expref: 24 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.344333:0:20252:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63036b96 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [3792896->5492735] (req 3792896->3796991) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5948 expref: 24 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.344340:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244adad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605156736:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298646.344370:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12d80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.344944:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244adad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605156736:192.168.203.25@tcp:104: 00010000:00010000:0.0:1780298646.344959:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001d4afe0b/0xaf35132e63036c4c lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [4222976->4227071] (req 4222976->4227071) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d59fe expref: 24 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.344977:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3000, rc 0) 00000100:00100000:0.0:1780298646.345008:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+24:29360:x1866777609786368:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 764us (832us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298646.345015:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10804 00000100:00100000:1.0:1780298646.345151:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.345176:0:20253:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609786496 00000100:00100000:2.0:1780298646.345181:0:20253:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609786496 00010000:00010000:2.0:1780298646.345196:0:20253:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63036b96 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [3792896->5492735] (req 3792896->3796991) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5948 expref: 24 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.345217:0:20253:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15367 00000100:00100000:2.0:1780298646.345225:0:20253:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bf100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+24:2399:x1866777609786496:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298646.345235:0:20253:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298646.345247:0:20253:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63036b96 lrc: 4/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [3792896->5492735] (req 3792896->3796991) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5948 expref: 24 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.345258:0:20253:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63036b96 lrc: 3/0,0 mode: PR/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [3792896->5492735] (req 3792896->3796991) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5948 expref: 24 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.345290:0:20253:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63036b96 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [3792896->5492735] (req 3792896->3796991) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5948 expref: 24 pid: 19992 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.345300:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(93 169) 00010000:00010000:2.0:1780298646.345304:0:20253:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(93 169) 00010000:00010000:2.0:1780298646.345312:0:20253:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000001d4afe0b/0xaf35132e63036c4c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [4222976->5492735] (req 4222976->4227071) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d59fe expref: 23 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.345321:0:20253:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001d4afe0b/0xaf35132e63036c4c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [4222976->5492735] (req 4222976->4227071) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d59fe expref: 23 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.345344:0:20253:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000001d4afe0b/0xaf35132e63036c4c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [4222976->5492735] (req 4222976->4227071) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d59fe expref: 23 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.345356:0:20253:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b0000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777605156864:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298646.345384:0:20253:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.345965:0:20253:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8b0000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:20253:1866777605156864:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298646.345976:0:20253:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298646.345998:0:20253:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bf100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2399:x1866777609786496:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 776us (876us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298646.346003:0:20253:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15367 00000100:00100000:1.0:1780298646.346414:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.346438:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609786624 00000100:00100000:3.0:1780298646.346444:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609786624 00000100:00100000:3.0:1780298646.346474:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10805 00000100:00100000:3.0:1780298646.346480:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2399:x1866777609786624:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298646.346486:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298646.346493:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002f8ed852/0xaf35132e63036c53 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5a05 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.346528:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433dce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605156992:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298646.346565:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2418563c0 going to sleep for 11 seconds 00002000:00010000:3.0:1780298646.347146:0:8358:0:(ofd_lvb.c:259:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb size: 4194304 -> 4226246 00000100:00100000:3.0:1780298646.347159:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433dce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605156992:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298646.347176:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002f8ed852/0xaf35132e63036c53 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a05 expref: 22 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.347190:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002f8ed852/0xaf35132e63036c53 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a05 expref: 22 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.347200:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92c00, rc 0) 00000100:00100000:3.0:1780298646.347238:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2399:x1866777609786624:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 758us (824us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298646.347244:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10805 00000100:00100000:1.0:1780298646.348061:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.348101:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609786752 00000100:00100000:0.0:1780298646.348106:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609786752 00000100:00100000:0.0:1780298646.348163:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10806 00000100:00100000:0.0:1780298646.348173:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7be680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2397:x1866777609786752:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.348181:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.348190:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e63036c5a lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5a0c expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.348220:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244adad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605157120:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.348273:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12d80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.348861:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244adad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605157120:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.348880:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e63036c5a lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a0c expref: 22 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.348894:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e63036c5a lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a0c expref: 22 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.348905:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3c00, rc 0) 00000100:00100000:0.0:1780298646.348936:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7be680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2397:x1866777609786752:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 766us (874us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.348944:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10806 00000100:00100000:1.0:1780298646.349729:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.349768:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609786880 00000100:00100000:2.0:1780298646.349773:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609786880 00000100:00100000:2.0:1780298646.349806:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10807 00000100:00100000:2.0:1780298646.349814:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f7bc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2398:x1866777609786880:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.349823:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298646.349830:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e63036c61 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5a13 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.349859:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605157248:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.349900:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.350556:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8b0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605157248:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298646.350571:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e63036c61 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a13 expref: 22 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.350594:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e63036c61 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a13 expref: 22 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.350609:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9a200, rc 0) 00000100:00100000:2.0:1780298646.350635:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f7bc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2398:x1866777609786880:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 823us (907us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298646.350643:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10807 00000100:00100000:1.0:1780298646.351638:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.351677:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609787008 00000100:00100000:0.0:1780298646.351682:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609787008 00000100:00100000:0.0:1780298646.351711:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10808 00000100:00100000:0.0:1780298646.351720:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2397:x1866777609787008:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.351728:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.351736:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008361787d/0xaf35132e63036c68 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5a1a expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.351762:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244adad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605157376:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.351796:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12d80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.352494:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244adad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605157376:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.352511:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008361787d/0xaf35132e63036c68 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a1a expref: 22 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.352526:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008361787d/0xaf35132e63036c68 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a1a expref: 22 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.352536:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2800, rc 0) 00000100:00100000:0.0:1780298646.352566:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2397:x1866777609787008:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 847us (929us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.352573:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10808 00000100:00100000:1.0:1780298646.353389:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.353428:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609787136 00000100:00100000:2.0:1780298646.353433:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609787136 00002000:00100000:2.0:1780298646.353458:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x12:0x0] (3375104->3379199) req@ffff9cf3481fd500 x1866777609787136/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:352/0 lens 488/0 e 0 to 0 dl 1780298657 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298646.353466:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fd500 with x1866777609787136 ext(3375104->3379199) 00002000:00010000:2.0:1780298646.353471:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf3481fd500 00000100:00100000:2.0:1780298646.353492:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6602 00000100:00100000:2.0:1780298646.353502:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609787136:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:2.0:1780298646.353649:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609787136:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 148us (258us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298646.353656:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6602 00002000:00100000:2.0:1780298646.353664:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x280000401:0x12:0x0] (3375104->3379199) req@ffff9cf3481fd500 x1866777609787136/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:352/0 lens 488/440 e 0 to 0 dl 1780298657 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1780298646.353672:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf3481fd500 with x1866777609787136 ext(3375104->3379199) 00002000:00010000:2.0:1780298646.353676:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf3481fd500 00000100:00100000:1.0:1780298646.354661:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.354690:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609787264 00000100:00100000:3.0:1780298646.354695:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609787264 00000100:00100000:3.0:1780298646.354721:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10809 00000100:00100000:3.0:1780298646.354725:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2398:x1866777609787264:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298646.354732:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298646.354742:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000186136f8/0xaf35132e63036c6f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5a21 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.354771:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433dce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605157504:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298646.354811:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2418563c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298646.355606:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433dce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605157504:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298646.355628:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000186136f8/0xaf35132e63036c6f lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a21 expref: 22 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.355645:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000186136f8/0xaf35132e63036c6f lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a21 expref: 22 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.355656:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93000, rc 0) 00000100:00100000:3.0:1780298646.355686:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2398:x1866777609787264:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 960us (1027us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298646.355692:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10809 00000100:00100000:1.0:1780298646.356599:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.356640:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609787392 00000100:00100000:0.0:1780298646.356645:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609787392 00000100:00100000:0.0:1780298646.356674:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10810 00000100:00100000:0.0:1780298646.356679:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2397:x1866777609787392:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.356688:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.356696:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e63036c76 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5a28 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.356734:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244adad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605157632:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.356779:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12d80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.357549:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244adad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605157632:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.357570:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e63036c76 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a28 expref: 22 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.357585:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e63036c76 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a28 expref: 22 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.357596:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3c00, rc 0) 00000100:00100000:0.0:1780298646.357626:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2397:x1866777609787392:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 946us (1029us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.357633:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10810 00000100:00100000:1.0:1780298646.358569:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.358613:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609787520 00000100:00100000:2.0:1780298646.358619:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609787520 00000100:00100000:2.0:1780298646.358649:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10811 00000100:00100000:2.0:1780298646.358659:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481fce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2397:x1866777609787520:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.358668:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298646.358676:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e63036c7d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5a2f expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.358711:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605157760:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.358758:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.359476:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8b0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605157760:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298646.359497:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e63036c7d lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a2f expref: 22 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.359508:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e63036c7d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a2f expref: 22 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.359514:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9a200, rc 0) 00000100:00100000:2.0:1780298646.359537:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481fce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2397:x1866777609787520:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 883us (972us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298646.359542:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10811 00000100:00100000:1.0:1780298646.360390:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.360422:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609787648 00000100:00100000:0.0:1780298646.360427:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609787648 00000100:00100000:0.0:1780298646.360453:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10812 00000100:00100000:0.0:1780298646.360459:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf3481ff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2398:x1866777609787648:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.360468:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.360476:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008361787d/0xaf35132e63036c84 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5a36 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.360509:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244adad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605157888:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.360552:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12d80 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.361225:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244adad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605157888:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.361240:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008361787d/0xaf35132e63036c84 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a36 expref: 22 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.361252:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008361787d/0xaf35132e63036c84 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a36 expref: 22 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.361259:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2800, rc 0) 00000100:00100000:0.0:1780298646.361293:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf3481ff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2398:x1866777609787648:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 834us (907us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.361298:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10812 00000100:00100000:1.0:1780298646.362248:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.362306:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609787776 00000100:00100000:3.0:1780298646.362311:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609787776 00000100:00100000:1.0:1780298646.362345:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.362360:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609787904 00000100:00100000:3.0:1780298646.362364:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609787904 00000100:00100000:2.0:1780298646.362380:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10813 00000100:00100000:2.0:1780298646.362392:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f755180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2397:x1866777609787776:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:0.0:1780298646.362396:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10814 00010000:00010000:2.0:1780298646.362405:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298646.362410:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f756680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609787904:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.362417:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63036c8b lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5a3d expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.362419:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.362432:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e63036c92 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5a44 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.362457:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605158016:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.362468:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244adad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605158144:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.362504:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.362513:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12d80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.363357:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8b0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605158016:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.363363:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244adad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605158144:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298646.363384:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63036c8b lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a3d expref: 24 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.363390:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e63036c92 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a44 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.363409:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63036c8b lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a3d expref: 24 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.363413:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f2b1eebc/0xaf35132e63036c92 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a44 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.363428:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9aa00, rc 0) 00010000:00010000:0.0:1780298646.363431:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3800, rc 0) 00000100:00100000:2.0:1780298646.363469:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f755180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2397:x1866777609787776:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1080us (1222us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.363471:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f756680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609787904:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1064us (1125us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298646.363483:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10813 00000100:00100000:0.0:1780298646.363500:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10814 00000100:00100000:1.0:1780298646.364367:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.364396:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609788032 00000100:00100000:2.0:1780298646.364401:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609788032 00000100:00100000:2.0:1780298646.364429:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10815 00000100:00100000:2.0:1780298646.364435:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f756300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609788032:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.364441:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298646.364448:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e63036c99 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5a4b expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.364463:0:9128:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000cd0af3fe/0xaf35132e63036b11 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58d1 expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.364469:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298646 v=(69 122) 00010000:00010000:2.0:1780298646.364473:0:9128:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298646 v=(70 122) 00010000:00010000:2.0:1780298646.364477:0:9128:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e63036c99 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2195456->2207743] (req 2195456->2207743) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a4b expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.364493:0:9128:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000cd0af3fe/0xaf35132e63036b11 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58d1 expref: 9 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.364503:0:9128:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000cd0af3fe/0xaf35132e63036b11 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58d1 expref: 9 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.364510:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605158272:192.168.203.25@tcp:104: 00000100:00100000:2.0:1780298646.364541:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.365187:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8b0000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605158272:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298646.365206:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e63036c99 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2195456->2207743] (req 2195456->2207743) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a4b expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.365214:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9ac00, rc 0) 00000100:00100000:2.0:1780298646.365236:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f756300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609788032:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 802us (871us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298646.365241:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10815 00000100:00100000:1.0:1780298646.365376:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.365421:0:6462:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609788160 00000100:00100000:2.0:1780298646.365426:0:6462:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609788160 00010000:00010000:2.0:1780298646.365447:0:6462:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000cd0af3fe/0xaf35132e63036b11 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58d1 expref: 10 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.365476:0:6462:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15368 00000100:00100000:2.0:1780298646.365483:0:6462:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f757b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609788160:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1780298646.365492:0:6462:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1780298646.365504:0:6462:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000cd0af3fe/0xaf35132e63036b11 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58d1 expref: 10 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.365517:0:6462:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000cd0af3fe/0xaf35132e63036b11 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58d1 expref: 10 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.365534:0:6462:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd0af3fe/0xaf35132e63036b11 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d58d1 expref: 10 pid: 8358 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.365549:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298646 v=(70 122) 00010000:00010000:2.0:1780298646.365553:0:6462:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298646 v=(70 122) 00010000:00010000:2.0:1780298646.365561:0:6462:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e63036c99 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2195456->35749887] (req 2195456->2207743) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a4b expref: 9 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.365571:0:6462:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e63036c99 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2195456->35749887] (req 2195456->2207743) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a4b expref: 9 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.365604:0:6462:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e63036c99 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2195456->35749887] (req 2195456->2207743) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a4b expref: 9 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.365614:0:6462:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b0000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777605158400:192.168.203.25@tcp:105: 00000100:00100000:2.0:1780298646.365652:0:6462:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.366472:0:6462:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8b0000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:6462:1866777605158400:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298646.366479:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:2.0:1780298646.366489:0:6462:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298646.366517:0:6462:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f757b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2399:x1866777609788160:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1034us (1141us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298646.366525:0:6462:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15368 00000100:00100000:3.0:1780298646.366558:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609788288 00000100:00100000:3.0:1780298646.366562:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609788288 00000100:00100000:3.0:1780298646.366638:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6603 00000100:00100000:3.0:1780298646.366645:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f755c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609788288:12345-192.168.203.25@tcp:22:fsx.0 00002000:00010000:3.0:1780298646.366782:0:19943:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size from disk: 2765177 -> 2765177 00002000:00010000:3.0:1780298646.366785:0:19943:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb mtime from disk: 1780298646 -> 1780298646 00002000:00010000:3.0:1780298646.366787:0:19943:0:(ofd_lvb.c:318:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb atime from disk: 1780298646 -> 1780298646 00002000:00010000:3.0:1780298646.366788:0:19943:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb ctime from disk: 1780298646 -> 1780298646 00002000:00010000:3.0:1780298646.366789:0:19943:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks from disk: 40 -> 64 00000100:00100000:3.0:1780298646.366817:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f755c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609788288:12345-192.168.203.25@tcp:22:fsx.0 Request processed in 173us (343us total) trans 4294969346 rc 0/0 00000100:00100000:3.0:1780298646.366822:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6603 00000100:00100000:1.0:1780298646.367907:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.367937:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.367954:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609788416 00000100:00100000:0.0:1780298646.367959:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609788416 00000100:00100000:3.0:1780298646.367981:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609788544 00000100:00100000:3.0:1780298646.367987:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609788544 00000100:00100000:0.0:1780298646.367999:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10816 00000100:00100000:1.0:1780298646.368012:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10817 00000100:00100000:0.0:1780298646.368013:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f757100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2398:x1866777609788416:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.368024:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298646.368027:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f754a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609788544:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.368036:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.368038:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008361787d/0xaf35132e63036ca0 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5a52 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.368047:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e63036ca7 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5a59 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.368087:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241a80a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605158528:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.368093:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244adad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605158656:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.368176:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12d80 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.368177:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e900 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.369034:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244adad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605158656:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298646.369042:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241a80a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605158528:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.369063:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008361787d/0xaf35132e63036ca0 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a52 expref: 22 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.369069:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e63036ca7 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a59 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.369093:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e63036ca7 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a59 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.369093:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008361787d/0xaf35132e63036ca0 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a52 expref: 22 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.369122:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279e00, rc 0) 00010000:00010000:0.0:1780298646.369122:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2800, rc 0) 00000100:00100000:0.0:1780298646.369190:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f757100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2398:x1866777609788416:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1183us (1284us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.369199:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10816 00000100:00100000:2.0:1780298646.369208:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f754a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609788544:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1186us (1269us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298646.369217:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10817 00000100:00100000:2.0:1780298646.370293:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.370316:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.370331:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609788672 00000100:00100000:0.0:1780298646.370338:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609788672 00000100:00100000:3.0:1780298646.370340:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609788800 00000100:00100000:3.0:1780298646.370346:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609788800 00000100:00100000:0.0:1780298646.370373:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10818 00000100:00100000:0.0:1780298646.370386:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2399:x1866777609788672:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298646.370387:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10819 00010000:00010000:0.0:1780298646.370399:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298646.370403:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609788800:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.370411:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e63036cae lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5a60 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.370412:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298646.370423:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63036cb5 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5a67 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.370449:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf244adad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605158784:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298646.370460:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433dce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605158912:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.370502:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12d80 going to sleep for 11 seconds 00000100:00100000:3.0:1780298646.370503:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2418563c0 going to sleep for 11 seconds 00000100:00100000:3.0:1780298646.371293:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433dce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:19992:1866777605158912:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.371312:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf244adad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605158784:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298646.371314:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63036cb5 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a67 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.371330:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e63036cae lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a60 expref: 22 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.371338:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63036cb5 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a67 expref: 10 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.371345:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c0d26bbc/0xaf35132e63036cae lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a60 expref: 22 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.371356:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93000, rc 0) 00010000:00010000:0.0:1780298646.371358:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3c00, rc 0) 00000100:00100000:0.0:1780298646.371392:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2399:x1866777609788672:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1008us (1098us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298646.371393:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2400:x1866777609788800:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 991us (1071us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.371405:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10818 00000100:00100000:3.0:1780298646.371413:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10819 00000100:00100000:1.0:1780298646.372560:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.372585:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.372596:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609788928 00000100:00100000:2.0:1780298646.372602:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609788928 00000100:00100000:0.0:1780298646.372627:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609789056 00000100:00100000:0.0:1780298646.372631:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609789056 00000100:00100000:2.0:1780298646.372647:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10820 00000100:00100000:2.0:1780298646.372658:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a81880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2397:x1866777609788928:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298646.372671:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10821 00010000:00010000:2.0:1780298646.372677:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298646.372681:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a81f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609789056:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.372689:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63036cbc lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5a6e expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.372691:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.372704:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005b92bd2b/0xaf35132e63036cc3 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5a75 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.372722:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605159040:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.372742:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241a82680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605159168:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.372765:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.372776:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e900 going to sleep for 11 seconds 00000100:00100000:3.0:1780298646.373520:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8b1880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605159040:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298646.373548:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63036cbc lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a6e expref: 24 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.373562:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241a82680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605159168:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298646.373565:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ad103424/0xaf35132e63036cbc lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a6e expref: 24 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.373578:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9aa00, rc 0) 00010000:00010000:1.0:1780298646.373586:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005b92bd2b/0xaf35132e63036cc3 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a75 expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.373601:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005b92bd2b/0xaf35132e63036cc3 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a75 expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.373613:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a81880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2397:x1866777609788928:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 956us (1052us total) trans 0 rc 301/301 00010000:00010000:1.0:1780298646.373621:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279c00, rc 0) 00000100:00100000:3.0:1780298646.373622:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10820 00000100:00100000:1.0:1780298646.373649:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a81f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609789056:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 971us (1061us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.373656:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10821 00000100:00100000:0.0:1780298646.374590:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.374624:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.374636:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609789184 00000100:00100000:1.0:1780298646.374643:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609789184 00000100:00100000:2.0:1780298646.374666:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609789312 00000100:00100000:2.0:1780298646.374680:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609789312 00000100:00100000:1.0:1780298646.374680:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10822 00000100:00100000:1.0:1780298646.374693:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2399:x1866777609789184:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.374703:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298646.374715:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10823 00010000:00010000:1.0:1780298646.374715:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63036cca lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5a7c expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.374727:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609789312:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.374736:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298646.374747:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c1d9addf/0xaf35132e63036cd1 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5a83 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.374754:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241a82680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605159296:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.374788:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605159424:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.374815:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e900 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.374819:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.375636:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241a82680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605159296:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298646.375658:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63036cca lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a7c expref: 24 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.375671:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8b0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605159424:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298646.375672:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63036cca lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a7c expref: 24 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.375685:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279a00, rc 0) 00010000:00010000:0.0:1780298646.375701:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c1d9addf/0xaf35132e63036cd1 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a83 expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.375718:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2399:x1866777609789184:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1027us (1138us total) trans 0 rc 301/301 00010000:00010000:0.0:1780298646.375721:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c1d9addf/0xaf35132e63036cd1 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a83 expref: 12 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.375732:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10822 00010000:00010000:0.0:1780298646.375738:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9a600, rc 0) 00000100:00100000:0.0:1780298646.375764:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609789312:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1039us (1140us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.375771:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10823 00000100:00100000:3.0:1780298646.376839:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.376890:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609789440 00000100:00100000:2.0:1780298646.376897:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609789440 00000100:00100000:2.0:1780298646.376945:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10824 00000100:00100000:2.0:1780298646.376956:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2397:x1866777609789440:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.376965:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298646.376975:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00010000:00010000:2.0:1780298646.376976:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e63036cd8 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5a8a expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.377009:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609789568 00000100:00100000:1.0:1780298646.377014:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609789568 00000100:00100000:2.0:1780298646.377026:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf37f8b2300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605159552:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.377042:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10825 00000100:00100000:1.0:1780298646.377051:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609789568:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.377060:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.377069:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e63036cdf lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5a91 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.377093:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.377098:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241a82680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605159680:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.377142:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e900 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.377837:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf37f8b2300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605159552:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.377851:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241a82680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605159680:192.168.203.25@tcp:106: 00010000:00010000:2.0:1780298646.377861:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e63036cd8 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a8a expref: 24 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.377874:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e63036cdf lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a91 expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.377880:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000019780e8d/0xaf35132e63036cd8 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a8a expref: 24 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.377904:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9a200, rc 0) 00010000:00010000:1.0:1780298646.377912:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e63036cdf lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a91 expref: 12 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.377923:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279e00, rc 0) 00000100:00100000:2.0:1780298646.377937:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2397:x1866777609789440:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 984us (1100us total) trans 0 rc 301/301 00000100:00100000:2.0:1780298646.377945:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10824 00000100:00100000:1.0:1780298646.377950:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2398:x1866777609789568:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 899us (974us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.377958:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10825 00000100:00100000:0.0:1780298646.378931:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.378957:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.378973:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609789696 00000100:00100000:1.0:1780298646.378978:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609789696 00000100:00100000:3.0:1780298646.378984:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609789824 00000100:00100000:3.0:1780298646.378989:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609789824 00000100:00100000:1.0:1780298646.379011:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10826 00000100:00100000:1.0:1780298646.379020:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2399:x1866777609789696:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298646.379029:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10827 00010000:00010000:1.0:1780298646.379030:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1780298646.379043:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244ad9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609789824:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.379043:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63036ce6 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5a98 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.379057:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298646.379068:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f8ed852/0xaf35132e63036ced lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5a9f expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.379082:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241a82680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605159808:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298646.379105:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433dca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605159936:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.379148:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e900 going to sleep for 11 seconds 00000100:00100000:3.0:1780298646.379180:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2418563c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.379823:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241a82680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20252:1866777605159808:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298646.379843:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63036ce6 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a98 expref: 24 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.379860:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005fc17034/0xaf35132e63036ce6 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a98 expref: 24 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.379876:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278a00, rc 0) 00000100:00100000:1.0:1780298646.379914:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2399:x1866777609789696:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 896us (983us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.379925:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10826 00000100:00100000:3.0:1780298646.379962:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433dca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605159936:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298646.379975:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002f8ed852/0xaf35132e63036ced lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a9f expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.380041:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f8ed852/0xaf35132e63036ced lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a9f expref: 12 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.380048:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e92c00, rc 0) 00000100:00100000:3.0:1780298646.380070:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244ad9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2400:x1866777609789824:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1033us (1113us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298646.380075:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10827 00000100:00100000:2.0:1780298646.381028:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.381111:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609789952 00000100:00100000:3.0:1780298646.381119:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609789952 00000100:00100000:3.0:1780298646.381202:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10828 00000100:00100000:3.0:1780298646.381214:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2397:x1866777609789952:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298646.381227:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298646.381240:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000186136f8/0xaf35132e63036cf4 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5aa6 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.381292:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.381312:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433dca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605160064:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.381338:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609790080 00000100:00100000:1.0:1780298646.381342:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609790080 00000100:00100000:1.0:1780298646.381374:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10829 00000100:00100000:1.0:1780298646.381384:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf244adb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609790080:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.381394:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.381404:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e63036cfb lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5aad expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.381407:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2418563c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.381432:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241a82680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605160192:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.381460:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e900 going to sleep for 11 seconds 00000100:00100000:3.0:1780298646.382105:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433dca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605160064:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298646.382122:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000186136f8/0xaf35132e63036cf4 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5aa6 expref: 22 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.382157:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000186136f8/0xaf35132e63036cf4 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5aa6 expref: 22 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.382164:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93000, rc 0) 00000100:00100000:1.0:1780298646.382207:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241a82680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605160192:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298646.382220:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2397:x1866777609789952:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1009us (1194us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298646.382228:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10828 00010000:00010000:1.0:1780298646.382228:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e63036cfb lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5aad expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.382243:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef7bcba0/0xaf35132e63036cfb lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5aad expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.382257:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279a00, rc 0) 00000100:00100000:1.0:1780298646.382294:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf244adb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:2398:x1866777609790080:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 914us (1007us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.382299:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10829 00000100:00100000:2.0:1780298646.383219:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.383279:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609790208 00000100:00100000:0.0:1780298646.383285:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609790208 00000100:00100000:0.0:1780298646.383316:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10830 00000100:00100000:0.0:1780298646.383324:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+9:29360:x1866777609790208:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.383331:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.383338:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0cae770/0xaf35132e63036d02 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5ab4 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.383355:0:12101:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e63036c99 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2195456->35749887] (req 2195456->2207743) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a4b expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.383361:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x280000401:0x12:0x0].0x0 time=1780298646 v=(70 122) 00010000:00010000:0.0:1780298646.383365:0:12101:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298646 v=(71 122) 00010000:00010000:0.0:1780298646.383368:0:12101:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0cae770/0xaf35132e63036d02 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3743744->3747839] (req 3743744->3747839) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5ab4 expref: 10 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.383384:0:12101:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e63036c99 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2195456->35749887] (req 2195456->2207743) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a4b expref: 10 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.383393:0:12101:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e63036c99 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2195456->35749887] (req 2195456->2207743) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a4b expref: 10 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.383400:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605160320:192.168.203.25@tcp:104: 00000100:00100000:0.0:1780298646.383434:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12d80 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.384010:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605160320:192.168.203.25@tcp:104: 00010000:00010000:2.0:1780298646.384023:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e0cae770/0xaf35132e63036d02 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3743744->3747839] (req 3743744->3747839) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5ab4 expref: 10 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.384034:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2600, rc 0) 00000100:00100000:2.0:1780298646.384064:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:29360:x1866777609790208:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 742us (846us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298646.384071:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10830 00000100:00100000:2.0:1780298646.384179:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.384217:0:6461:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609790336 00000100:00100000:1.0:1780298646.384222:0:6461:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609790336 00010000:00010000:1.0:1780298646.384241:0:6461:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e63036c99 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [2195456->35749887] (req 2195456->2207743) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a4b expref: 11 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.384279:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15369 00000100:00100000:1.0:1780298646.384287:0:6461:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609790336:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298646.384298:0:6461:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1780298646.384311:0:6461:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e63036c99 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2195456->35749887] (req 2195456->2207743) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a4b expref: 11 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.384323:0:6461:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e63036c99 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2195456->35749887] (req 2195456->2207743) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a4b expref: 11 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.384340:0:6461:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005fd24608/0xaf35132e63036c99 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [2195456->35749887] (req 2195456->2207743) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5a4b expref: 11 pid: 9128 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.384351:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298646 v=(71 122) 00010000:00010000:1.0:1780298646.384355:0:6461:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298646 v=(71 122) 00010000:00010000:1.0:1780298646.384363:0:6461:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000e0cae770/0xaf35132e63036d02 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3743744->37298175] (req 3743744->3747839) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5ab4 expref: 9 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.384372:0:6461:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0cae770/0xaf35132e63036d02 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3743744->37298175] (req 3743744->3747839) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5ab4 expref: 9 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.384400:0:6461:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000e0cae770/0xaf35132e63036d02 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3743744->37298175] (req 3743744->3747839) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5ab4 expref: 9 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.384410:0:6461:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241a82680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6461:1866777605160448:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298646.384445:0:6461:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e900 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.385163:0:6461:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241a82680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:6461:1866777605160448:192.168.203.25@tcp:105: 00010000:00010000:2.0:1780298646.385178:0:6461:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1780298646.385203:0:6461:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609790336:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 917us (1026us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298646.385210:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15369 00000100:00100000:0.0:1780298646.385477:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.385540:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609790464 00000100:00100000:3.0:1780298646.385544:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609790464 00000100:00100000:3.0:1780298646.385570:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10831 00000100:00100000:3.0:1780298646.385579:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243060000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2400:x1866777609790464:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298646.385588:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298646.385600:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000cd0af3fe/0xaf35132e63036d09 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5abb expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.385612:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.385634:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433dca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605160576:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.385647:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609790592 00000100:00100000:0.0:1780298646.385651:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609790592 00000100:00100000:3.0:1780298646.385673:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2418563c0 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.385685:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10832 00000100:00100000:0.0:1780298646.385692:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a82a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609790592:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.385698:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.385706:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c703984a/0xaf35132e63036d10 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5ac2 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.385732:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605160704:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.385765:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12300 going to sleep for 11 seconds 00000100:00100000:3.0:1780298646.386414:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433dca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605160576:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298646.386434:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000cd0af3fe/0xaf35132e63036d09 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5abb expref: 22 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.386448:0:8359:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000cd0af3fe/0xaf35132e63036d09 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5abb expref: 22 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.386456:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93400, rc 0) 00000100:00100000:3.0:1780298646.386478:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243060000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2400:x1866777609790464:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 901us (1002us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298646.386484:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10831 00000100:00100000:0.0:1780298646.386522:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605160704:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.386540:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c703984a/0xaf35132e63036d10 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5ac2 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.386550:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c703984a/0xaf35132e63036d10 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5ac2 expref: 11 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.386571:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2000, rc 0) 00000100:00100000:0.0:1780298646.386601:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a82a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609790592:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 910us (992us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.386608:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10832 00000100:00100000:2.0:1780298646.387498:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.387523:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609790720 00000100:00100000:3.0:1780298646.387528:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609790720 00002000:00100000:3.0:1780298646.387552:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x12:0x0] (3743744->3747839) req@ffff9cf37f8b1880 x1866777609790720/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:352/0 lens 488/0 e 0 to 0 dl 1780298657 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298646.387559:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f8b1880 with x1866777609790720 ext(3743744->3747839) 00002000:00010000:3.0:1780298646.387562:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f8b1880 00000100:00100000:3.0:1780298646.387579:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6604 00000100:00100000:3.0:1780298646.387584:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf37f8b1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609790720:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298646.387704:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf37f8b1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2398:x1866777609790720:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 120us (210us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298646.387708:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6604 00002000:00100000:3.0:1780298646.387712:0:16972:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x280000401:0x12:0x0] (3743744->3747839) req@ffff9cf37f8b1880 x1866777609790720/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:352/0 lens 488/440 e 0 to 0 dl 1780298657 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298646.387722:0:16972:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf37f8b1880 with x1866777609790720 ext(3743744->3747839) 00002000:00010000:3.0:1780298646.387724:0:16972:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf37f8b1880 00000100:00100000:1.0:1780298646.388768:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.388804:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.388831:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609790848 00000100:00100000:2.0:1780298646.388836:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609790848 00000100:00100000:0.0:1780298646.388845:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609790976 00000100:00100000:0.0:1780298646.388851:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609790976 00000100:00100000:2.0:1780298646.388871:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10833 00000100:00100000:2.0:1780298646.388879:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a82d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2400:x1866777609790848:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.388886:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298646.388888:0:20252:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10834 00000100:00100000:0.0:1780298646.388898:0:20252:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a82680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609790976:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.388899:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e63036d17 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5ac9 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.388910:0:20252:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.388919:0:20252:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008361787d/0xaf35132e63036d1e lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5ad0 expref: -99 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.388942:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605160832:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.388947:0:20252:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605160960:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.389009:0:20252:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12300 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.389010:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.389841:0:20252:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20252:1866777605160960:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.389842:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372dc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8358:1866777605160832:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298646.389897:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e63036d17 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5ac9 expref: 22 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.389898:0:20252:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008361787d/0xaf35132e63036d1e lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5ad0 expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.389940:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c1d9addf/0xaf35132e63036d17 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5ac9 expref: 22 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.389945:0:20252:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008361787d/0xaf35132e63036d1e lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5ad0 expref: 11 pid: 20252 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.389967:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9a600, rc 0) 00010000:00010000:0.0:1780298646.389972:0:20252:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2800, rc 0) 00000100:00100000:1.0:1780298646.390016:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a82d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:2400:x1866777609790848:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1138us (1254us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.390016:0:20252:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a82680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:2399:x1866777609790976:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1119us (1208us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.390035:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10833 00000100:00100000:0.0:1780298646.390044:0:20252:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10834 00000100:00100000:3.0:1780298646.391206:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.391245:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609791104 00000100:00100000:1.0:1780298646.391252:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609791104 00000100:00100000:1.0:1780298646.391301:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10835 00000100:00100000:1.0:1780298646.391309:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e427850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+10:29360:x1866777609791104:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.391325:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.391337:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e63036d25 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5ad7 expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.391356:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298646 v=(71 122) 00010000:00010000:1.0:1780298646.391360:0:19992:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x12:0x0].0x0 time=1780298646 v=(71 122) 00010000:00010000:1.0:1780298646.391367:0:19992:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e63036d25 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3739648->3743743] (req 3739648->3743743) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5ad7 expref: 11 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.391379:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009aa43742/0xaf35132e63036d25 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x12:0x0].0x0 rrc: 3 type: EXT [3739648->3743743] (req 3739648->3743743) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5ad7 expref: 11 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.391384:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279e00, rc 0) 00000100:00100000:1.0:1780298646.391416:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e427850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:29360:x1866777609791104:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 108us (215us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298646.391421:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10835 00000100:00100000:2.0:1780298646.392349:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.392398:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609791232 00000100:00100000:0.0:1780298646.392405:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609791232 00002000:00100000:0.0:1780298646.392439:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x12:0x0] (3739648->3743743) req@ffff9cf34372f100 x1866777609791232/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:352/0 lens 488/0 e 0 to 0 dl 1780298657 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298646.392451:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf34372f100 with x1866777609791232 ext(3739648->3743743) 00002000:00010000:0.0:1780298646.392456:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf34372f100 00000100:00100000:0.0:1780298646.392481:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6605 00000100:00100000:0.0:1780298646.392491:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609791232:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:0.0:1780298646.392623:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2400:x1866777609791232:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 135us (275us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298646.392631:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6605 00002000:00100000:0.0:1780298646.392638:0:8363:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x280000401:0x12:0x0] (3739648->3743743) req@ffff9cf34372f100 x1866777609791232/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:352/0 lens 488/440 e 0 to 0 dl 1780298657 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298646.392653:0:8363:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf34372f100 with x1866777609791232 ext(3739648->3743743) 00002000:00010000:0.0:1780298646.392657:0:8363:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf34372f100 00000100:00100000:2.0:1780298646.394000:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.394040:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609791360 00000100:00100000:0.0:1780298646.394047:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609791360 00002000:00100000:0.0:1780298646.394073:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (3739648->3747839) req@ffff9cf34372f480 x1866777609791360/t0(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:352/0 lens 8680/0 e 0 to 0 dl 1780298657 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298646.394084:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf34372f480 with x1866777609791360 ext(3739648->3747839) 00002000:00010000:0.0:1780298646.394093:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf34372f480 00000100:00100000:0.0:1780298646.394118:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6606 00000100:00100000:0.0:1780298646.394149:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+11:2398:x1866777609791360:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:0.0:1780298646.394660:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2398:x1866777609791360:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 535us (660us total) trans 4294969347 rc 0/0 00000100:00100000:0.0:1780298646.394669:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6606 00002000:00100000:0.0:1780298646.394677:0:8362:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x280000401:0x12:0x0] (3739648->3747839) req@ffff9cf34372f480 x1866777609791360/t4294969347(0) o4->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:352/0 lens 8680/448 e 0 to 0 dl 1780298657 ref 1 fl Complete:/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298646.394686:0:8362:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf34372f480 with x1866777609791360 ext(3739648->3747839) 00002000:00010000:0.0:1780298646.394693:0:8362:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff9cf34372f480 00000100:00100000:3.0:1780298646.395527:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.395569:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609791488 00000100:00100000:0.0:1780298646.395574:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609791488 00000100:00100000:0.0:1780298646.395609:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10836 00000100:00100000:0.0:1780298646.395618:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+13:2398:x1866777609791488:12345-192.168.203.25@tcp:16:fsx.0 00000001:00080000:1.0:1780298646.396531:0:8351:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0000: transno 4294969347 is committed 00000001:00080000:1.0:1780298646.396547:0:8351:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0000: transno 4294969346 is committed 00000100:00100000:0.0:1780298646.396669:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609791488:12345-192.168.203.25@tcp:16:fsx.0 Request processed in 1049us (1141us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298646.396679:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10836 00000100:00100000:2.0:1780298646.398451:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.398491:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609791616 00000100:00100000:1.0:1780298646.398500:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609791616 00000100:00100000:1.0:1780298646.398543:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10837 00000100:00100000:3.0:1780298646.398554:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.398563:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2397:x1866777609791616:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.398577:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1780298646.398587:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609791744 00010000:00010000:1.0:1780298646.398592:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63036d2c lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5ade expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.398598:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609791744 00000100:00100000:2.0:1780298646.398653:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10838 00000100:00100000:1.0:1780298646.398658:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241a81880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605161088:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.398669:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609791744:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.398681:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1780298646.398695:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b4561703/0xaf35132e63036d33 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5ae5 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.398741:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:2.0:1780298646.398758:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf34372d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605161216:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.398808:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.399700:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241a81880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605161088:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298646.399733:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63036d2c lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5ade expref: 24 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.399752:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ef7bcba0/0xaf35132e63036d2c lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5ade expref: 24 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.399767:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279a00, rc 0) 00002000:00010000:3.0:1780298646.399788:0:8358:0:(ofd_lvb.c:259:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb size: 2765177 -> 3747840 00002000:00010000:3.0:1780298646.399795:0:8358:0:(ofd_lvb.c:283:ofd_lvbo_update()) res: [0x280000401:0x12:0x0] updating lvb blocks: 64 -> 80 00000100:00100000:3.0:1780298646.399813:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf34372d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605161216:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.399813:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2397:x1866777609791616:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1253us (1362us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.399838:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10837 00010000:00010000:3.0:1780298646.399858:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b4561703/0xaf35132e63036d33 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5ae5 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.399879:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b4561703/0xaf35132e63036d33 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5ae5 expref: 10 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.399891:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9b600, rc 0) 00000100:00100000:3.0:1780298646.399965:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609791744:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1265us (1383us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298646.399974:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10838 00000100:00100000:2.0:1780298646.401114:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.401183:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609791872 00000100:00100000:1.0:1780298646.401190:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609791872 00000100:00100000:1.0:1780298646.401232:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10839 00000100:00100000:2.0:1780298646.401248:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.401252:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2399:x1866777609791872:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.401277:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.401291:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63036d3a lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5aec expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.401325:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609792000 00000100:00100000:0.0:1780298646.401332:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609792000 00000100:00100000:1.0:1780298646.401337:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241a81880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605161344:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.401368:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10840 00000100:00100000:0.0:1780298646.401375:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf34372dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609792000:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.401384:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.401399:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e2cfab1/0xaf35132e63036d41 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5af3 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.401409:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.401432:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605161472:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.401464:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12300 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.402321:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241a81880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605161344:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298646.402347:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63036d3a lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5aec expref: 24 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.402367:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63036d3a lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5aec expref: 24 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.402380:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278c00, rc 0) 00000100:00100000:1.0:1780298646.402419:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2399:x1866777609791872:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1177us (1304us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.402431:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10839 00000100:00100000:0.0:1780298646.402440:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605161472:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.402464:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003e2cfab1/0xaf35132e63036d41 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5af3 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.402477:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e2cfab1/0xaf35132e63036d41 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5af3 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.402486:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2e00, rc 0) 00000100:00100000:0.0:1780298646.402510:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf34372dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2400:x1866777609792000:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1135us (1262us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.402517:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10840 00000100:00100000:2.0:1780298646.403920:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.403956:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609792128 00000100:00100000:3.0:1780298646.403963:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609792128 00000100:00100000:3.0:1780298646.404003:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10841 00000100:00100000:3.0:1780298646.404012:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5dad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+21:29360:x1866777609792128:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:3.0:1780298646.404025:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298646.404035:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000cd0af3fe/0xaf35132e63036d48 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5afa expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.404059:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000001d4afe0b/0xaf35132e63036c4c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [4222976->5492735] (req 4222976->4227071) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d59fe expref: 22 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.404069:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 1 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(93 169) 00010000:00010000:3.0:1780298646.404073:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(94 169) 00010000:00010000:3.0:1780298646.404078:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000cd0af3fe/0xaf35132e63036d48 lrc: 4/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [4866048->4878335] (req 4866048->4878335) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5afa expref: 22 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.404104:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000001d4afe0b/0xaf35132e63036c4c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [4222976->5492735] (req 4222976->4227071) gid 0 flags: 0x50000000020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d59fe expref: 22 pid: 20252 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.404119:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000001d4afe0b/0xaf35132e63036c4c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [4222976->5492735] (req 4222976->4227071) gid 0 flags: 0x70000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d59fe expref: 22 pid: 20252 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.404165:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433dd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605161600:192.168.203.25@tcp:104: 00000100:00100000:3.0:1780298646.404228:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf2418563c0 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.404882:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433dd880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:8359:1866777605161600:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298646.404897:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000cd0af3fe/0xaf35132e63036d48 lrc: 3/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [4866048->4878335] (req 4866048->4878335) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5afa expref: 22 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.404906:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93400, rc 0) 00000100:00100000:1.0:1780298646.404930:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5dad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:29360:x1866777609792128:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 921us (1014us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298646.404935:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10841 00000100:00100000:2.0:1780298646.405255:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.405315:0:19949:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609792256 00000100:00100000:0.0:1780298646.405324:0:19949:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609792256 00002000:00100000:0.0:1780298646.405359:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (4222976->4226245) req@ffff9cf36e5d8e00 x1866777609792256/t0(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:352/0 lens 3760/0 e 0 to 0 dl 1780298657 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298646.405370:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e5d8e00 with x1866777609792256 ext(4222976->4226245) 00010000:00010000:0.0:1780298646.405379:0:19949:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000001d4afe0b/0xaf35132e63036c4c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [4222976->5492735] (req 4222976->4227071) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d59fe expref: 23 pid: 20252 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.405396:0:19949:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000001d4afe0b/0xaf35132e63036c4c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [4222976->5492735] (req 4222976->4227071) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d59fe expref: 23 pid: 20252 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1780298646.405405:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf36e5d8e00 00000100:00100000:0.0:1780298646.405433:0:19949:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 227 00000100:00100000:0.0:1780298646.405442:0:19949:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+23:2399:x1866777609792256:12345-192.168.203.25@tcp:4:fsx.0 00000100:00100000:0.0:1780298646.405995:0:19949:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+25:2399:x1866777609792256:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 554us (739us total) trans 4294969828 rc 0/0 00000100:00100000:0.0:1780298646.406004:0:19949:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 227 00002000:00100000:0.0:1780298646.406013:0:19949:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x2c0000401:0x11:0x0] (4222976->4226245) req@ffff9cf36e5d8e00 x1866777609792256/t4294969828(0) o4->e30cf99f-4c41-4c6b-8649-e9856bf0abcc@192.168.203.25@tcp:352/0 lens 3760/448 e 0 to 0 dl 1780298657 ref 1 fl Complete:H/600/0 rc 0/0 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1780298646.406023:0:19949:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf36e5d8e00 with x1866777609792256 ext(4222976->4226245) 00010000:00010000:0.0:1780298646.406030:0:19949:0:(ldlm_extent.c:839:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000001d4afe0b/0xaf35132e63036c4c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [4222976->5492735] (req 4222976->4227071) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d59fe expref: 25 pid: 20252 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.406050:0:19949:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000001d4afe0b/0xaf35132e63036c4c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [4222976->5492735] (req 4222976->4227071) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d59fe expref: 25 pid: 20252 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1780298646.406059:0:19949:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff9cf36e5d8e00 00000100:00100000:2.0:1780298646.406877:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.406914:0:6463:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609792384 00000100:00100000:1.0:1780298646.406919:0:6463:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609792384 00010000:00010000:1.0:1780298646.406940:0:6463:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000001d4afe0b/0xaf35132e63036c4c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [4222976->5492735] (req 4222976->4227071) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d59fe expref: 25 pid: 20252 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.406970:0:6463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15370 00000100:00100000:1.0:1780298646.406979:0:6463:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf348b1c050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+25:2398:x1866777609792384:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1780298646.406990:0:6463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:1.0:1780298646.407001:0:6463:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x11:0x0] updating lvb blocks from disk: 560 -> 568 00010000:00010000:1.0:1780298646.407005:0:6463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000001d4afe0b/0xaf35132e63036c4c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [4222976->5492735] (req 4222976->4227071) gid 0 flags: 0x60000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d59fe expref: 25 pid: 20252 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.407014:0:6463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000001d4afe0b/0xaf35132e63036c4c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [4222976->5492735] (req 4222976->4227071) gid 0 flags: 0x50000400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d59fe expref: 25 pid: 20252 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.407028:0:6463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001d4afe0b/0xaf35132e63036c4c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 7 type: EXT [4222976->5492735] (req 4222976->4227071) gid 0 flags: 0x44801400020020 nid: 192.168.203.25@tcp remote: 0x11c228eb194d59fe expref: 25 pid: 20252 timeout: 935 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.407037:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(94 169) 00010000:00010000:1.0:1780298646.407040:0:6463:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x11:0x0].0x0 time=1780298646 v=(94 169) 00010000:00010000:1.0:1780298646.407046:0:6463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000cd0af3fe/0xaf35132e63036d48 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [4866048->5492735] (req 4866048->4878335) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5afa expref: 21 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.407052:0:6463:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000cd0af3fe/0xaf35132e63036d48 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [4866048->5492735] (req 4866048->4878335) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5afa expref: 21 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.407073:0:6463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000cd0af3fe/0xaf35132e63036d48 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [4866048->5492735] (req 4866048->4878335) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5afa expref: 21 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.407084:0:6463:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241a81880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6463:1866777605161728:192.168.203.25@tcp:105: 00000100:00100000:1.0:1780298646.407158:0:6463:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306ec00 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.407846:0:6463:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241a81880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:6463:1866777605161728:192.168.203.25@tcp:105: 00010000:00010000:1.0:1780298646.407856:0:6463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1780298646.407878:0:6463:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf348b1c050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+24:2398:x1866777609792384:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 901us (1004us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298646.407885:0:6463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15370 00000100:00100000:2.0:1780298646.408751:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.408781:0:8358:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609792512 00000100:00100000:3.0:1780298646.408786:0:8358:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609792512 00000100:00100000:2.0:1780298646.408794:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.408833:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609792640 00000100:00100000:2.0:1780298646.408837:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609792640 00000100:00100000:0.0:1780298646.408853:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10842 00000100:00100000:3.0:1780298646.408881:0:8358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10843 00000100:00100000:0.0:1780298646.408882:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2400:x1866777609792512:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:3.0:1780298646.408897:0:8358:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609792640:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.408900:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1780298646.408909:0:8358:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.408920:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e63036d4f lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5b01 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.408922:0:8358:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63036d56 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5b08 expref: -99 pid: 8358 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.408998:0:8358:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf2433ded80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605161856:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.409020:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605161984:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298646.409046:0:8358:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241856000 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.409094:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12300 going to sleep for 11 seconds 00000100:00100000:3.0:1780298646.409889:0:8358:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf2433ded80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8358:1866777605161856:192.168.203.25@tcp:106: 00010000:00010000:3.0:1780298646.409916:0:8358:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63036d56 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5b08 expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.409931:0:8358:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000186136f8/0xaf35132e63036d56 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5b08 expref: 9 pid: 8358 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.409940:0:8358:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf375e93000, rc 0) 00000100:00100000:0.0:1780298646.409948:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:12101:1866777605161984:192.168.203.25@tcp:106: 00000100:00100000:3.0:1780298646.409971:0:8358:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609792640:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1079us (1176us total) trans 0 rc 301/301 00010000:00010000:0.0:1780298646.409978:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e63036d4f lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5b01 expref: 23 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.409983:0:8358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10843 00010000:00010000:0.0:1780298646.410006:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f2b1eebc/0xaf35132e63036d4f lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5b01 expref: 23 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.410019:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff3800, rc 0) 00000100:00100000:0.0:1780298646.410058:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2400:x1866777609792512:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1191us (1303us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.410066:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10842 00000100:00100000:2.0:1780298646.411070:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.411107:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609792768 00000100:00100000:0.0:1780298646.411114:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609792768 00000100:00100000:0.0:1780298646.411184:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10844 00000100:00100000:2.0:1780298646.411193:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.411207:0:9128:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609792896 00000100:00100000:0.0:1780298646.411207:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5d9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2400:x1866777609792768:12345-192.168.203.25@tcp:101:fsx.0 00000100:00100000:1.0:1780298646.411216:0:9128:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609792896 00010000:00010000:0.0:1780298646.411224:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.411238:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e63036d5d lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5b0f expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.411257:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10845 00000100:00100000:1.0:1780298646.411282:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf36e5da300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609792896:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.411292:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298646.411303:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605162112:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298646.411305:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63036d64 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5b16 expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.411358:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241a81880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605162240:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.411362:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12300 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.411400:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e540 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.412164:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:8356:1866777605162112:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.412217:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e63036d5d lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5b0f expref: 23 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.412235:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003e2cfab1/0xaf35132e63036d5d lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5b0f expref: 23 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.412236:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241a81880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:9128:1866777605162240:192.168.203.25@tcp:106: 00010000:00010000:0.0:1780298646.412261:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2e00, rc 0) 00010000:00010000:1.0:1780298646.412299:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63036d64 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5b16 expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.412306:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5d9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2400:x1866777609792768:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1103us (1232us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.412332:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10844 00010000:00010000:1.0:1780298646.412352:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078761089/0xaf35132e63036d64 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5b16 expref: 9 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.412370:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279200, rc 0) 00000100:00100000:1.0:1780298646.412408:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf36e5da300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2399:x1866777609792896:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1131us (1222us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.412416:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10845 00000100:00100000:2.0:1780298646.413751:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.413788:0:19943:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609793024 00000100:00100000:3.0:1780298646.413793:0:19943:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609793024 00002000:00100000:3.0:1780298646.413821:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (3469312->3477503) req@ffff9cf255493480 x1866777609793024/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:352/0 lens 488/0 e 0 to 0 dl 1780298657 ref 1 fl New:/600/ffffffff rc 0/-1 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298646.413829:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255493480 with x1866777609793024 ext(3469312->3477503) 00002000:00010000:3.0:1780298646.413833:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255493480 00000100:00100000:3.0:1780298646.413850:0:19943:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6607 00000100:00100000:3.0:1780298646.413856:0:19943:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255493480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2398:x1866777609793024:12345-192.168.203.25@tcp:3:fsx.0 00000100:00100000:3.0:1780298646.414023:0:19943:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255493480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+22:2398:x1866777609793024:12345-192.168.203.25@tcp:3:fsx.0 Request processed in 166us (273us total) trans 0 rc 8192/8192 00000100:00100000:3.0:1780298646.414038:0:19943:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6607 00002000:00100000:3.0:1780298646.414046:0:19943:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x2c0000401:0x11:0x0] (3469312->3477503) req@ffff9cf255493480 x1866777609793024/t0(0) o3->a9f27b46-1e58-4b3a-bc39-449ecb1af8ca@192.168.203.25@tcp:352/0 lens 488/8632 e 0 to 0 dl 1780298657 ref 1 fl Complete:/600/0 rc 8192/8192 job:'fsx.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1780298646.414054:0:19943:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff9cf255493480 with x1866777609793024 ext(3469312->3477503) 00002000:00010000:3.0:1780298646.414058:0:19943:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff9cf255493480 00000100:00100000:3.0:1780298646.415117:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.415187:0:19992:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609793152 00000100:00100000:1.0:1780298646.415193:0:19992:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609793152 00000100:00100000:1.0:1780298646.415237:0:19992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10846 00000100:00100000:2.0:1780298646.415253:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.415273:0:19992:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+24:2397:x1866777609793152:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.415283:0:19992:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1780298646.415292:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609793280 00010000:00010000:1.0:1780298646.415299:0:19992:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63036d6b lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5b1d expref: -99 pid: 19992 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.415301:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609793280 00000100:00100000:0.0:1780298646.415350:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10847 00000100:00100000:1.0:1780298646.415355:0:19992:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241a81880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605162368:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.415371:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255492300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609793280:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:0.0:1780298646.415384:0:12101:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1780298646.415399:0:12101:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c703984a/0xaf35132e63036d72 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5b24 expref: -99 pid: 12101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.415420:0:19992:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e540 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.415445:0:12101:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605162496:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.415491:0:12101:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12300 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.416204:0:19992:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241a81880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:19992:1866777605162368:192.168.203.25@tcp:106: 00000100:00100000:0.0:1780298646.416220:0:12101:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:12101:1866777605162496:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298646.416235:0:19992:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63036d6b lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5b1d expref: 25 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.416251:0:12101:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c703984a/0xaf35132e63036d72 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5b24 expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.416256:0:19992:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a6416acb/0xaf35132e63036d6b lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5b1d expref: 25 pid: 19992 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.416303:0:19992:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278c00, rc 0) 00010000:00010000:0.0:1780298646.416307:0:12101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c703984a/0xaf35132e63036d72 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5b24 expref: 10 pid: 12101 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.416324:0:12101:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf344ff2000, rc 0) 00000100:00100000:1.0:1780298646.416348:0:19992:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+24:2397:x1866777609793152:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1101us (1231us total) trans 0 rc 301/301 00000100:00100000:0.0:1780298646.416362:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255492300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609793280:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 995us (1105us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.416363:0:19992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10846 00000100:00100000:0.0:1780298646.416379:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10847 00000100:00100000:2.0:1780298646.417375:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.417408:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.417410:0:20252:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609793408 00000100:00100000:0.0:1780298646.417415:0:20252:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609793408 00000100:00100000:1.0:1780298646.417460:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609793536 00000100:00100000:1.0:1780298646.417466:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609793536 00000100:00100000:2.0:1780298646.417473:0:9128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10848 00000100:00100000:2.0:1780298646.417490:0:9128:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255490380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+24:2400:x1866777609793408:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:2.0:1780298646.417501:0:9128:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1780298646.417511:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10849 00010000:00010000:2.0:1780298646.417514:0:9128:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d71a6aba/0xaf35132e63036d79 lrc: 2/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5b2b expref: -99 pid: 9128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.417525:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255492a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609793536:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.417551:0:8356:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.417562:0:8356:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b25cdf87/0xaf35132e63036d80 lrc: 2/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5b32 expref: -99 pid: 8356 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1780298646.417583:0:9128:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf255493800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605162624:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.417592:0:8356:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241a81880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605162752:192.168.203.25@tcp:106: 00000100:00100000:2.0:1780298646.417676:0:9128:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf243e6c180 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.417677:0:8356:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e540 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.418375:0:8356:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241a81880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8356:1866777605162752:192.168.203.25@tcp:106: 00010000:00010000:1.0:1780298646.418391:0:8356:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b25cdf87/0xaf35132e63036d80 lrc: 1/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5b32 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.418408:0:8356:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b25cdf87/0xaf35132e63036d80 lrc: 0/0,0 mode: --/PR res: [0x280000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5b32 expref: 10 pid: 8356 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.418415:0:8356:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243278200, rc 0) 00000100:00100000:3.0:1780298646.418449:0:9128:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf255493800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:9128:1866777605162624:192.168.203.25@tcp:106: 00000100:00100000:1.0:1780298646.418449:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255492a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609793536:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 930us (1041us total) trans 0 rc 301/301 00000100:00100000:1.0:1780298646.418459:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10849 00010000:00010000:3.0:1780298646.418479:0:9128:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d71a6aba/0xaf35132e63036d79 lrc: 1/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5b2b expref: 25 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.418499:0:9128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d71a6aba/0xaf35132e63036d79 lrc: 0/0,0 mode: --/PR res: [0x2c0000401:0x11:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5b2b expref: 25 pid: 9128 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.418510:0:9128:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243d9bc00, rc 0) 00000100:00100000:3.0:1780298646.418534:0:9128:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255490380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+24:2400:x1866777609793408:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 1049us (1156us total) trans 0 rc 301/301 00000100:00100000:3.0:1780298646.418540:0:9128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10848 00000100:00100000:3.0:1780298646.419813:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.419855:0:8359:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609793664 00000100:00100000:1.0:1780298646.419861:0:8359:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609793664 00000100:00100000:1.0:1780298646.419894:0:8359:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 10850 00000100:00100000:1.0:1780298646.419902:0:8359:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:29360:x1866777609793664:12345-192.168.203.25@tcp:101:fsx.0 00010000:00010000:1.0:1780298646.419912:0:8359:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1780298646.419926:0:8359:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000017dab243/0xaf35132e63036d87 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x13:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x11c228eb194d5b39 expref: -99 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.419950:0:8359:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e0f99016/0xaf35132e6302c37a lrc: 2/0,0 mode: PR/PR res: [0x280000401:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194cbf72 expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.419956:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x13:0x0].0x0 time=1780298635 v=(0 0) 00010000:00010000:1.0:1780298646.419959:0:8359:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x13:0x0].0x0 time=1780298646 v=(0 0) 00010000:00010000:1.0:1780298646.419963:0:8359:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000017dab243/0xaf35132e63036d87 lrc: 4/0,0 mode: --/PW res: [0x280000401:0x13:0x0].0x0 rrc: 3 type: EXT [0->8191] (req 0->8191) gid 0 flags: 0x50000000000000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5b39 expref: 10 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.419979:0:8359:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e0f99016/0xaf35132e6302c37a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cbf72 expref: 10 pid: 8359 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.419989:0:8359:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e0f99016/0xaf35132e6302c37a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cbf72 expref: 10 pid: 8359 timeout: 935 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.419997:0:8359:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf241a81880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605162880:192.168.203.25@tcp:104: 00000100:00100000:1.0:1780298646.420042:0:8359:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf24306e540 going to sleep for 11 seconds 00000100:00100000:1.0:1780298646.420786:0:8359:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf241a81880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8359:1866777605162880:192.168.203.25@tcp:104: 00010000:00010000:1.0:1780298646.420804:0:8359:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000017dab243/0xaf35132e63036d87 lrc: 3/0,0 mode: --/PW res: [0x280000401:0x13:0x0].0x0 rrc: 3 type: EXT [0->8191] (req 0->8191) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5b39 expref: 10 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.420817:0:8359:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff9cf243279800, rc 0) 00000100:00100000:1.0:1780298646.420850:0:8359:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:29360:x1866777609793664:12345-192.168.203.25@tcp:101:fsx.0 Request processed in 949us (1037us total) trans 0 rc 0/0 00000100:00100000:1.0:1780298646.420857:0:8359:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 10850 00000100:00100000:3.0:1780298646.420997:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.421084:0:7400:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609793792 00000100:00100000:0.0:1780298646.421091:0:7400:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609793792 00010000:00010000:0.0:1780298646.421113:0:7400:0:(ldlm_lockd.c:2676:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e0f99016/0xaf35132e6302c37a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cbf72 expref: 10 pid: 8359 timeout: 935 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.421165:0:7400:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 15371 00000100:00100000:0.0:1780298646.421175:0:7400:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433dce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2399:x1866777609793792:12345-192.168.203.25@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1780298646.421186:0:7400:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1780298646.421200:0:7400:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e0f99016/0xaf35132e6302c37a lrc: 4/0,0 mode: PR/PR res: [0x280000401:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cbf72 expref: 10 pid: 8359 timeout: 935 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.421209:0:7400:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e0f99016/0xaf35132e6302c37a lrc: 3/0,0 mode: PR/PR res: [0x280000401:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cbf72 expref: 10 pid: 8359 timeout: 935 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.421229:0:7400:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0f99016/0xaf35132e6302c37a lrc: 0/0,0 mode: --/PR res: [0x280000401:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.25@tcp remote: 0x11c228eb194cbf72 expref: 10 pid: 8359 timeout: 935 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.421239:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x13:0x0].0x0 time=1780298646 v=(0 0) 00010000:00010000:0.0:1780298646.421242:0:7400:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x280000401:0x13:0x0].0x0 time=1780298646 v=(0 0) 00010000:00010000:0.0:1780298646.421248:0:7400:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000017dab243/0xaf35132e63036d87 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5b39 expref: 9 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.421255:0:7400:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000017dab243/0xaf35132e63036d87 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x40080000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5b39 expref: 9 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.421289:0:7400:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000017dab243/0xaf35132e63036d87 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->8191) gid 0 flags: 0x40000000020000 nid: 192.168.203.25@tcp remote: 0x11c228eb194d5b39 expref: 9 pid: 8359 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1780298646.421297:0:7400:0:(client.c:1904:ptlrpc_send_new_req()) Sending RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:7400:1866777605163008:192.168.203.25@tcp:105: 00000100:00100000:0.0:1780298646.421340:0:7400:0:(client.c:2691:ptlrpc_set_wait()) set ffff9cf241b12300 going to sleep for 11 seconds 00000100:00100000:0.0:1780298646.422190:0:7400:0:(client.c:2390:ptlrpc_check_set()) Completed RPC req@ffff9cf243061500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:7400:1866777605163008:192.168.203.25@tcp:105: 00010000:00010000:0.0:1780298646.422202:0:7400:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1780298646.422229:0:7400:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433dce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2399:x1866777609793792:12345-192.168.203.25@tcp:103:ldlm_bl.0 Request processed in 1058us (1235us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298646.422235:0:7400:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 15371 00000100:00100000:2.0:1780298646.437970:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.438033:0:16972:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609793920 00000100:00100000:3.0:1780298646.438045:0:16972:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609793920 00000100:00100000:3.0:1780298646.438120:0:16972:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6608 00000100:00100000:3.0:1780298646.438174:0:16972:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf255493800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+24:2399:x1866777609793920:12345-192.168.203.25@tcp:4:fsx.0 00002000:00010000:2.0:1780298646.438258:0:16972:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x2c0000401:0x12:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x0 00010000:00010000:2.0:1780298646.438278:0:16972:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63036d8e lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x12:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 16972 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.438292:0:16972:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x12:0x0].0x0 time=0 v=(0 0) 00010000:00010000:2.0:1780298646.438296:0:16972:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x12:0x0].0x0 time=1780298646 v=(0 0) 00010000:00010000:2.0:1780298646.438301:0:16972:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63036d8e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x12:0x0].0x0 rrc: 2 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 16972 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.438313:0:16972:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63036d8e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x12:0x0].0x0 rrc: 2 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 16972 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.439092:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:0.0:1780298646.439151:0:8363:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609794048 00000100:00100000:0.0:1780298646.439158:0:8363:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609794048 00000100:00100000:0.0:1780298646.439203:0:8363:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6609 00000100:00100000:0.0:1780298646.439211:0:8363:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+25:2399:x1866777609794048:12345-192.168.203.25@tcp:4:fsx.0 00010000:00010000:0.0:1780298646.439235:0:8363:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e63036d95 lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8363 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.439246:0:8363:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x12:0x0].0x0 time=1780298646 v=(0 0) 00010000:00010000:0.0:1780298646.439249:0:8363:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x12:0x0].0x0 time=1780298646 v=(0 0) 00010000:00010000:0.0:1780298646.439254:0:8363:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e63036d95 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x12:0x0].0x0 rrc: 3 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8363 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1780298646.439276:0:8363:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e63036d95 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x12:0x0].0x0 rrc: 3 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8363 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.439751:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:1.0:1780298646.439862:0:8362:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609794176 00000100:00100000:1.0:1780298646.439871:0:8362:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609794176 00000100:00100000:1.0:1780298646.439942:0:8362:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 6610 00000100:00100000:1.0:1780298646.439952:0:8362:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf2433de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+26:2399:x1866777609794176:12345-192.168.203.25@tcp:4:fsx.0 00010000:00010000:1.0:1780298646.439980:0:8362:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63036d9c lrc: 3/0,1 mode: --/PW res: [0x2c0000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8362 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.439992:0:8362:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x12:0x0].0x0 time=1780298646 v=(0 0) 00010000:00010000:1.0:1780298646.439996:0:8362:0:(ldlm_extent.c:456:ldlm_update_contention()) add 0 to [0x2c0000401:0x12:0x0].0x0 time=1780298646 v=(0 0) 00010000:00010000:1.0:1780298646.440000:0:8362:0:(ldlm_resource.c:1745:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63036d9c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x12:0x0].0x0 rrc: 4 type: EXT [8388608->9072639] (req 8388608->9072639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8362 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.440011:0:8362:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63036d9c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x12:0x0].0x0 rrc: 4 type: EXT [8388608->9072639] (req 8388608->9072639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8362 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1780298646.446621:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969829 is committed 00000001:00080000:3.0:1780298646.446651:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969826 is committed 00002000:00010000:1.0:1780298646.446789:0:8362:0:(ofd_lvb.c:305:ofd_lvbo_update()) res: [0x2c0000401:0x12:0x0] updating lvb size from disk: 0 -> 9068588 00002000:00010000:1.0:1780298646.446795:0:8362:0:(ofd_lvb.c:312:ofd_lvbo_update()) res: [0x2c0000401:0x12:0x0] updating lvb mtime from disk: 0 -> 1780298646 00002000:00010000:1.0:1780298646.446799:0:8362:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000401:0x12:0x0] updating lvb ctime from disk: 0 -> 1780298646 00002000:00010000:1.0:1780298646.446802:0:8362:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x12:0x0] updating lvb blocks from disk: 0 -> 1336 00010000:00010000:1.0:1780298646.446816:0:8362:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63036d9c lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x12:0x0].0x0 rrc: 4 type: EXT [8388608->9072639] (req 8388608->9072639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8362 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.446831:0:8362:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63036d9c lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x12:0x0].0x0 rrc: 4 type: EXT [8388608->9072639] (req 8388608->9072639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8362 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.446840:0:8362:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63036d9c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x12:0x0].0x0 rrc: 4 type: EXT [8388608->9072639] (req 8388608->9072639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8362 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.446854:0:8362:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf243278200 already unused, calling callback (ffffffffc0b15480) 00010000:00010000:1.0:1780298646.446859:0:8362:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63036d9c lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x12:0x0].0x0 rrc: 4 type: EXT [8388608->9072639] (req 8388608->9072639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8362 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.446869:0:8362:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63036d9c lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x12:0x0].0x0 rrc: 4 type: EXT [8388608->9072639] (req 8388608->9072639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8362 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.446887:0:8362:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63036d9c lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x12:0x0].0x0 rrc: 4 type: EXT [8388608->9072639] (req 8388608->9072639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8362 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1780298646.446895:0:8362:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b25cdf87/0xaf35132e63036d9c lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x12:0x0].0x0 rrc: 4 type: EXT [8388608->9072639] (req 8388608->9072639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8362 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1780298646.446981:0:8362:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+14:2399:x1866777609794176:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 7012us (7217us total) trans 4294969829 rc 0/0 00000100:00100000:1.0:1780298646.446995:0:8362:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6610 00000001:00080000:0.0:1780298646.464280:0:9421:0:(tgt_lastrcvd.c:988:tgt_cb_last_committed()) lustre-OST0001: transno 4294969831 is committed 00002000:00010000:2.0:1780298646.464559:0:8363:0:(ofd_lvb.c:330:ofd_lvbo_update()) res: [0x2c0000401:0x12:0x0] updating lvb blocks from disk: 1336 -> 17720 00010000:00010000:2.0:1780298646.464577:0:8363:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e63036d95 lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x12:0x0].0x0 rrc: 3 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8363 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.464587:0:8363:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e63036d95 lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x12:0x0].0x0 rrc: 3 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8363 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.464598:0:8363:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e63036d95 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x12:0x0].0x0 rrc: 4 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8363 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.464603:0:8363:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf344ff2000 already unused, calling callback (ffffffffc0b15480) 00010000:00010000:3.0:1780298646.464614:0:16972:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63036d8e lrc: 3/0,1 mode: PW/PW res: [0x2c0000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 16972 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.464627:0:16972:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63036d8e lrc: 2/0,0 mode: PW/PW res: [0x2c0000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 16972 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.464635:0:16972:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63036d8e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 16972 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.464638:0:8363:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e63036d95 lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x12:0x0].0x0 rrc: 4 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8363 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.464650:0:16972:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cf243d9ac00 already unused, calling callback (ffffffffc0b15480) 00010000:00010000:2.0:1780298646.464656:0:8363:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e63036d95 lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x12:0x0].0x0 rrc: 3 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8363 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.464659:0:16972:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63036d8e lrc: 3/0,0 mode: PW/PW res: [0x2c0000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 16972 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.464678:0:16972:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63036d8e lrc: 4/0,0 mode: PW/PW res: [0x2c0000401:0x12:0x0].0x0 rrc: 3 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 16972 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.464682:0:8363:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e63036d95 lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x12:0x0].0x0 rrc: 4 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8363 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.464688:0:16972:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63036d8e lrc: 2/0,0 mode: --/PW res: [0x2c0000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 16972 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1780298646.464694:0:8363:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c703984a/0xaf35132e63036d95 lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x12:0x0].0x0 rrc: 4 type: EXT [4194304->8388607] (req 4194304->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8363 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1780298646.464697:0:16972:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005fd24608/0xaf35132e63036d8e lrc: 0/0,0 mode: --/PW res: [0x2c0000401:0x12:0x0].0x0 rrc: 4 type: EXT [0->4194303] (req 0->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 16972 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1780298646.464814:0:16972:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf255493800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609793920:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 26637us (26845us total) trans 4294969830 rc 0/0 00000100:00100000:2.0:1780298646.464814:0:8363:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf2433ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+11:2399:x1866777609794048:12345-192.168.203.25@tcp:4:fsx.0 Request processed in 25596us (25716us total) trans 4294969831 rc 0/0 00000100:00100000:3.0:1780298646.464829:0:16972:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6608 00000100:00100000:2.0:1780298646.464836:0:8363:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 6609 00000100:00100000:0.0:1780298646.470996:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.471079:0:15026:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609794432 00000100:00100000:3.0:1780298646.471089:0:15026:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609794432 00000100:00100000:1.0:1780298646.471174:0:6479:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 135 00000100:00100000:1.0:1780298646.471190:0:6479:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243061500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+51:29360:x1866777609794432:12345-192.168.203.25@tcp:35:fsx.0 00000001:00080000:1.0:1780298646.471443:0:6479:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991140 00000100:00100000:1.0:1780298646.471566:0:6479:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243061500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+53:29360:x1866777609794432:12345-192.168.203.25@tcp:35:fsx.0 Request processed in 381us (579us total) trans 4294991141 rc 0/0 00000100:00100000:1.0:1780298646.471575:0:6479:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 135 00000020:00080000:1.0:1780298646.471581:0:6479:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1780298646.477559:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.477614:0:6480:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609794688 00000100:00100000:2.0:1780298646.477622:0:6480:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609794688 00000100:00100000:2.0:1780298646.477663:0:6480:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 136 00000100:00100000:2.0:1780298646.477672:0:6480:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf380ca1050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+53:29360:x1866777609794688:12345-192.168.203.25@tcp:35:fsx.0 00000001:00080000:2.0:1780298646.477847:0:6480:0:(tgt_lastrcvd.c:2129:tgt_txn_stop_cb()) More than one transaction 4294991142 00000100:00100000:2.0:1780298646.477923:0:6480:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf380ca1050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+55:29360:x1866777609794688:12345-192.168.203.25@tcp:35:fsx.0 Request processed in 253us (374us total) trans 4294991143 rc 0/0 00000100:00100000:2.0:1780298646.477930:0:6480:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 136 00000020:00080000:2.0:1780298646.477934:0:6480:0:(tgt_main.c:783:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1780298646.478775:0:3616:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298646.478828:0:6479:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609794944 00000100:00100000:3.0:1780298646.478834:0:6479:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609794944 00000100:00100000:3.0:1780298646.478868:0:6479:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 137 00000100:00100000:3.0:1780298646.478877:0:6479:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf241a81880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+56:29360:x1866777609794944:12345-192.168.203.25@tcp:35:fsx.0 00000100:00100000:3.0:1780298646.479020:0:6479:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf241a81880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+57:29360:x1866777609794944:12345-192.168.203.25@tcp:35:fsx.0 Request processed in 143us (243us total) trans 4294991144 rc 0/0 00000100:00100000:3.0:1780298646.479028:0:6479:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 137 00000100:00100000:0.0:1780298646.479794:0:3618:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298646.479837:0:6480:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609795200 00000100:00100000:2.0:1780298646.479842:0:6480:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609795200 00000100:00100000:2.0:1780298646.479869:0:6480:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 138 00000100:00100000:2.0:1780298646.479877:0:6480:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243060e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+55:29360:x1866777609795200:12345-192.168.203.25@tcp:35:fsx.0 00000100:00100000:2.0:1780298646.479954:0:6480:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243060e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+56:29360:x1866777609795200:12345-192.168.203.25@tcp:35:fsx.0 Request processed in 78us (159us total) trans 4294991145 rc 0/0 00000100:00100000:2.0:1780298646.479962:0:6480:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 138 00000001:02000400:3.0:1780298647.122885:0:21556:0:(debug.c:645:libcfs_debug_mark_buffer()) DEBUG MARKER: sanityn test_16b: @@@@@@ FAIL: fsx with tiny write failed. 00000100:00100000:0.0:1780298647.148992:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298647.149033:0:6471:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609795328 00000100:00100000:0.0:1780298647.149033:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298647.149044:0:6471:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609795328 00000100:00100000:0.0:1780298647.149069:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298647.149082:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609795456 00000100:00100000:3.0:1780298647.149087:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609795456 00000100:00100000:2.0:1780298647.149092:0:6471:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 578 00000100:00100000:2.0:1780298647.149107:0:6471:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf347c98c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4a8786ce-7c8a-40a7-8357-9ee004d06219+12:2398:x1866777609795328:12345-192.168.203.25@tcp:400:kworker.0 00000100:00100000:0.0:1780298647.149107:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298647.149153:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609795584 00000100:00100000:3.0:1780298647.149156:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609795584 00000100:00100000:0.0:1780298647.149162:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298647.149188:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1383 00000100:00100000:3.0:1780298647.149197:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243061500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+56:2397:x1866777609795456:12345-192.168.203.25@tcp:400:kworker.0 00000100:00100000:2.0:1780298647.149203:0:6471:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf347c98c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4a8786ce-7c8a-40a7-8357-9ee004d06219+12:2398:x1866777609795328:12345-192.168.203.25@tcp:400:kworker.0 Request processed in 99us (215us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298647.149208:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298647.149217:0:6471:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 578 00000100:00100000:3.0:1780298647.149238:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243061500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+56:2397:x1866777609795456:12345-192.168.203.25@tcp:400:kworker.0 Request processed in 43us (205us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298647.149244:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1383 00000100:00100000:2.0:1780298647.149271:0:6476:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609795968 00000100:00100000:2.0:1780298647.149275:0:6476:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609795968 00000100:00100000:3.0:1780298647.149279:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1384 00000100:00100000:3.0:1780298647.149290:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf243060000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609795584:12345-192.168.203.25@tcp:400:kworker.0 00000100:00100000:0.0:1780298647.149298:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298647.149304:0:6476:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1385 00000100:00100000:2.0:1780298647.149312:0:6476:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b41c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+57:2397:x1866777609795968:12345-192.168.203.25@tcp:400:kworker.0 00000100:00100000:3.0:1780298647.149330:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf243060000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2398:x1866777609795584:12345-192.168.203.25@tcp:400:kworker.0 Request processed in 43us (262us total) trans 0 rc 0/0 00000100:00100000:3.0:1780298647.149336:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1384 00000100:00100000:2.0:1780298647.149343:0:6476:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b41c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+57:2397:x1866777609795968:12345-192.168.203.25@tcp:400:kworker.0 Request processed in 32us (138us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298647.149349:0:6476:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1385 00000100:00100000:3.0:1780298647.149350:0:6477:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609796096 00000100:00100000:3.0:1780298647.149354:0:6477:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609796096 00000100:00100000:3.0:1780298647.149393:0:6477:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 1386 00000100:00100000:0.0:1780298647.149395:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:3.0:1780298647.149402:0:6477:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b43800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+5:2398:x1866777609796096:12345-192.168.203.25@tcp:400:kworker.0 00000100:00100000:0.0:1780298647.149436:0:3617:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.25@tcp (source: 12345-192.168.203.25@tcp) 00000100:00100000:2.0:1780298647.149476:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609795712 00000100:00100000:2.0:1780298647.149481:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609795712 00000100:00100000:0.0:1780298647.149494:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609795840 00000100:00100000:0.0:1780298647.149498:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609795840 00000100:00100000:2.0:1780298647.149512:0:8356:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609796224 00000100:00100000:2.0:1780298647.149515:0:8356:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609796224 00000100:00100000:0.0:1780298647.149531:0:12101:0:(service.c:2280:ptlrpc_server_handle_req_in()) unwrap req x1866777609796352 00000100:00100000:0.0:1780298647.149535:0:12101:0:(service.c:2345:ptlrpc_server_handle_req_in()) got req x1866777609796352 00000100:00100000:2.0:1780298647.149544:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 865 00000100:00100000:2.0:1780298647.149553:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b41880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609795712:12345-192.168.203.25@tcp:400:kworker.0 00000100:00100000:3.0:1780298647.149564:0:6477:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+5:2398:x1866777609796096:12345-192.168.203.25@tcp:400:kworker.0 Request processed in 100us (205us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298647.149568:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 866 00000100:00100000:3.0:1780298647.149576:0:6477:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 1386 00000100:00100000:0.0:1780298647.149582:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609795840:12345-192.168.203.25@tcp:400:kworker.0 00000100:00100000:2.0:1780298647.149583:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b41880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+9:2397:x1866777609795712:12345-192.168.203.25@tcp:400:kworker.0 Request processed in 32us (480us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298647.149593:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 865 00000100:00100000:2.0:1780298647.149608:0:8356:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 867 00000100:00100000:2.0:1780298647.149616:0:8356:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609796224:12345-192.168.203.25@tcp:400:kworker.0 00000100:00100000:0.0:1780298647.149620:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:e30cf99f-4c41-4c6b-8649-e9856bf0abcc+10:2398:x1866777609795840:12345-192.168.203.25@tcp:400:kworker.0 Request processed in 41us (459us total) trans 0 rc 0/0 00000100:00080000:3.0:1780298647.149626:0:6473:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1780298647.149630:0:6473:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1780298647.149630:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 866 00000100:00100000:2.0:1780298647.149641:0:8356:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2397:x1866777609796224:12345-192.168.203.25@tcp:400:kworker.0 Request processed in 27us (251us total) trans 0 rc 0/0 00000100:00100000:2.0:1780298647.149647:0:8356:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 867 00000100:00100000:0.0:1780298647.149653:0:12101:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.25@tcp, seq: 868 00000100:00100000:0.0:1780298647.149663:0:12101:0:(service.c:2531:ptlrpc_server_handle_request()) Handling RPC req@ffff9cf242b40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609796352:12345-192.168.203.25@tcp:400:kworker.0 00000100:00100000:0.0:1780298647.149698:0:12101:0:(service.c:2589:ptlrpc_server_handle_request()) Handled RPC req@ffff9cf242b40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:a9f27b46-1e58-4b3a-bc39-449ecb1af8ca+8:2398:x1866777609796352:12345-192.168.203.25@tcp:400:kworker.0 Request processed in 38us (264us total) trans 0 rc 0/0 00000100:00100000:0.0:1780298647.149705:0:12101:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.25@tcp, seq: 868 00010000:00010000:1.0:1780298647.331362:0:7850:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) Debug log: 69048 lines, 69048 kept, 0 dropped, 0 bad.